MikroTik – ROSv7 – VPLS over IPv6 MPLS with LDPv6


The great migration

IPv6 adoption has really picked up in the last 12 months and MikroTik RouterOSv7 development is no exception. Dual stack networks are still the most common and easiest to initially deploy for carriers.

However, single stack networks with IPv4 as a service overlay are definitely on the horizon for MikroTik users now that MPLS can operate purely on IPv6.

Single stack networks are easier and cheaper to operate in the long run and are a natural evolution of dual stack networks as we begin to turn IPv4 off for underlay infrastructure.

LDPv6

There are a few different ways to distribute labels in IPv6 MPLS. SR-MPLS (less common and usually with IS-IS) and SRv6 are the other options besides LDPv6.

While I generally am in favor of SR-MPLS/SRv6 long term due to the protocol simplification and traffic management capabilities, having an IPv6 MPLS stack is a great starting point for MikroTik.

LDPv6 is defined by RFC 7552 and is fairly recent as it finalized in 2015. It generally operates in much the same way as LDPv4.


Use Cases

The most common use case among MikroTik users is more efficient subnetting of IPv4 and directly replaces LDPv4 for this task.

This is the use case tested in this lab by putting 203.0.113.0/24 onto a VPLS interface as an overlay transported by LDPv6.

Other use cases include: Private L2 transport, IPv6 overlay (common when used with v4 overlay to the same subscribers) and IPTV transport to simplify multicast requirements.


Lab overview

Because RouterOSv7 is so new and has hardware dependencies, I’ve opted to do a lot of the testing on actual MikroTik equipment instead of just the CHR VM inside of EVE-NG as I would normally do when testing/labbing protocols.

Hardware Offload – Officially, IPv6 HW offload isn’t listed as “done”, but some of the elements are there in the command output and the ‘H’ flag for hardware offload can be seen in the IPv4 and IPv6 routing tables.

Example of IPv4 hw offloaded routes on the CCR2116


Example of IPv6 hw offloaded routes on the CCR2116 – still not officially supported



Lab topology

The physical lab consists of a CCR2116 as an LSR and two CCR2004s acting as LERs. They are part of the larger 100G development lab so the relevant part has been highlighted on the physical drawing.


https://iparchitechs.com/contact


Lab testing

Here is an overview of the config and verification of LDPv6 operation with an IPv4 ping test between the two LER routers.


LER-01 (CCR2004-16G-2S+)

verification:


config:

# may/15/2022 12:22:03 by RouterOS 7.3beta40
# model = CCR2004-16G-2S+
/interface bridge
add name=br-vpls-101-ipv6
add name=lo-ipv6
/interface vpls
add disabled=no mac-address=02:DA:E8:D2:E5:83 name=vpls-101-ipv6 peer=200:127:1::2 pw-type=raw-ethernet vpls-id=101:6
/interface bonding
add mode=802.3ad name=bonding1 slaves=sfp-sfpplus1,sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
/interface vlan
add interface=bonding1 name=vlan3100 vlan-id=3100
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=br-vpls-101-ipv6 interface=vpls-101-ipv6
/ip address
add address=203.0.113.1/24 interface=br-vpls-101-ipv6 network=203.0.113.0
/ipv6 address
add address=200:126:1::1 advertise=no interface=vlan3100
add address=200:127:1::1/128 advertise=no interface=lo-ipv6
add address=203:0:113::1 advertise=no interface=br-vpls-101-ipv6
/mpls interface
add disabled=no input=yes interface=vlan3100 mpls-mtu=1530
add disabled=no input=yes interface=vpls-101-ipv6
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.1 transport-addresses=200:127:1::1 vrf=main
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3100 transport-addresses=200:127:1::1
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3100 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
/system identity
set name=ccr2004-2splus-01.test.lab.ipa

LER-02 (CCR2004-16G-2S+)

verification:


config:

# may/15/2022 14:15:04 by RouterOS 7.3beta40
# model = CCR2004-16G-2S+
/interface bridge
add name=br-vpls-101
add name=lo-ipv6
/interface vpls
add disabled=no mac-address=02:87:6F:98:EE:32 name=vpls-101-ipv6 peer=200:127:1::1 pw-type=raw-ethernet vpls-id=101:6
/interface bonding
add mode=802.3ad name=bonding1 slaves=sfp-sfpplus1,sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
/interface vlan
add interface=bonding1 name=vlan3200 vlan-id=3200
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=br-vpls-101 interface=vpls-101-ipv6
/ip address
add address=203.0.113.2/24 interface=br-vpls-101 network=203.0.113.0
add address=100.126.1.9/29 interface=vlan3200
/ipv6 address
add address=200:127:1::2/128 advertise=no interface=lo-ipv6
add address=200:126:2::2 advertise=no interface=vlan3200
add address=203:0:113::2 interface=br-vpls-101
/mpls interface
add disabled=no input=yes interface=vlan3200 mpls-mtu=1530
add disabled=no input=yes interface=vpls-101-ipv6
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.2 transport-addresses=200:127:1::2
/mpls ldp interface
add afi=ipv6 disabled=no interface=vlan3200 transport-addresses=200:127:1::2
add afi=ip disabled=yes interface=vlan3200 transport-addresses=100.127.1.2
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3200 type=ptp
/system identity
set name=ccr2004-2splus-02.test.lab.ipa

LSR-01 (CCR2116-12G-4S+)

verification:

config:

# may/15/2022 11:54:07 by RouterOS 7.3beta40
# model = CCR2116-12G-4S+

/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge-main vlan-filtering=yes
add name=lo-ipv6
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] auto-negotiation=no speed=10Gbps
set [ find default-name=sfp-sfpplus2 ] auto-negotiation=no speed=10Gbps
/interface vlan
add interface=bridge-main name=vlan3100 vlan-id=3100
add interface=bridge-main name=vlan3200 vlan-id=3200
/interface bonding
add mode=802.3ad name=bonding1-40g-crs326-01 slaves=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-3-and-4
/interface ethernet switch
set 0 l3-hw-offloading=yes
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=bridge-main interface=bonding1-40g-crs326-01
/interface bridge vlan
add bridge=bridge-main tagged=bonding1-40g-crs326-01,bridge-main vlan-ids=3100
add bridge=bridge-main tagged=bonding1-40g-crs326-01,bridge-main vlan-ids=3200
/ipv6 address
add address=200:126:1::3 advertise=no interface=vlan3100
add address=200:127:1::3/128 advertise=no interface=lo-ipv6
add address=200:126:2::3 advertise=no interface=vlan3200
/mpls interface
add disabled=no input=yes interface=vlan3100 mpls-mtu=1530
add disabled=no input=yes interface=vlan3200 mpls-mtu=1530
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.3 transport-addresses=200:127:1::3 vrf=main
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3100 transport-addresses=200:127:1::3
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3200 transport-addresses=200:127:1::3
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3100 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3200 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
/system identity
set name=ccr2116-01.test.lab.ipa.net


“Why use 200::/7 for labbing instead of 2001:db8::/32?”

IPv6 is long overdue for a dedicated GUA labbing space. I’ve been working with Ed Horley (@ehorley) and several others on a dedicated range for labbing that allows for modeling not just within a /32 but beyond a /32

draft-horley-v6ops-lab-02 – Expanding the IPv6 Lab Use Space (ietf.org)

Networking Field Day: Service Provider – Arista VPLS – UPDATED

During the first networking field day: service provider one of the big topics was EVPN versus VPLS. Arista has put a lot of work into their EVPN deployment and this has give then a ton of success in the data center. However, a large portion of the provider space, especially last mile providers, rely on VPLS heavily. This naturally led to discussion about Arista VPLS support.

I’m pleased to see that there is now basic support in EOS as of EOS 4.27.2F and more on the roadmap. Hopefully, we’ll see the off ramp, RFC8560, from VPLS to EVPN which was a hot button topic throughout the week.

In the release notes for EOS 4.27.2F it calls our basic VPLS support. So I took a look. Reviewing the new 4.27.2F manual I found support for LDP PWs on RFC4447 which is virtual private wire support. This also appeared to be in EOS 4.26 but not earlier. Thanks to Arista for providing more documentation on their support for RFC4762 – LDP signaled VPLS.

In the meantime lets review how this works:

mpls ip
!
mpls ldp
   router-id 100.127.0.3
   transport-address interface Loopback0
   no shutdown
   !
   pseudowires
      pseudowire TEST-PW
         neighbor 100.127.0.1
         pseudowire-id 1
         mtu 1500
!
patch panel
   patch TEST
   !
   patch TEST-PW-PATCH
      connector 1 pseudowire ldp TEST-PW
      connector 2 interface Ethernet3
!

You have to define the end point for the LDP signaling in the LDP configuration. The configuration requires an endpoint (neighbor), pseudowire-id, and mtu. Without all three of these the PW won’t establish.

Then tie the port you want to use the PW with a patch panel connector. In this case we tied ethernet3 to PW TEST-PW.

Everything that comes in on Ethernet3 will be pushed into the PW and on to the endpoint. Let’s verify that the signaling mechanism works:

arista-11#show patch panel detail
PW Fault Legend:
   ET-IN - Ethernet receive fault
   ET-OUT - Ethernet transmit fault
   TUN-IN - Tunnel receive fault
   TUN-OUT - Tunnel transmit fault
   NF - Pseudowire not forwarding (other reason)

Patch: TEST, Status: Down, Last change: 0:26:17 ago
Patch: TEST-PW-PATCH, Status: Up, Last change: 16:35:05 ago
   Connector 1: LDP neighbor 100.127.0.1 PW ID 1
      Status: Up
      Local MPLS label: 116384, Group ID: 0x0
         MTU: 1500, 802.1Q VLAN request sent: -
         Flow label capability: none
         Supported VCCV CV types: LSP ping
         Supported VCCV CC types: Router alert label
      Neighbor MPLS label: 116384, Group ID: 0x0
         MTU: 1500, 802.1Q VLAN request received: -
         Flow label capability: none
         Supported VCCV CV types: LSP ping
         Supported VCCV CC types: Router alert label
      PW type: 5 (raw), Control word: N
      Flow label used: no
      Tunnel type: LDP, Tunnel index: 1
   Connector 2: Ethernet3
      Status: Up
CE-1#ping 172.16.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 14/18/22 ms

Now we have a functional layer 2 link between the distance CEs.

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg
iparchitechs.com/contact

An important note if you want to put this into production is you have to use service routing protocols model multi-agent which requires a reboot of your devices.

There are also some restrictions in vlan translation/passing which I will explore in a future post. Now let’s check out the basic configuration.

After reviewing the documents for LDP signaled VPLS we built the topology above. All 3 PEs are in the same mesh so the 3 CE routers are all layer 2 adjacent.

I probably made every mistake you could as I started building this but the CLI is pretty helpful in what is wrong.

arista-11#show vpls
VPLS: TEST-VPLS
VLAN: 10, 802.1Q tag: -
MAC withdrawal trigger for local interface going down: Y
Pseudowire group: MESH, split-horizon
MAC withdrawal trigger on pseudowire failure: N
MAC withdrawal propagation: locally triggered
LDP neighbor 100.127.0.1 PW ID 1 PW name ARISTA-10
   Status: No remote, Interface: Pseudowire3.0
LDP neighbor 100.127.0.4 PW ID 1 PW name ARISTA-13
   Status: CLI incomplete

I originally missed specifying the MTU on the CLI so it told me my configuration was incomplete. I thought this was pretty neat as it prevented me from going down a bunch of different paths to determine why my original build was broken.

arista-13
mpls ldp
   router-id 100.127.0.4
   transport-address interface Loopback0
   no shutdown
   !
   pseudowires
      pseudowire ARISTA-10
         neighbor 100.127.0.1
         pseudowire-id 1
         mtu 1500
      !
      pseudowire ARISTA-11
         neighbor 100.127.0.3
         pseudowire-id 1
         mtu 1500
!
router vpls
   vpls TEST-VPLS
      ldp mac withdrawal trigger interface
      !
      vlan 10
      !
      pseudowire ldp MESH split-horizon
         pseudowire ARISTA-10
         pseudowire ARISTA-11
!
end
arista-13#show vpls
VPLS: TEST-VPLS
VLAN: 10, 802.1Q tag: -
MAC withdrawal trigger for local interface going down: Y
Pseudowire group: MESH, split-horizon
MAC withdrawal trigger on pseudowire failure: N
MAC withdrawal propagation: locally triggered
LDP neighbor 100.127.0.1 PW ID 1 PW name ARISTA-10
   Status: Up, Interface: Pseudowire1.0
LDP neighbor 100.127.0.3 PW ID 1 PW name ARISTA-11
   Status: Up, Interface: Pseudowire2.0
CE-3#ping 172.16.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.0.2, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 11/14/17 ms
CE-3#ping 172.16.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.0.1, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 11/12/14 ms
CE-3#

If you need help with your deployment reach out to us at IP architechs.