Category: IPv6

MikroTik – ROSv7 – VPLS over IPv6 MPLS with LDPv6

Kevin Myers 4 Comments


The great migration

IPv6 adoption has really picked up in the last 12 months and MikroTik RouterOSv7 development is no exception. Dual stack networks are still the most common and easiest to initially deploy for carriers.

However, single stack networks with IPv4 as a service overlay are definitely on the horizon for MikroTik users now that MPLS can operate purely on IPv6.

Single stack networks are easier and cheaper to operate in the long run and are a natural evolution of dual stack networks as we begin to turn IPv4 off for underlay infrastructure.

LDPv6

There are a few different ways to distribute labels in IPv6 MPLS. SR-MPLS (less common and usually with IS-IS) and SRv6 are the other options besides LDPv6.

While I generally am in favor of SR-MPLS/SRv6 long term due to the protocol simplification and traffic management capabilities, having an IPv6 MPLS stack is a great starting point for MikroTik.

LDPv6 is defined by RFC 7552 and is fairly recent as it finalized in 2015. It generally operates in much the same way as LDPv4.


Use Cases

The most common use case among MikroTik users is more efficient subnetting of IPv4 and directly replaces LDPv4 for this task.

This is the use case tested in this lab by putting 203.0.113.0/24 onto a VPLS interface as an overlay transported by LDPv6.

Other use cases include: Private L2 transport, IPv6 overlay (common when used with v4 overlay to the same subscribers) and IPTV transport to simplify multicast requirements.


Lab overview

Because RouterOSv7 is so new and has hardware dependencies, I’ve opted to do a lot of the testing on actual MikroTik equipment instead of just the CHR VM inside of EVE-NG as I would normally do when testing/labbing protocols.

Hardware Offload – Officially, IPv6 HW offload isn’t listed as “done”, but some of the elements are there in the command output and the ‘H’ flag for hardware offload can be seen in the IPv4 and IPv6 routing tables.

Example of IPv4 hw offloaded routes on the CCR2116


Example of IPv6 hw offloaded routes on the CCR2116 – still not officially supported



Lab topology

The physical lab consists of a CCR2116 as an LSR and two CCR2004s acting as LERs. They are part of the larger 100G development lab so the relevant part has been highlighted on the physical drawing.



Lab testing

Here is an overview of the config and verification of LDPv6 operation with an IPv4 ping test between the two LER routers.


LER-01 (CCR2004-16G-2S+)

verification:


config:

# may/15/2022 12:22:03 by RouterOS 7.3beta40
# model = CCR2004-16G-2S+
/interface bridge
add name=br-vpls-101-ipv6
add name=lo-ipv6
/interface vpls
add disabled=no mac-address=02:DA:E8:D2:E5:83 name=vpls-101-ipv6 peer=200:127:1::2 pw-type=raw-ethernet vpls-id=101:6
/interface bonding
add mode=802.3ad name=bonding1 slaves=sfp-sfpplus1,sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
/interface vlan
add interface=bonding1 name=vlan3100 vlan-id=3100
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=br-vpls-101-ipv6 interface=vpls-101-ipv6
/ip address
add address=203.0.113.1/24 interface=br-vpls-101-ipv6 network=203.0.113.0
/ipv6 address
add address=200:126:1::1 advertise=no interface=vlan3100
add address=200:127:1::1/128 advertise=no interface=lo-ipv6
add address=203:0:113::1 advertise=no interface=br-vpls-101-ipv6
/mpls interface
add disabled=no input=yes interface=vlan3100 mpls-mtu=1530
add disabled=no input=yes interface=vpls-101-ipv6
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.1 transport-addresses=200:127:1::1 vrf=main
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3100 transport-addresses=200:127:1::1
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3100 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
/system identity
set name=ccr2004-2splus-01.test.lab.ipa

LER-02 (CCR2004-16G-2S+)

verification:


config:

# may/15/2022 14:15:04 by RouterOS 7.3beta40
# model = CCR2004-16G-2S+
/interface bridge
add name=br-vpls-101
add name=lo-ipv6
/interface vpls
add disabled=no mac-address=02:87:6F:98:EE:32 name=vpls-101-ipv6 peer=200:127:1::1 pw-type=raw-ethernet vpls-id=101:6
/interface bonding
add mode=802.3ad name=bonding1 slaves=sfp-sfpplus1,sfp-sfpplus2 transmit-hash-policy=layer-3-and-4
/interface vlan
add interface=bonding1 name=vlan3200 vlan-id=3200
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=br-vpls-101 interface=vpls-101-ipv6
/ip address
add address=203.0.113.2/24 interface=br-vpls-101 network=203.0.113.0
add address=100.126.1.9/29 interface=vlan3200
/ipv6 address
add address=200:127:1::2/128 advertise=no interface=lo-ipv6
add address=200:126:2::2 advertise=no interface=vlan3200
add address=203:0:113::2 interface=br-vpls-101
/mpls interface
add disabled=no input=yes interface=vlan3200 mpls-mtu=1530
add disabled=no input=yes interface=vpls-101-ipv6
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.2 transport-addresses=200:127:1::2
/mpls ldp interface
add afi=ipv6 disabled=no interface=vlan3200 transport-addresses=200:127:1::2
add afi=ip disabled=yes interface=vlan3200 transport-addresses=100.127.1.2
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3200 type=ptp
/system identity
set name=ccr2004-2splus-02.test.lab.ipa

LSR-01 (CCR2116-12G-4S+)

verification:

config:

# may/15/2022 11:54:07 by RouterOS 7.3beta40
# model = CCR2116-12G-4S+

/interface bridge
add frame-types=admit-only-vlan-tagged name=bridge-main vlan-filtering=yes
add name=lo-ipv6
/interface ethernet
set [ find default-name=sfp-sfpplus1 ] auto-negotiation=no speed=10Gbps
set [ find default-name=sfp-sfpplus2 ] auto-negotiation=no speed=10Gbps
/interface vlan
add interface=bridge-main name=vlan3100 vlan-id=3100
add interface=bridge-main name=vlan3200 vlan-id=3200
/interface bonding
add mode=802.3ad name=bonding1-40g-crs326-01 slaves=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-3-and-4
/interface ethernet switch
set 0 l3-hw-offloading=yes
/routing ospf instance
add disabled=no name=ospf-instance-v3-ipv6 version=3
/routing ospf area
add disabled=no instance=ospf-instance-v3-ipv6 name=ospf-area-0-ipv6
/interface bridge port
add bridge=bridge-main interface=bonding1-40g-crs326-01
/interface bridge vlan
add bridge=bridge-main tagged=bonding1-40g-crs326-01,bridge-main vlan-ids=3100
add bridge=bridge-main tagged=bonding1-40g-crs326-01,bridge-main vlan-ids=3200
/ipv6 address
add address=200:126:1::3 advertise=no interface=vlan3100
add address=200:127:1::3/128 advertise=no interface=lo-ipv6
add address=200:126:2::3 advertise=no interface=vlan3200
/mpls interface
add disabled=no input=yes interface=vlan3100 mpls-mtu=1530
add disabled=no input=yes interface=vlan3200 mpls-mtu=1530
/mpls ldp
add afi=ipv6 disabled=no lsr-id=100.127.1.3 transport-addresses=200:127:1::3 vrf=main
/mpls ldp interface
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3100 transport-addresses=200:127:1::3
add accept-dynamic-neighbors=yes afi=ipv6 disabled=no interface=vlan3200 transport-addresses=200:127:1::3
/routing ospf interface-template
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3100 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=vlan3200 type=ptp
add area=ospf-area-0-ipv6 disabled=no interfaces=lo-ipv6 passive
/system identity
set name=ccr2116-01.test.lab.ipa.net


“Why use 200::/7 for labbing instead of 2001:db8::/32?”

IPv6 is long overdue for a dedicated GUA labbing space. I’ve been working with Ed Horley (@ehorley) and several others on a dedicated range for labbing that allows for modeling not just within a /32 but beyond a /32

draft-horley-v6ops-lab-02 – Expanding the IPv6 Lab Use Space (ietf.org)

MikroTik – RouterOSv7 first look – feedback on routing filters

Kevin Myers 4 Comments

Overview

Routing filters have been a hot topic lately in the world of RouterOSv7. The first implementation of routing filters in ROSv7 was difficult to work with and documented in the two articles below:

MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP

MikroTik RouterOS – v7.0.3 stable (chateau) and status of general release


MikroTik then made some changes and opened up discussion to get feedback. I did a lot of work and testing using ROS 7.1beta7 which never made it to public release and was close to publishing the results when 7.1rc1 came out so this post will use that version.

https://forum.mikrotik.com/viewtopic.php?f=1&t=175201#p867750


RouterOSv7.1rc1 syntax example

Here is an example of the latest syntax in ROSv7.1rc1

CLI

### MikroTik RouterOS 7.1rc1 ###

/routing filter rule
add chain=dead.beef.101 rule="if (dst==200:dead:beef:101::/64) {accept}"
add chain=dead.beef.102 rule="if (dst==200:dead:beef:102::/64) {accept}"
add chain=dead.beef.agg rule="if (dst in 200:dead:beef::/48) {accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.101) {set bgp-local-pref 300; accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.102) {accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.agg && dst-len<128) {set bgp-local-pref 150; accept}"

Winbox


And the corresponding routes received (for the v7 filter rules)

Comparable syntax in ROSv6 (note: recursive routing is not supported in IPv6 so the iBGP routes used to test v7rc1 would not be possible)

### MikroTik RouterOS 6.48.10 (long-term) ###

/routing filter
add action=accept chain=dead.beef.101 prefix=200:dead:beef:101::/64
add action=accept chain=dead.beef.102 prefix=200:dead:beef:102::/64
add action=accept chain=dead.beef.agg prefix=200:dead:beef::/48
add action=accept chain=bgp-out-v6 match-chain=dead.beef.101 set-bgp-local-pref=300
add action=accept chain=bgp-out-v6 match-chain=dead.beef.102 
add action=accept chain=bgp-out-v6 match-chain=dead.beef.agg prefix-length=48-128 set-bgp-local-pref=150



Challenges and Feedback

  • Coding vs. Network CLI – The single biggest resistance to the new style of filtering is the string format is hard to work with. It requires knowledge of the match and action statements as well as how to write an expression to correctly parse them. Network engineers are not software engineers and route filtering (much like firewalling) has the capacity to be complex and require many lines of config – which means the format should align with v6 route filters and/or firewall rules.
  • Tab complete – The lack of tab complete is a big gap. Network engineers, admins and technicians expect to be able to tab complete when creating a config. This is rumored to be in the works and hopefully it will make it into later release candidates.

  • Using context sensitive help with “?” – There are a couple issues with context sensitive help in ROS 7.1rc1. The first issue is using the F1 key for help in any part of ROSv7 (not just filtering) instead of the ‘?’. This should at least be an option that can be set.

    The second issue is the lack of context sensitive help for the routing filters – if an engineer is unsure of the syntax, it’s currently not possible to get help from the command line.

    This has been a fundamental part of CLI based network operating systems for over 30 years. It needs to be added back.


    Conclusions

    One thing is clear, everyone I discussed it with on Facebook, Reddit, MikroTik Forums and with clients and engineers on my team did not like the new format.

    It’s worth noting that MikroTik equipment is often used in remote locations where it isn’t practical to pull up the help docs and engineers in the field rely on the ability to use tab-complete and context sensitive help to finish configuration tasks. This is a critical feature for a network operating system to have.

    To MikroTik – please consider implementing the filters so that at a minimum, the features in ROSv6 (like tab-complete, context sensitive help and a non-coding syntax) are maintained while allowing for new functionality.


Examples of filtering in other well-known operating systems

Here are some examples of the same filtering rules in different network operating systems for comparison. All of them support tab-complete and context sensitive help.

Free Range Routing is probably one of my favorites because it’s open and is being actively developed. JunOS is very popular from a filtering standpoint because the OS is easy to work with programatically.

Cisco is included because they are pervasive but isn’t at the top of my list because the syntax isn’t anything special and IOS-XR equipment tends to be incredibly expensive even though bugs are still commonplace – so it’s not a great value.

FRRouting · GitHub

Free Range Routing (https://frrouting.org)

### Free Range Routing v6.0.2 ###

ipv6 prefix-list dead.beef.101 seq 10 permit 200:dead:beef:101::/64
ipv6 prefix-list dead.beef.102 seq 10 permit 200:dead:beef:102::/64
ipv6 prefix-list dead.beef.agg seq 10 permit 200:dead:beef::/48 le 128
!
route-map bgp-out-v6 permit 10
 match ipv6 address prefix-list dead.beef.101
 set local-preference 300
!
route-map bgp-out-v6 permit 20
 match ipv6 address prefix-list dead.beef.102
!
route-map bgp-out-v6 permit 30
 match ipv6 address prefix-list dead.beef.agg
 set local-preference 150
### Free Range Routing v6.x ###!

File:Juniper Networks logo.svg - Wikimedia Commons

Juniper Networks JunOS (https://junipernetworks.com)

### Juniper JunOS 18.x ###

policy-options {
    prefix-list dead.beef.101 {
        200:dead:beef:101::/64;
    }
    prefix-list dead.beef.102 {
        200:dead:beef:102::/64;
    }
    prefix-list dead.beef.agg {
        200:dead:beef::/48;
    }
    policy-statement bgp-out-v6 {
        term dead.beef.101 {
            from {
                prefix-list dead.beef.101;
            }
            then {
                local-preference 300;
                accept;
            }
        }
        term dead.beef.102 {
            from {
                prefix-list dead.beef.102;
            }
            then accept;
        }
        term dead.beef.agg {
            from {
                prefix-list-filter dead.beef.agg orlonger;
            }
            then {
                local-preference 150;
                accept;
            }
        }
    }
}

Cisco Systems - Wikipedia

Cisco Networks IOS XR (https://cisco.com)

### Cisco IOS-XR x.x ###

prefix-set dead.beef.101
  200:dead:beef:101::/64
end-set
!
prefix-set dead.beef.102
  200:dead:beef:102::/64
end-set
!
prefix-set dead.beef.agg
  200:dead:beef::/48 le 128
end-set
!
route-policy bgp-out-v6
  if destination in dead.beef.101 then
    set local-preference 300
    pass
  elseif destination in dead.beef.102 then
    pass
  elseif destination in dead.beef.agg then
    set local-preference 150
    pass
  endif
end-policy