Let’s talk MPLS-VPLS, part 1 – use cases

Hey guys, today we’ll be taking a look at what are the main reasons that MPLS – and in particular VPLS – is a useful technology tool to get traffic from point A to point Z. There’s really 2 major use cases that I deal with regularly: IPv4 conservation and L2VPN.

Why am I talking about VPLS specifically? Well, mostly because many times I end up working with Mikrotik routers, which only support VPLS. What is VPLS? It stands for Virtual Private Line Service, and it’s a way to deliver layer 2 services over a layer 3 network. Said another way, it connects a single broadcast domain to multiple endpoints across a routed network. I’ll discuss why MPLS is better for you and your network than switching/bridging in part 2 of this series – for now just know that MPLS/VPLS will allow you to offer enhanced services without the risks of extending layer 2 (I’ll talk more about that below, and why that’s bad in part 2, also).


Use case 1: IPv4 Conservation

OK, so let’s visualize the problem with IP conservation on a small /24 allocation.

Subnetted /24 network

If you’re like most other service providers, you have a limited capacity of IPv4 and constantly using subnetting to carve out IP space for various locations. Inevitably, you end up needing a bigger network somewhere (like A) and in other places you’ve allocated too many IPs and they’re going wasted and unused (like B). Not to mention, with subnetting we’re forced to burn a network ID and broadcast each time we pull out the network scalpel – ouch!

As you can see, in this network, subnetting has resulted in 16 IPs used for IDs and subnets, 8 more for gateways, and over in area B let’s say there’s 37 unassigned IPs – that’s 60 IPs out of 254 that customers can’t use, almost 25% of this expensive revenue generating resource is tied up!

Here’s a solution based on MPLS/VPLS that efficiently leverages the entire /24 allocation:

/24 used efficiently with VPLS

Now we’re only using 1 network ID, 1 broadcast, and 1 gateway for our /24. The entire rest of the allocation can be delivered to any endpoint that’s connected the MPLS cloud – leaving 253 addresses available for customer assignments! And growth can happen at any tower – we no longer have to care which customers are on which tower, and how much IP space is available at the tower layer.

Compared to the above scenario, our IP allocation overhead went down from 25% to just over 1%! Now that’s efficient use of IPv4 space.



https://iparchitechs.com/contact


Use case 2: L2VPN

Connecting multiple offices across a service provider network can often be a very lucrative way to offer transparent LAN service to customers. The reason we call it transparent is because it allows whatever customer traffic is input in one end of the network to be seen at another end of the network. In a typical routed network, BUM traffic (which I’ll cover more in part 2) gets isolated at layer 3, routed boundaries. With VPLS, your network gets the full benefits of the layer 3 boundaries, and your customer gets the connectivity benefits of being connected layer 2!

Some of the more common customers looking for a L2VPN service include government, energy and banks. But really, any customer who’d like to connect the same subnet at several different physical locations that are served by different equipment across your network are a great candidates. Sometimes they’d like to pass their own VLAN traffic, other times they need multicast for a video conferencing application, or even something as simple as a small network with only 1 DHCP server / firewall / internet gateway and they want to force all traffic through that central point. Regardless of the reason, this premium service can bring in great revenue.

Can’t I just provide L2 service by extending a VLAN? Absolutely, you can. Should I just extend my bridged/switched network to provide this service? NO! You should absolutely avoid this as much as possible! As we’ll see in part 2, isolating fault domains and adding L3 boundaries are key to protecting your network from bad days.

Let’s see an example:

Don’t bridge to customers

Here, we’ve extended the network using bridging and switching (L2) constructs to connect customer locations A B and C. When we have a bad day on this network, the scope of our network meltdown includes the entirety of ALL the gear and ALL our customers. Ouch!

Whether it’s a customer introduced loop, infected customer device, or a customer host spewing garbage onto to their network, there’s no protection between the customer edge (CE) and provider edge (PE) and no boundaries to stop that bad traffic.

Fortunately, there’s a better way:

Friends don’t let friends bridge networks

With MPLS/VPLS, you can see that the scope of our outage is isolated to the customer edge (the port handoff to A, B, and C), or at worst only the VPLS router endpoints, depending on the gear, QoS policies, and configuration. Now, a bad day on the customer’s network isn’t necessarily a bad day for our service provider network.


I hope you enjoyed this article and keep an eye out for part 2 where I’ll talk more in depth about L2 (bridging/switching) vs L3 (routing) and why MPLS/VPLS is referred to as layer 2.5.

MikroTik RouterOS – v7.0.3 stable (chateau) and status of general release

If you don’t already use it, the MIkroTik v7 BETA forum (forum.mikrotik.com) is a fantastic source of information


When will stable be released?

This is the million dollar question. Technically, it already has been for one hardware platform…

!! Spoiler Alert – There is *already* a stable release of ROSv7 – v7.0.3!!

The Chateau 5G router originally shipped with a beta version of ROSv7 but was quietly moved to a stable version that’s developed specifically for that platform.

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

Because of the way MikroTik’s code repo works, this version can’t easily be added to the main download page and support provides the software:

ROSv7.0.3 Stable Download (!!! Chateau Only – will brick other hardware !!!)

https://box.mikrotik.com/f/7e3cad5779804d0b878d/?dl=1

It’s worth repeating MikroTik’s warning about using this on any platform other than the Chateau

v7 launch date – MikroTikhttps://forum.mikrotik.com/viewtopic.php?f=1&t=175201#p865452


https://iparchitechs.com/contact



What’s holding up v7 from being released?

If you’ve been around MikroTik for a while, then you know that version 7 has been in the works for a long time to add new functionality and address limitations of the older Linux kernel in ROSv6.

MikroTik recently added a detailed update on where the development roadmap is at and what the challenges are:

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

What does this mean?

  • Routing filters need to be rewritten to simplify the syntax and operation – there were a lot of complaints with the original syntax.
  • Routing protocols like OSPF and BGP have been unstable in beta1 through beta6 and need some work to stabilize them.

What’s the issue with routing filters?

The original v7 routing filters were very complicated to work with and had esoteric terms for operations like ‘subsumes’ and required ‘rule’ and ‘select-rule’ config to actually reference the filter in the routing process.


Previous filter syntax:

I wrote an article in late 2020 on IPv6 with BGP/OSPF using beta2 and captured a few screenshots that aren’t in the online docs anymore.

More details are in the article here:

MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP – StubArea51.net

New syntax?

If the current filter documentation represents the newer style, there are several differences in the format. The more complicated language like ‘subsumes’ is gone and only one filter rule is required to use the filter in the routing process – the ‘select-rule’ syntax is gone.


The new filter syntax appears to have made it onto help.mikrotik.com under /routing/filter but this may change in the coming weeks.

Example:

Similar to the OSPF example above, this is the example listed under the filter section for the new format.

The options below represent the matching order and the possible readable and set parameters.

Accepted Syntax:

if ( [matchers] ) { [actions] } else { [actions] }

[matchers]:
[prop readable] [bool operator] [prop readable]

[actions]:
[action] [prop writeable] [value]



Accepted parameters:

[num prop readable]
dst-len
bgp-path-len
bgp-input-local-as
bgp-input-remote-as
bgp-output-local-as
bgp-output-remote-as
ospf-metric
ospf-tag
rip-metric
rip-tag

[num prop writable]
distance
scope
scope-target
bgp-weigth
bgp-med
bgp-out-med
bgp-local-pref
bgp-igp-metric
bgp-path-peer-prepend
bgp-path-prepend
ospf-ext-metric
ospf-ext-tag
rip-ext-metric 
rip-ext-tag 

[flag prop readable]
active
bgp-attomic-aggregate
bgp-communities-empty
bgp-communities-ext-empty
bgp-communities-large-empty
bgp-network
ospf-dn

[flag prop writable]
ospf-ext-dn
blackhole
use-te-nexthop

[predicate]
bgp-communities|bgp-communities-ext|bgp-communities-large
    equal|any|includes|subset
	{inline set}	
    equal-set|any-set|includes-set|subset-set
	{set name}
    any-regexp|subset-regexp
	{regexp}
comment
    text|find|regexp
	{string}
chain
    {chain name}
vrf
    {vrf}
rtab
    {rtab}
gw-interface
    {interface}
gw-check
    none|arp|icmp|bfd|bfd-mh	
afi
    ipv4|ipv6|l2vpn|l2vpn-cisco|vpnv4|vpnv6
	,...
protocol
    connected|static|bgp|ospf|rip|dhcp|fantasy|modem|vpn
	,...
bpg-origin
    igp|egp|incomplete
	,...
bgp-as-path
    {regexp}
rpki
    valid|invalid|unknown
ospf-type
    intra|inter|ext1|ext2|nssa1|nssa2
ospf-ext-type
    type1|type2
[num prop readable]
    in
	{int..int}|{int-int}
    ==|!=|<=|>=|<|>
	{int}
[prfx prop readable]
    !=|==|in
	{address 46/}
[flag prop readable]


[block]
if ([predicate] &&/|| ...) { [block] } [ else {[block]} ]
accept|reject|return
jump {chain name}
unset
    pref-src|bgp-med|bgp-out-med|bgp-local-pref
bgp-communities|bgp-communities-ext|bgp-communities-large
    append|replace|filter-in|filter-not-in
	{inline community set}
    append-set|replace-set|filter-in-set|filter-not-in-set
	{set name}
    filter-in-regexp|filter-not-in-regexp
	{regexp}
    delete
	wk|other <-- for communities
	    ,...
	rt|soo|other <-- for ext-communities
	    ,...
	all <-- for large-communities
rpki-verify 
    {rpki group name}
comment
    set|append
	{string}
set
    [num prop writable]
	[num prop readable]|[num prop writable]
	+/-	
	    [num prop readable]|[num prop writable]
    gw
	interface
	    {interface}
	{address 46i}
    gw-check
	none|arp|icmp|bfd|bfd-mh	
    pref-src
	{address 46}
    bgp-origin
	igp|egp|incomplete
    ospf-ext-fwd
	{address 46}
    ospf-ext-type
	type1|type2



[num prop readable]
    in
        {int..int}|{int-int}
    ==|!=|<=|>=|<|>
        {int}
        [num prop readable]

What are the problems with routing protocols?

The routing stack has been completely re-written for ROSv7 from what we are told. This takes some time.

When I was last out in Silicon Valley, I met with a company that just emerged from stealth and had designed a new Network OS. They spent 3 years in stealth working on nothing but the OS with no actual product being sold – just coding and development.

So it’s not surprising this process has taken a while.

Stabilizing protocol issues

There are a number of bugs we’ve seen in the early versions of ROSv7 beta for routing protocols that are being worked through. Things like OSPF checksum, interface templates, high cpu when areas are disabled/enabled….etc.

Now that we know routing protocols are a priority and the new filter syntax is taking shape, I would expect to see some improvement across the next 2 to 3 beta releases to get routing protocols stable with simple configs.

L3 Switching

From talking to a lot of people that write code for Network OSes and work on the interaction with the ASIC, this is one of the hardest areas to get right – pushing routes from the RIB down into the HW FIB.

MikroTik hired new developers to meet this challenge:

https://forum.mikrotik.com/viewtopic.php?f=1&t=175369#p859571

MikroTik has updated the L3 HW doc pages to provide a roadmap of features and functionality for the beta series.

It appears that Jumbo MTU is the next major feature to be added for L3 HW in ROSv7.1beta7

https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading

When will ROSv7 move to release candidate and then stable?

When asked about the state of ROSv7, my typical answer has been that we’ll see a stable version in mid 2022 based on the pace of development.

I think that’s still a fair answer based on the pace of development.

It seems like the routing protocols and filters we need a few more beta versions to get working and then they’ll move to release candidates – my estimate is to look for the RCs towards the end of 2021.

Hopefully this has been helpful…i’ll probably write another summary on the state of ROSv7 once more progress has been made.