A guide for working with MTU and MPLS in a Wireless ISP
MPLS/VPLS MTU math can be complicated and is always a struggle to unravel.
To make it a little easier and put it into a WISP context, I designed this cheat sheet on 8.5 x 11 (to print for those that actually trust printers) and used common WISP equipment like MikroTik routers, Ubnt and Cambium radios with real world MTU values.
The MTU values are displayed in layers to make it easier to see where each value fits.
These values are meant to be a starting point by representing the minimum values required for MPLS/VPLS with a single 802.1q VLAN tag.
In general, after going through hundreds of WISP migrations, I’ve found it to be easier to implement the minimum values required when working on a production WISP to identify the effective lowest MTU in the network.
Once the network equipment has been modified and has been running in a stable way on the minimum values, then higher values can be considered and implemented (now that the effective lowest MTU on the network is documented)
It’s worth repeating MikroTik’s warning about using this on any platform other than the Chateau
What’s holding up v7 from being released?
If you’ve been around MikroTik for a while, then you know that version 7 has been in the works for a long time to add new functionality and address limitations of the older Linux kernel in ROSv6.
MikroTik recently added a detailed update on where the development roadmap is at and what the challenges are:
What does this mean?
Routing filters need to be rewritten to simplify the syntax and operation – there were a lot of complaints with the original syntax.
Routing protocols like OSPF and BGP have been unstable in beta1 through beta6 and need some work to stabilize them.
What’s the issue with routing filters?
The original v7 routing filters were very complicated to work with and had esoteric terms for operations like ‘subsumes’ and required ‘rule’ and ‘select-rule’ config to actually reference the filter in the routing process.
Previous filter syntax:
I wrote an article in late 2020 on IPv6 with BGP/OSPF using beta2 and captured a few screenshots that aren’t in the online docs anymore.
If the current filter documentation represents the newer style, there are several differences in the format. The more complicated language like ‘subsumes’ is gone and only one filter rule is required to use the filter in the routing process – the ‘select-rule’ syntax is gone.
The new filter syntax appears to have made it onto help.mikrotik.com under /routing/filter but this may change in the coming weeks.
Similar to the OSPF example above, this is the example listed under the filter section for the new format.
The options below represent the matching order and the possible readable and set parameters.
The routing stack has been completely re-written for ROSv7 from what we are told. This takes some time.
When I was last out in Silicon Valley, I met with a company that just emerged from stealth and had designed a new Network OS. They spent 3 years in stealth working on nothing but the OS with no actual product being sold – just coding and development.
So it’s not surprising this process has taken a while.
Stabilizing protocol issues
There are a number of bugs we’ve seen in the early versions of ROSv7 beta for routing protocols that are being worked through. Things like OSPF checksum, interface templates, high cpu when areas are disabled/enabled….etc.
Now that we know routing protocols are a priority and the new filter syntax is taking shape, I would expect to see some improvement across the next 2 to 3 beta releases to get routing protocols stable with simple configs.
From talking to a lot of people that write code for Network OSes and work on the interaction with the ASIC, this is one of the hardest areas to get right – pushing routes from the RIB down into the HW FIB.
MikroTik hired new developers to meet this challenge:
MikroTik has updated the L3 HW doc pages to provide a roadmap of features and functionality for the beta series.
It appears that Jumbo MTU is the next major feature to be added for L3 HW in ROSv7.1beta7
When will ROSv7 move to release candidate and then stable?
When asked about the state of ROSv7, my typical answer has been that we’ll see a stable version in mid 2022 based on the pace of development.
I think that’s still a fair answer based on the pace of development.
It seems like the routing protocols and filters we need a few more beta versions to get working and then they’ll move to release candidates – my estimate is to look for the RCs towards the end of 2021.
Hopefully this has been helpful…i’ll probably write another summary on the state of ROSv7 once more progress has been made.