MikroTik – RouterOSv7 first look – feedback on routing filters

Overview

Routing filters have been a hot topic lately in the world of RouterOSv7. The first implementation of routing filters in ROSv7 was difficult to work with and documented in the two articles below:

MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP

MikroTik RouterOS – v7.0.3 stable (chateau) and status of general release


MikroTik then made some changes and opened up discussion to get feedback. I did a lot of work and testing using ROS 7.1beta7 which never made it to public release and was close to publishing the results when 7.1rc1 came out so this post will use that version.

https://forum.mikrotik.com/viewtopic.php?f=1&t=175201#p867750


RouterOSv7.1rc1 syntax example

Here is an example of the latest syntax in ROSv7.1rc1

CLI

### MikroTik RouterOS 7.1rc1 ###

/routing filter rule
add chain=dead.beef.101 rule="if (dst==200:dead:beef:101::/64) {accept}"
add chain=dead.beef.102 rule="if (dst==200:dead:beef:102::/64) {accept}"
add chain=dead.beef.agg rule="if (dst in 200:dead:beef::/48) {accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.101) {set bgp-local-pref 300; accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.102) {accept}"
add chain=bgp-out-v6 rule="if (chain dead.beef.agg && dst-len<128) {set bgp-local-pref 150; accept}"

Winbox


And the corresponding routes received (for the v7 filter rules)

Comparable syntax in ROSv6 (note: recursive routing is not supported in IPv6 so the iBGP routes used to test v7rc1 would not be possible)

### MikroTik RouterOS 6.48.10 (long-term) ###

/routing filter
add action=accept chain=dead.beef.101 prefix=200:dead:beef:101::/64
add action=accept chain=dead.beef.102 prefix=200:dead:beef:102::/64
add action=accept chain=dead.beef.agg prefix=200:dead:beef::/48
add action=accept chain=bgp-out-v6 match-chain=dead.beef.101 set-bgp-local-pref=300
add action=accept chain=bgp-out-v6 match-chain=dead.beef.102 
add action=accept chain=bgp-out-v6 match-chain=dead.beef.agg prefix-length=48-128 set-bgp-local-pref=150



Challenges and Feedback

  • Coding vs. Network CLI – The single biggest resistance to the new style of filtering is the string format is hard to work with. It requires knowledge of the match and action statements as well as how to write an expression to correctly parse them. Network engineers are not software engineers and route filtering (much like firewalling) has the capacity to be complex and require many lines of config – which means the format should align with v6 route filters and/or firewall rules.
  • Tab complete – The lack of tab complete is a big gap. Network engineers, admins and technicians expect to be able to tab complete when creating a config. This is rumored to be in the works and hopefully it will make it into later release candidates.

  • Using context sensitive help with “?” – There are a couple issues with context sensitive help in ROS 7.1rc1. The first issue is using the F1 key for help in any part of ROSv7 (not just filtering) instead of the ‘?’. This should at least be an option that can be set.

    The second issue is the lack of context sensitive help for the routing filters – if an engineer is unsure of the syntax, it’s currently not possible to get help from the command line.

    This has been a fundamental part of CLI based network operating systems for over 30 years. It needs to be added back.


    Conclusions

    One thing is clear, everyone I discussed it with on Facebook, Reddit, MikroTik Forums and with clients and engineers on my team did not like the new format.

    It’s worth noting that MikroTik equipment is often used in remote locations where it isn’t practical to pull up the help docs and engineers in the field rely on the ability to use tab-complete and context sensitive help to finish configuration tasks. This is a critical feature for a network operating system to have.

    To MikroTik – please consider implementing the filters so that at a minimum, the features in ROSv6 (like tab-complete, context sensitive help and a non-coding syntax) are maintained while allowing for new functionality.


Examples of filtering in other well-known operating systems

Here are some examples of the same filtering rules in different network operating systems for comparison. All of them support tab-complete and context sensitive help.

Free Range Routing is probably one of my favorites because it’s open and is being actively developed. JunOS is very popular from a filtering standpoint because the OS is easy to work with programatically.

Cisco is included because they are pervasive but isn’t at the top of my list because the syntax isn’t anything special and IOS-XR equipment tends to be incredibly expensive even though bugs are still commonplace – so it’s not a great value.

FRRouting · GitHub

Free Range Routing (https://frrouting.org)

### Free Range Routing v6.0.2 ###

ipv6 prefix-list dead.beef.101 seq 10 permit 200:dead:beef:101::/64
ipv6 prefix-list dead.beef.102 seq 10 permit 200:dead:beef:102::/64
ipv6 prefix-list dead.beef.agg seq 10 permit 200:dead:beef::/48 le 128
!
route-map bgp-out-v6 permit 10
 match ipv6 address prefix-list dead.beef.101
 set local-preference 300
!
route-map bgp-out-v6 permit 20
 match ipv6 address prefix-list dead.beef.102
!
route-map bgp-out-v6 permit 30
 match ipv6 address prefix-list dead.beef.agg
 set local-preference 150
### Free Range Routing v6.x ###!

File:Juniper Networks logo.svg - Wikimedia Commons

Juniper Networks JunOS (https://junipernetworks.com)

### Juniper JunOS 18.x ###

policy-options {
    prefix-list dead.beef.101 {
        200:dead:beef:101::/64;
    }
    prefix-list dead.beef.102 {
        200:dead:beef:102::/64;
    }
    prefix-list dead.beef.agg {
        200:dead:beef::/48;
    }
    policy-statement bgp-out-v6 {
        term dead.beef.101 {
            from {
                prefix-list dead.beef.101;
            }
            then {
                local-preference 300;
                accept;
            }
        }
        term dead.beef.102 {
            from {
                prefix-list dead.beef.102;
            }
            then accept;
        }
        term dead.beef.agg {
            from {
                prefix-list-filter dead.beef.agg orlonger;
            }
            then {
                local-preference 150;
                accept;
            }
        }
    }
}

Cisco Systems - Wikipedia

Cisco Networks IOS XR (https://cisco.com)

### Cisco IOS-XR x.x ###

prefix-set dead.beef.101
  200:dead:beef:101::/64
end-set
!
prefix-set dead.beef.102
  200:dead:beef:102::/64
end-set
!
prefix-set dead.beef.agg
  200:dead:beef::/48 le 128
end-set
!
route-policy bgp-out-v6
  if destination in dead.beef.101 then
    set local-preference 300
    pass
  elseif destination in dead.beef.102 then
    pass
  elseif destination in dead.beef.agg then
    set local-preference 150
    pass
  endif
end-policy

MikroTik RouterOS – v7.0.3 stable (chateau) and status of general release

If you don’t already use it, the MIkroTik v7 BETA forum (forum.mikrotik.com) is a fantastic source of information


When will stable be released?

This is the million dollar question. Technically, it already has been for one hardware platform…

!! Spoiler Alert – There is *already* a stable release of ROSv7 – v7.0.3!!

The Chateau 5G router originally shipped with a beta version of ROSv7 but was quietly moved to a stable version that’s developed specifically for that platform.

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

Because of the way MikroTik’s code repo works, this version can’t easily be added to the main download page and support provides the software:

ROSv7.0.3 Stable Download (!!! Chateau Only – will brick other hardware !!!)

https://box.mikrotik.com/f/7e3cad5779804d0b878d/?dl=1

It’s worth repeating MikroTik’s warning about using this on any platform other than the Chateau

v7 launch date – MikroTikhttps://forum.mikrotik.com/viewtopic.php?f=1&t=175201#p865452




What’s holding up v7 from being released?

If you’ve been around MikroTik for a while, then you know that version 7 has been in the works for a long time to add new functionality and address limitations of the older Linux kernel in ROSv6.

MikroTik recently added a detailed update on where the development roadmap is at and what the challenges are:

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

What does this mean?

  • Routing filters need to be rewritten to simplify the syntax and operation – there were a lot of complaints with the original syntax.
  • Routing protocols like OSPF and BGP have been unstable in beta1 through beta6 and need some work to stabilize them.

What’s the issue with routing filters?

The original v7 routing filters were very complicated to work with and had esoteric terms for operations like ‘subsumes’ and required ‘rule’ and ‘select-rule’ config to actually reference the filter in the routing process.


Previous filter syntax:

I wrote an article in late 2020 on IPv6 with BGP/OSPF using beta2 and captured a few screenshots that aren’t in the online docs anymore.

More details are in the article here:

MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP – StubArea51.net

New syntax?

If the current filter documentation represents the newer style, there are several differences in the format. The more complicated language like ‘subsumes’ is gone and only one filter rule is required to use the filter in the routing process – the ‘select-rule’ syntax is gone.


The new filter syntax appears to have made it onto help.mikrotik.com under /routing/filter but this may change in the coming weeks.

Example:

Similar to the OSPF example above, this is the example listed under the filter section for the new format.

The options below represent the matching order and the possible readable and set parameters.

Accepted Syntax:

if ( [matchers] ) { [actions] } else { [actions] }

[matchers]:
[prop readable] [bool operator] [prop readable]

[actions]:
[action] [prop writeable] [value]



Accepted parameters:

[num prop readable]
dst-len
bgp-path-len
bgp-input-local-as
bgp-input-remote-as
bgp-output-local-as
bgp-output-remote-as
ospf-metric
ospf-tag
rip-metric
rip-tag

[num prop writable]
distance
scope
scope-target
bgp-weigth
bgp-med
bgp-out-med
bgp-local-pref
bgp-igp-metric
bgp-path-peer-prepend
bgp-path-prepend
ospf-ext-metric
ospf-ext-tag
rip-ext-metric 
rip-ext-tag 

[flag prop readable]
active
bgp-attomic-aggregate
bgp-communities-empty
bgp-communities-ext-empty
bgp-communities-large-empty
bgp-network
ospf-dn

[flag prop writable]
ospf-ext-dn
blackhole
use-te-nexthop

[predicate]
bgp-communities|bgp-communities-ext|bgp-communities-large
    equal|any|includes|subset
	{inline set}	
    equal-set|any-set|includes-set|subset-set
	{set name}
    any-regexp|subset-regexp
	{regexp}
comment
    text|find|regexp
	{string}
chain
    {chain name}
vrf
    {vrf}
rtab
    {rtab}
gw-interface
    {interface}
gw-check
    none|arp|icmp|bfd|bfd-mh	
afi
    ipv4|ipv6|l2vpn|l2vpn-cisco|vpnv4|vpnv6
	,...
protocol
    connected|static|bgp|ospf|rip|dhcp|fantasy|modem|vpn
	,...
bpg-origin
    igp|egp|incomplete
	,...
bgp-as-path
    {regexp}
rpki
    valid|invalid|unknown
ospf-type
    intra|inter|ext1|ext2|nssa1|nssa2
ospf-ext-type
    type1|type2
[num prop readable]
    in
	{int..int}|{int-int}
    ==|!=|<=|>=|<|>
	{int}
[prfx prop readable]
    !=|==|in
	{address 46/}
[flag prop readable]


[block]
if ([predicate] &&/|| ...) { [block] } [ else {[block]} ]
accept|reject|return
jump {chain name}
unset
    pref-src|bgp-med|bgp-out-med|bgp-local-pref
bgp-communities|bgp-communities-ext|bgp-communities-large
    append|replace|filter-in|filter-not-in
	{inline community set}
    append-set|replace-set|filter-in-set|filter-not-in-set
	{set name}
    filter-in-regexp|filter-not-in-regexp
	{regexp}
    delete
	wk|other <-- for communities
	    ,...
	rt|soo|other <-- for ext-communities
	    ,...
	all <-- for large-communities
rpki-verify 
    {rpki group name}
comment
    set|append
	{string}
set
    [num prop writable]
	[num prop readable]|[num prop writable]
	+/-	
	    [num prop readable]|[num prop writable]
    gw
	interface
	    {interface}
	{address 46i}
    gw-check
	none|arp|icmp|bfd|bfd-mh	
    pref-src
	{address 46}
    bgp-origin
	igp|egp|incomplete
    ospf-ext-fwd
	{address 46}
    ospf-ext-type
	type1|type2



[num prop readable]
    in
        {int..int}|{int-int}
    ==|!=|<=|>=|<|>
        {int}
        [num prop readable]

What are the problems with routing protocols?

The routing stack has been completely re-written for ROSv7 from what we are told. This takes some time.

When I was last out in Silicon Valley, I met with a company that just emerged from stealth and had designed a new Network OS. They spent 3 years in stealth working on nothing but the OS with no actual product being sold – just coding and development.

So it’s not surprising this process has taken a while.

Stabilizing protocol issues

There are a number of bugs we’ve seen in the early versions of ROSv7 beta for routing protocols that are being worked through. Things like OSPF checksum, interface templates, high cpu when areas are disabled/enabled….etc.

Now that we know routing protocols are a priority and the new filter syntax is taking shape, I would expect to see some improvement across the next 2 to 3 beta releases to get routing protocols stable with simple configs.

L3 Switching

From talking to a lot of people that write code for Network OSes and work on the interaction with the ASIC, this is one of the hardest areas to get right – pushing routes from the RIB down into the HW FIB.

MikroTik hired new developers to meet this challenge:

https://forum.mikrotik.com/viewtopic.php?f=1&t=175369#p859571

MikroTik has updated the L3 HW doc pages to provide a roadmap of features and functionality for the beta series.

It appears that Jumbo MTU is the next major feature to be added for L3 HW in ROSv7.1beta7

https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading

When will ROSv7 move to release candidate and then stable?

When asked about the state of ROSv7, my typical answer has been that we’ll see a stable version in mid 2022 based on the pace of development.

I think that’s still a fair answer based on the pace of development.

It seems like the routing protocols and filters we need a few more beta versions to get working and then they’ll move to release candidates – my estimate is to look for the RCs towards the end of 2021.

Hopefully this has been helpful…i’ll probably write another summary on the state of ROSv7 once more progress has been made.