MikroTik RouterOS – v7.0.3 stable (chateau) and status of general release

If you don’t already use it, the MIkroTik v7 BETA forum (forum.mikrotik.com) is a fantastic source of information


When will stable be released?

This is the million dollar question. Technically, it already has been for one hardware platform…

!! Spoiler Alert – There is *already* a stable release of ROSv7 – v7.0.3!!

The Chateau 5G router originally shipped with a beta version of ROSv7 but was quietly moved to a stable version that’s developed specifically for that platform.

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

Because of the way MikroTik’s code repo works, this version can’t easily be added to the main download page and support provides the software:

ROSv7.0.3 Stable Download (!!! Chateau Only – will brick other hardware !!!)

https://box.mikrotik.com/f/7e3cad5779804d0b878d/?dl=1

It’s worth repeating MikroTik’s warning about using this on any platform other than the Chateau

v7 launch date – MikroTikhttps://forum.mikrotik.com/viewtopic.php?f=1&t=175201#p865452


https://iparchitechs.com/contact



What’s holding up v7 from being released?

If you’ve been around MikroTik for a while, then you know that version 7 has been in the works for a long time to add new functionality and address limitations of the older Linux kernel in ROSv6.

MikroTik recently added a detailed update on where the development roadmap is at and what the challenges are:

https://forum.mikrotik.com/viewtopic.php?t=175201#p865329

What does this mean?

  • Routing filters need to be rewritten to simplify the syntax and operation – there were a lot of complaints with the original syntax.
  • Routing protocols like OSPF and BGP have been unstable in beta1 through beta6 and need some work to stabilize them.

What’s the issue with routing filters?

The original v7 routing filters were very complicated to work with and had esoteric terms for operations like ‘subsumes’ and required ‘rule’ and ‘select-rule’ config to actually reference the filter in the routing process.


Previous filter syntax:

I wrote an article in late 2020 on IPv6 with BGP/OSPF using beta2 and captured a few screenshots that aren’t in the online docs anymore.

More details are in the article here:

MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP – StubArea51.net

New syntax?

If the current filter documentation represents the newer style, there are several differences in the format. The more complicated language like ‘subsumes’ is gone and only one filter rule is required to use the filter in the routing process – the ‘select-rule’ syntax is gone.


The new filter syntax appears to have made it onto help.mikrotik.com under /routing/filter but this may change in the coming weeks.

Example:

Similar to the OSPF example above, this is the example listed under the filter section for the new format.

The options below represent the matching order and the possible readable and set parameters.

Accepted Syntax:

if ( [matchers] ) { [actions] } else { [actions] }

[matchers]:
[prop readable] [bool operator] [prop readable]

[actions]:
[action] [prop writeable] [value]



Accepted parameters:

[num prop readable]
dst-len
bgp-path-len
bgp-input-local-as
bgp-input-remote-as
bgp-output-local-as
bgp-output-remote-as
ospf-metric
ospf-tag
rip-metric
rip-tag

[num prop writable]
distance
scope
scope-target
bgp-weigth
bgp-med
bgp-out-med
bgp-local-pref
bgp-igp-metric
bgp-path-peer-prepend
bgp-path-prepend
ospf-ext-metric
ospf-ext-tag
rip-ext-metric 
rip-ext-tag 

[flag prop readable]
active
bgp-attomic-aggregate
bgp-communities-empty
bgp-communities-ext-empty
bgp-communities-large-empty
bgp-network
ospf-dn

[flag prop writable]
ospf-ext-dn
blackhole
use-te-nexthop

[predicate]
bgp-communities|bgp-communities-ext|bgp-communities-large
    equal|any|includes|subset
	{inline set}	
    equal-set|any-set|includes-set|subset-set
	{set name}
    any-regexp|subset-regexp
	{regexp}
comment
    text|find|regexp
	{string}
chain
    {chain name}
vrf
    {vrf}
rtab
    {rtab}
gw-interface
    {interface}
gw-check
    none|arp|icmp|bfd|bfd-mh	
afi
    ipv4|ipv6|l2vpn|l2vpn-cisco|vpnv4|vpnv6
	,...
protocol
    connected|static|bgp|ospf|rip|dhcp|fantasy|modem|vpn
	,...
bpg-origin
    igp|egp|incomplete
	,...
bgp-as-path
    {regexp}
rpki
    valid|invalid|unknown
ospf-type
    intra|inter|ext1|ext2|nssa1|nssa2
ospf-ext-type
    type1|type2
[num prop readable]
    in
	{int..int}|{int-int}
    ==|!=|<=|>=|<|>
	{int}
[prfx prop readable]
    !=|==|in
	{address 46/}
[flag prop readable]


[block]
if ([predicate] &&/|| ...) { [block] } [ else {[block]} ]
accept|reject|return
jump {chain name}
unset
    pref-src|bgp-med|bgp-out-med|bgp-local-pref
bgp-communities|bgp-communities-ext|bgp-communities-large
    append|replace|filter-in|filter-not-in
	{inline community set}
    append-set|replace-set|filter-in-set|filter-not-in-set
	{set name}
    filter-in-regexp|filter-not-in-regexp
	{regexp}
    delete
	wk|other <-- for communities
	    ,...
	rt|soo|other <-- for ext-communities
	    ,...
	all <-- for large-communities
rpki-verify 
    {rpki group name}
comment
    set|append
	{string}
set
    [num prop writable]
	[num prop readable]|[num prop writable]
	+/-	
	    [num prop readable]|[num prop writable]
    gw
	interface
	    {interface}
	{address 46i}
    gw-check
	none|arp|icmp|bfd|bfd-mh	
    pref-src
	{address 46}
    bgp-origin
	igp|egp|incomplete
    ospf-ext-fwd
	{address 46}
    ospf-ext-type
	type1|type2



[num prop readable]
    in
        {int..int}|{int-int}
    ==|!=|<=|>=|<|>
        {int}
        [num prop readable]

What are the problems with routing protocols?

The routing stack has been completely re-written for ROSv7 from what we are told. This takes some time.

When I was last out in Silicon Valley, I met with a company that just emerged from stealth and had designed a new Network OS. They spent 3 years in stealth working on nothing but the OS with no actual product being sold – just coding and development.

So it’s not surprising this process has taken a while.

Stabilizing protocol issues

There are a number of bugs we’ve seen in the early versions of ROSv7 beta for routing protocols that are being worked through. Things like OSPF checksum, interface templates, high cpu when areas are disabled/enabled….etc.

Now that we know routing protocols are a priority and the new filter syntax is taking shape, I would expect to see some improvement across the next 2 to 3 beta releases to get routing protocols stable with simple configs.

L3 Switching

From talking to a lot of people that write code for Network OSes and work on the interaction with the ASIC, this is one of the hardest areas to get right – pushing routes from the RIB down into the HW FIB.

MikroTik hired new developers to meet this challenge:

https://forum.mikrotik.com/viewtopic.php?f=1&t=175369#p859571

MikroTik has updated the L3 HW doc pages to provide a roadmap of features and functionality for the beta series.

It appears that Jumbo MTU is the next major feature to be added for L3 HW in ROSv7.1beta7

https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading

When will ROSv7 move to release candidate and then stable?

When asked about the state of ROSv7, my typical answer has been that we’ll see a stable version in mid 2022 based on the pace of development.

I think that’s still a fair answer based on the pace of development.

It seems like the routing protocols and filters we need a few more beta versions to get working and then they’ll move to release candidates – my estimate is to look for the RCs towards the end of 2021.

Hopefully this has been helpful…i’ll probably write another summary on the state of ROSv7 once more progress has been made.

MikroTik – RouterOSv7 first look – MLAG on CRS 3xx switches

What is MLAG?

Multi-Chassis Link Aggregation Group or MLAG is an idea that’s been around for a while.

It allows for the ability to form LACP channels across multiple physical switches.

Wikipedia shows a few different topology examples here


Vendor implementations are proprietary but the idea of MLAG was first mentioned in 802.1AX-2008 in 2008.

It first started to become popular in data center networking in the late 2000s

What makes the addition of MLAG to MikroTik’s RouterOS feature set notable is that it lowers the barrier to entry for this particular feature.

CRS 3xx switches are very inexpensive (starting at $149 USD) and may very well be the lowest cost MLAG capable hardware available on the market.


Introduced in 7.1beta6

MLAG has been asked for by the MikroTik community a number of times and the most active feature request thread started here in 2020:

new feature request MLAG!!! – MikroTik

MikroTik added several version 7 beta releases in 2021 and included MLAG for all CRS 3xx series switches in 7.1beta6 on May 18th, 2021.

Overview of protocol requirements

MLAG is fairly consistent across vendors with the need for a link between physical devices that manages the MLAG groups. In MikroTik, these are called peer ports which facilitate the ICCP.

Here are a few terms for MikroTik MLAG:

ICCP (Inter Chassis Control Protocol). – Responsible for determining active/secondary switches and maintaining and updating the bridge table between physical switches.

Peer port – An interface that will be used as a peer port. Both peer devices use inter-chassis communication over the peer ports to establish MLAG and update the host table. The Peer port should be isolated on a different untagged VLAN using a pvid setting. The Peer port can be configured as a bonding interface.

System-id – The lowest MAC address between both peer bridges will be used as the system-id. This system-id is used for (R)STP bridge identifier.

Active-role – The peer with the lowest bridge MAC address will be acting as a primary device. The primary device is responsible for sending the correct LACP system ID on all MLAG ports.

mlag-id – An integer from 0 to 4294967295, it is used to set the MLAG ID for bonding interfaces. The same MLAG ID should be used on both peer devices to successfully create a single MLAG.

MikroTik’s requirements for ICCP and MLAG are:

  • RouterOS ICCP does not require an IP configuration
  • It should be isolated from the rest of the network using a dedicated untagged VLAN
  • Peer ports can also be configured as LACP bonding interfaces
  • MLAG requires enabled STP or RSTP protocol.


In order to present a single MAC address for the L2 spanning tree topology, ICCP functions on top of the peer ports to manage the MLAG/LACP system-id.

The system-id is used as the MAC address presented to the LACP client for RSTP/MSTP bridge identification.

reference for images and MLAG definitions: Multi-chassis Link Aggregation Group – RouterOS – MikroTik Documentation


Lab Example

In order to test the new MLAG functionality, we decided to setup a lab with CRS326-24S+2Q switches and CCR2004-1G-12S+2XS routers.

Below is the lab physical and logical topology.

Configuring an MLAG Group

Configure Bond and MLAG ID on CSW-01

/interface bonding
add mlag-id=100 mode=802.3ad name=Po1 slaves=sfp-sfpplus1


Configure Bond and MLAG ID on CSW-02

/interface bonding
add mlag-id=100 mode=802.3ad name=Po1 slaves=sfp-sfpplus1


* Apply each configuration step below on both switches to complete mlag setup and mlag-id 100. *


Configure the bridge and enable VLAN filtering. Add MLAG bonded interfaces and peer port to the bridge.

/interface bridge
add name=Bridge-MLAG vlan-filtering=yes
/interface bridge port
add bridge=Bridge-MLAG interface=Po1
add bridge=Bridge-MLAG interface=qsfpplus1-1 pvid=777


Configure a VLAN to be used over the MLAG

/interface bridge vlan
add bridge=Bridge-MLAG tagged=Po1 vlan-ids=3000


Set the peer port

/interface bridge mlag
set bridge=Bridge-MLAG peer-port=qsfpplus1-1


Validate the MLAG group


Show the status of the MLAG group, active and secondary ports and verify the system-id the client LACP receives

######## MLAG Switches - 2 x CRS326 ############

[admin@CSW-01] > interface/bridge/mlag/monitor 
       status: connected
    system-id: 48:8F:5A:3A:44:BA
  active-role: primary

[admin@CSW-02] > interface/bridge/mlag/monitor                        
       status: connected
    system-id: 48:8F:5A:3A:44:BA
  active-role: secondary

######## NON-MLAG LACP Router ############

[admin@RTR-01] > /interface bonding monitor Po1
                    mode: 802.3ad
            active-ports: sfp-sfpplus3,sfp-sfpplus4
          inactive-ports: 
          lacp-system-id: 48:8F:5A:00:4F:80
    lacp-system-priority: 65535
  lacp-partner-system-id: 48:8F:5A:3A:44:BA


Configurations

RTR-01

/interface bridge
add name=Lo0
/interface bonding
add mode=802.3ad name=Po1 slaves=sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-2-and-3
/interface vlan
add interface=Po1 name=vlan3000 vlan-id=3000
/routing table
add fib name=""
/ip address
add address=100.126.0.1/29 interface=vlan3000 network=100.126.0.0
add address=100.127.0.1 interface=Lo0 network=100.127.0.1
/ipv6 address
add address=200:100:126::1 interface=vlan3000
add address=200:100:127::1/128 advertise=no interface=Lo0
/system identity
set name=RTR-01

RTR-02

/interface bridge
add name=Lo0
/interface bonding
add mode=802.3ad name=Po1 slaves=sfp-sfpplus3,sfp-sfpplus4 transmit-hash-policy=layer-2-and-3
/interface vlan
add interface=Po1 name=vlan3000 vlan-id=3000
/routing table
add fib name=""
/ip address
add address=100.126.0.2/29 interface=vlan3000 network=100.126.0.0
add address=100.127.0.2 interface=Lo0 network=100.127.0.2
/ipv6 address
add address=200:100:126::2 interface=vlan3000
add address=200:100:127::2/128 advertise=no interface=Lo0
/system identity
set name=RTR-02

CSW-01

/interface bridge
add name=Bridge-MLAG vlan-filtering=yes
/interface bonding
add mlag-id=100 mode=802.3ad name=Po1 slaves=sfp-sfpplus1
add mlag-id=101 mode=802.3ad name=Po2 slaves=sfp-sfpplus2
/interface bridge mlag
set bridge=Bridge-MLAG peer-port=qsfpplus1-1
/interface bridge port
add bridge=Bridge-MLAG interface=Po1
add bridge=Bridge-MLAG interface=qsfpplus1-1 pvid=777
add bridge=Bridge-MLAG interface=Po2
/interface bridge vlan
add bridge=Bridge-MLAG tagged=Po1,Po2 vlan-ids=3000
/system identity
set name=CSW-01

CSW-02

/interface bridge
add name=Bridge-MLAG vlan-filtering=yes
/interface bonding
add mlag-id=100 mode=802.3ad name=Po1 slaves=sfp-sfpplus1
add mlag-id=101 mode=802.3ad name=Po2 slaves=sfp-sfpplus2
/interface bridge mlag
set bridge=Bridge-MLAG peer-port=qsfpplus1-1
/interface bridge port
add bridge=Bridge-MLAG interface=Po1
add bridge=Bridge-MLAG interface=qsfpplus1-1 pvid=777
add bridge=Bridge-MLAG interface=Po2
/interface bridge vlan
add bridge=Bridge-MLAG tagged=Po1,Po2 vlan-ids=3000
/system identity
set name=CSW-02