MikroTik – RouterOSv7 first look – Dynamic routing with IPv6 and OSPFv3/BGP

If you missed it, take a look at MikroTik’s video on RouterOS v7 routing performance and changes.



Overview

One of the long awaited benefits of RouterOS version 7 is a new routing protocol stack that enables new capabilities and fixes limitations in RouterOSv6 caused by the use of a very old Linux kernel.

The new routing stack in v7 has created quite a buzz in the MikroTik community as lab tests have shown that it’s significantly more efficient in processing large numbers of BGP routes.

The ability to use MikroTik’s new generation of CCR routers with ARM64 to quickly process BGP routes is a blog post all to itself and we’ll tackle that in the future – however, the information below provides a quick look into the performance comparison between ROS v6 and v7.

The new routing stack also paves the way to add a number of features that have been needed for a long time like RPKI and large community support.

Using a lab based on EVE-NG, we’ll take a look at configuration changes and iBGP using the IPv6 AFI with OSPFv3 as the IGP for loopback/next hop reachability. Prior to 7.1beta2, this has been nonfunctional for years due to routing recursion limitations.

v7 Routing Protocol Status

For the most up to date information about features and capabilities in v7, MikroTik created a page that tracks feature status across the different beta releases

https://help.mikrotik.com/docs/display/ROS/v7+Routing+Protocol+Status



Lab design

ROS Version: 7.1beta2 (7.1beta3 was released just before I published this – at some point i’ll update with testing on beta3)
Network Modeling: EVE-NG Pro



OSPFv3

One of the biggest changes in OSPF for both version 2 (IPv4) and version 3 (IPv6) is the consolidation of menus into a single location for OSPF configuration.

In ROSv7, all configuration occurs under /routing/ospf/ and instances can be created for v2 or v3.

Change from ROSv6: OSPF Menu options have changed in ROSv7, this is partly due to combining OPSFv2 and OSPFv3 into a single configuration framework.

OSPF command options in ROSv6 for OSPFv2 and OSPFv3


OSPF command options in ROSv7 for both versions of OSPF

Change from ROSv6: There is a new flag in the IPv6 routing table for ECMP and no flag for RIP

When looking at the new output for the routing table, a few things stand out. ECMP has a new flag using the “+” symbol to denote two or more equal paths.

ECMP in IPv6 is a feature limitation that RouterOSv6 had and this will make it easier to deploy IPv6 networks with MikroTik.

RIP or Routing Internet Protocol is missing from the routing flags. It’s unclear at this point whether RIPv2 or RIP-NG will make it into RouterOSv7 since it’s not used very often anymore in prod networks.

Correcting issues with recursive routing in IPv6.

Being able to use recursive routing for advertising loopbacks and using iBGP with IPv6 has been a limitation of ROSv6 for a long time due to the older linux kernel in use.

Now that ROSv7 has added the initial support for OSPF and BGP, we are able to test IPv6 routing recursion.

Here is a test from PE-1 to PE-2 (2001:db8:101::12) using iBGP

It works!


Change from ROSv6: Using filters in OSPF

One of the first major challenges I had to solve when working with ROSv7 was figuring out why every route available became advertised into OSPF.

At first it looked like a bug, but when I dug deeper, I came across this snippet in the new MikroTik help docs

ROSv7 Basic Routing Examples – RouterOS – MikroTik Documentation

As it turns out, the default behavior is to advertise all routes in the absence of an outbound filter.

The next challenge was figuring out the new filtering syntax.

In order to use a rule in ROSv7, the “/routing filter select-rule” command must be used and reference the filter rule or no action will be taken.

In the example above, only interfaces that have been configured for OSPF will be advertised.

OSPF Config

Here is a summary of the OSPF configuration from the PE-1 router



BGP

As with OSPF, BGP saw a change in menu structure.

In ROSv7, BGP configuration has been revamped and is much closer to the style of configuration that Cisco/Juniper use with config elements that can be nested and reused.

Considering all the work that’s being done to improve full table convergence time on ROSv7, this change is a step in the right direction to allow MikroTik to compete with larger network vendors in the area of peering and transit.

Change from ROSv6: BGP Menu options have changed in ROSv7 to accommodate new features like Templates and RPKI

BGP command options in ROSv6


BGP command options in ROSv7



New Feature: BGP Roles

This is a new capability in BGP as of July 2020 and MikroTik was one of the first to have it implemented.

draft-ietf-idr-bgp-open-policy-13 – Route Leak Prevention using Roles in Update and Open messages


The main goal is to classify peerings into different roles that prevent inadvertent route leaks by adopting some basic filtering policies as a component of the role assignment.

Acceptable pairings are:

Here is an example of role types in ROSv7


This is an overview of how the roles deal with route advertisements and filtering.



New Feature: BGP Templates

BGP Templates allow specific settings for a peer connection to be reused in the connection configuration.

This saves quite a bit of time when deploying a large number of iBGP peerings, transit peerings, IX peerings, etc

Options available to set in templates

Here is a BGP template as configured in the lab for this post. The template is referenced by the connection config (aka peer config)

New Feature: iBGP ECMP for IPv6

ECMP has been working in ROSv6 for a ling time, but due to kernel limitations, it hasn’t been available in IPv6 due to the problems in routing recursion and making iBGP operational.

Now that routing recursion is fixed for IPv6, ECMP is possible.

ECMP capable IPV6 routes in BGP noted by the new “+” symbol in the routing table for ECMP.

Here is an example of a traceroute to the same prefix that’s using two different paths with ECMP.


BGP Configuration

Here is an overview of the BGP configuration for PE-1



Lab configurations


All Lab configs for ROSv7 are listed below (tested in 7.1beta2)

PE-1

CORE-1

PE-2

MikroTik – RouterOSv7 first look – L3 ASIC performance testing

When MikroTik announced the CRS3xx series switches a few years ago, one of the most exciting aspects of that news release was the prospect of L3 forwarding in hardware on very inexpensive devices.

A quick review of the Marvell Prestera ASIC family showed a number of advanced routing, switching, MPLS and VxLAN capabilites.

Fast forward to 2020, where MikroTik has started to enable some of those features in RouterOS v7 beta.

Now we can finally take some of the CRS3xx switches and test their capabilities with L3 forwarding performance in hardware


CRS 3xx series capabilities overview

Before getting into the testing, it’s probably helpful to review some of the basic specs and capabilities of the CRS3xx switch line.

Here is a chart from MikroTik that outlines ACL rule count, Unicast FDB entries and MTU size.

CRS 3xx model comparison

MIkroTik has been working on the development of the features listed below to offload into hardware.

For the tests in this article, we’ll be using IPv4 Unicast and Inter-VLAN routing.

Supported feature list

Currently, the following switches are supported.

For the testing in this article, we are using the CRS317-1G-16S+

Switches supported by 7.1beta2



Performance testing – overview

The physical setup for testing is fairly straightforward.

  • HP DL380 gen7 with ProxMox PVE 6.2-4
  • 2 x 10G Multimode OM4 fiber jumpers
  • 4 x 10G Multimode SFPs
  • CRS 317-1G-16S+ running RouterOS 7.1beta2

The logical setup is also very straightforward

  • ProxMox/KVM hypervisor using PVE 6.2-4
  • Two RFC1918 subnets and VLANs to test intervlan routing performance
  • Two Ubuntu 20.04 LTS VMs
  • iperf3 on both Ubuntu VMs

MikroTik Configuration


Test #1 – iperf3 testing at 96 byte MTU

CPU – 0% to 3%

CRS 317 – interface sfp-sfpplus15

iperf3 results

Test #2 – iperf3 testing at 512 byte MTU

CPU – 0% to 3%

CRS 317 – interface sfp-sfpplus15

iperf3 results

Test #3 – iperf3 testing at 1500 byte MTU

CPU – 0% to 3%

CRS 317 – interface sfp-sfpplus15

iperf3 results

Conclusions

The initial results are very promising. Getting close to 10G sustained L3 throughput using an ASIC on a device that lists for $399 USD is unheard of.

The most noteworthy items for improvement are the number of TCP retransmits in the iperf testing and the speed reduction as the MTU is reduced.

Normally, most ASIC based platforms will push 96 bytes through as fast as 1500 bytes. The retransmits suggest that more work is needed in the way RouterOS interfaces with the switching buffers for L3 HW offload

This is also on a beta version of RouterOS that will still go through many revisions before going into prod so I would expect to see the performance to improve as the code matures.

All things considered though, things are looking great to take the CRS3xx series and be able to deploy them as a true L3 switch in prod sometime in 2021.