MikroTik CCR1072-1G-8S+ Review – update on Part 3 – Throughput

[adrotate banner=”4″]

Breaking the 80 Gbps barrier!!!

799px-Bell_X-1_46-062_(in_flight)

The long wait for real-world 1072 performance numbers is almost over – the last two VMWARE server chassis we needed to push the full 80 Gpbs arrived in the StubArea51 lab today. Thanks to everyone who wrote in and commented on the first two reviews we did on the CCR-1072-1G-8S+.  We initially began work on performance testing throughput for the CCR1072 in late July of this year, but had to order a lot of parts to get enough 10 Gig PCIe cards, SFP+ modules and fiber to be able to push 80 Gbps of traffic through this router.

Challenges

There have been a number of things that we have had to work through to get to 80 Gbps but we are very close. This will be detailed in the full review we plan to release next week but here are a few.

  • VMWARE ESXi – LACP Hashing – Initially we built LACP channels between the ESXi hosts and the 1072 expecting to load the links by using multiple source and destination IPs but we ran into issues with traffic getting stuck on one side of the LACP channel and had to move to independent links.
  • CPU Resources – TCP consumes an enormous amount of CPU cycles and we were only able to get 27 Gbps per ESXi host (we had two) and 54 Gbps total
  • TCP Offload – Most NICs these days allow for offloading of TCP segmentation by hardware in the NIC rather than the CPU, we were never able to get this working properly in VM WARE to reduce the load on the hypervisor host CPUs.
  • MTU – While the CCR1072 supports 10,226 bytes max MTU, VMWARE vswitch only supports 9000 so we lost some potential throughput by having to lower the MTU by more than 10%

Results preview

Here is a snapshot of where we have been able to get to using 6.30.4 bugfix so far

Single TCP Stream performance at 10 Gig in iperf at 9000 bytes MTU (L2/L3)

1072-10gig-tcp-stream

Throughput in WinBox

10-Gig-TCP-stream-interface

Max throughput so far with two ESXi 6.0 hosts (HP DL360 G6 with 2 Intel Xeon x5570 CPUs)

CCR1072-Traffic

 

 

Two more servers arrived today – Full results coming very soon!

Once we hit the performance limit at 54 Gbps, we ordered two more servers and they just arrived. We will be racking them into the lab immediately and should have some results by next week. Thanks for being patient and check back soon!!

 

 

MikroTik unveils new RouterOS development cycle

 

[adrotate banner=”4″]

 

As we all patiently await the release of RouterOS (RoS) v7 beta, MikroTik has announced a change in the way RoS development is organized.  There will now be three different tracks of development:

Bugfix only – When a current build is released, only fixes to known bugs will be added to this branch of development

Current – Current release will contain bugfixes and new features

Release Candidate – The release candidate will remain the development build for the next “current” release.

 

Graphical Overview of the new development cycle

MikroTik-bugfix-map

Image and notes below are from here

A small addendum: the Bugfix only will only contain verified fixes, and no new features. The Current release contains the same fixes but also new features and other improvements, sometimes also less critical fixes than in Bugfix. And finally the Release Candidate is more likely to a nightly build. We will not to intensive testing before publishing these, only quick check if upgrade can be done and if most features work fine.

Origin

The idea originally came out of this thread and after a flurry of positive commentary, it became a working practice shortly therafter.

We plan to make sub-version releases that only contain fixes and no new features.

For example, we would release 6.30 with some new features in FastPath. Then, if some issues are found, we would make 6.30.1 with only fixes and no new functionality. When we add a new option or feature, we would make 6.31.

6.XX.Y – only bug fixes
6.XX – fixes and feature changes

We could even make several fix-only releases in a row, if needed.

The idea is that you can upgrade to a sub-point release without risking new bugs, that could come with new features being added.

As the v6 cycle is coming to an end, there is simply no other ‘branch’ to put new features in, as v7 is still too ‘raw’. We could stop with the sub-point releases once v7 is mature enough for general use.

MikroTik download page gets a slight refresh with a new drop down

MikroTik-Download

Thoughts

This a very positive change for MikroTik and will go a long way with customers and integrators to be able to select code based on the needs of the environment. This release cycle is very similar to other network vendors (Think of Cisco IOS General Deployment vs Maintenance Deployment) and works well.

Code selection is such a critical element to the stability of a network and it’s helpful to have the ability to load a RoS code that has been patched but without new features. It’s a bit early to tell as to how much of an impact this will make in the MikroTik community, but the results so far look promising. I’ve done upgrades on a number of CCRs and 2011 series routers from 6.30 –> 6.30.1 –> 6.30.2 bugfix only and haven’t seen any major stability issue s or other bugs when running on a BGP/MPLS/OSPF service provider network.

There was a lot of churn with bugs and working features breaking early on in 6.x and this change should help to avoid at least some of that upheaval when 7.x makes it to a “current” release.