MikroTik ISP Design: Building an 802.1q trunk between sites using VPLS and S-tag

Use Case

ISPs that use MikroTik are always looking for new ways to deliver services to customers and expand their offerings. Delivering Layer 2 at scale for customers is a design challenge that comes up frequently.

While it’s easy enough to build a VLAN nested inside of another VLAN  (see below), this requires you to build all of the VLANs a customer wants to use into the PE router or handoff switch.

However, if you have a client that needs a layer 2 service delivered to two or more points and wants to be able to treat it just like an 802.1q trunk and add VLANs in an ad-hoc way, then using the S-Tag feature in RouterOS along with VPLS transport is a great option.

What’s the S-tag do???

S-Tag

Clients will often ask me “what’s the S-Tag check box for?”

So a little background on this, there is a protocol for using outer and inner VLAN tags specified in IEEE 802.1ad that uses Service Tag (or S-Tag) to denote the outer VLAN tag used to transport Customer Tags (or C-Tags).

What makes the S-Tag/C-Tag a little bit different is that it actually changes the ethertype of the Frame.

ProtocolEthertype
802.1q (Normal VLAN Tags) 0x8100
802.1ad (S-tag) 0x88a8

Here is an overview of the frame format of each and links to the Metro Ethernet Forum Wiki for more info.

S-Tag

https://wiki.mef.net/display/CESG/S-Tag
S-Tag

C-Tag

https://wiki.mef.net/display/CESG/C-Tag

C-Tag

Lab Scenario

Here is a very common example of a deployment for a Layer 2 service to an end customer that rides on top of the ISP MPLS core.

In this lab we are using Cisco switches trunked to each other using VLAN 101 and 201 over a VPLS pseudowire with an S-Tag of 777.

s-tag lab

After configuring the P routers, PE routers and Cisco switches, let’s take a look at the Cisco switch and see if we can ping the SVI on the other switch on both trunked VLANs.

Here are the subnets used on the customer side:

Switch-1 subnets

Now let’s ping the .2 address for each VLAN on Switch-2

VLAN 101

vlan-101-ping

VLAN 201

vlan-201-ping

Notes on MTU

A note on MTU sizing, in order to hand off a 1500 byte packet with VPLS, you normally need an MPLS and L2MTU of 1530 bytes. In order to pass a second VLAN tag you’ll want to make sure your network equipment can go up to 1534 for Layer 2 and MPLS MTUs to pass 1500 byte packet with S-Tag.

Configs for the lab

In the section below, here are all the configs for this deployment

Cisco Switch-1

version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname Switch-1
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/1
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/2
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/3
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/0
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/1
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/2
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/3
 media-type rj45
 negotiation auto
!
interface Vlan101
 description customer-vlan
 ip address 192.168.101.1 255.255.255.0
!
interface Vlan201
 description customer vlan 2
 ip address 192.168.201.1 255.255.255.0
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
!
!
!
!
!
control-plane
!
banner exec ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
banner incoming ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
banner login ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
!
line con 0
line aux 0
line vty 0 4
!
!
end

Cisco Switch-2

version 15.2
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
service compress-config
!
hostname Switch-2
!
boot-start-marker
boot-end-marker
!
!
!
no aaa new-model
!
!
!
!
!
!
!
!
ip cef
no ipv6 cef
!
!
!
spanning-tree mode pvst
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
!
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
 switchport trunk encapsulation dot1q
 switchport mode trunk
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/1
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/2
 media-type rj45
 negotiation auto
!
interface GigabitEthernet0/3
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/0
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/1
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/2
 media-type rj45
 negotiation auto
!
interface GigabitEthernet1/3
 media-type rj45
 negotiation auto
!
interface Vlan101
 description customer-vlan
 ip address 192.168.101.2 255.255.255.0
!
interface Vlan201
 description customer vlan 2
 ip address 192.168.201.2 255.255.255.0
!
ip forward-protocol nd
!
no ip http server
no ip http secure-server
!
!
!
!
!
!
control-plane
!
banner exec ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
banner incoming ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
banner login ^C
**************************************************************************
* IOSv is strictly limited to use for evaluation, demonstration and IOS  *
* education. IOSv is provided as-is and is not supported by Cisco's      *
* Technical Advisory Center. Any use or disclosure, in whole or in part, *
* of the IOSv Software or Documentation to any third party for any       *
* purposes is expressly prohibited except as otherwise authorized by     *
* Cisco in writing.                                                      *
**************************************************************************^C
!
line con 0
line aux 0
line vty 0 4
!
!
end

MikroTik PE-1

/interface bridge
add name=Lo0
add name=vpls-bridge-vlan-777
/interface vpls
add disabled=no l2mtu=1500 mac-address=02:D5:C2:72:3A:1A name=vpls777 
    pw-type=tagged-ethernet remote-peer=100.127.1.2 vpls-id=8675309:777
/ip neighbor discovery
set ether2 discover=no
/interface vlan
add interface=vpls777 name=vlan777 use-service-tag=yes vlan-id=777
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/routing ospf instance
add name=ospf1 router-id=100.127.1.1
/interface bridge port
add bridge=Lo0 interface=ether2
add bridge=Lo0 interface=vlan777
/ip address
add address=100.64.0.1/29 interface=ether1 network=100.64.0.0
add address=100.127.1.1 interface=Lo0 network=100.127.1.1
/ip dhcp-client
add disabled=no interface=ether1
/mpls interface
set [ find default=yes ] mpls-mtu=1534
/mpls ldp
set enabled=yes lsr-id=100.127.1.1 transport-address=100.127.1.1
/mpls ldp interface
add interface=ether1 transport-address=100.127.1.1
/routing ospf network
add area=backbone network=100.64.0.0/29
add area=backbone network=100.127.1.1/32
/system identity
set name=MikroTik-PE-1
/tool romon
set enabled=yes

MikroTik PE-2

/interface bridge
add name=Lo0
add name=vpls-bridge-vlan-777
/interface vpls
add disabled=no l2mtu=1500 mac-address=02:C1:71:EB:0E:E7 name=vpls777 
    pw-type=tagged-ethernet remote-peer=100.127.1.1 vpls-id=8675309:777
/ip neighbor discovery
set ether2 discover=no
/interface vlan
add interface=vpls777 name=vlan777-s-tag use-service-tag=yes vlan-id=777
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/routing ospf instance
add name=ospf1 router-id=100.127.1.2
/interface bridge port
add bridge=Lo0 interface=vlan777-s-tag
add bridge=Lo0 interface=ether2
/ip address
add address=100.64.0.18/29 interface=ether1 network=100.64.0.16
add address=100.127.1.2 interface=Lo0 network=100.127.1.2
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/mpls interface
set [ find default=yes ] mpls-mtu=1534
/mpls ldp
set enabled=yes lsr-id=100.127.1.2 transport-address=100.127.1.2
/mpls ldp interface
add interface=ether1 transport-address=100.127.1.2
/routing ospf network
add area=backbone network=100.64.0.16/29
add area=backbone network=100.127.1.2/32
/system identity
set name=MikroTik-PE-2
/tool romon
set enabled=yes

MikroTik P-CORE-1

/interface bridge
add name=Lo0
/interface vlan
add interface=ether1 name=vlan101 vlan-id=777
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/routing ospf instance
set [ find default=yes ] router-id=100.127.0.1
/ip address
add address=100.64.0.2/29 interface=ether1 network=100.64.0.0
add address=100.127.0.1 interface=Lo0 network=100.127.0.1
add address=100.64.0.9/29 interface=ether2 network=100.64.0.8
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/mpls interface
set [ find default=yes ] mpls-mtu=1534
/mpls ldp
set enabled=yes lsr-id=100.127.0.1 transport-address=100.127.0.1
/mpls ldp interface
add interface=ether1 transport-address=100.127.0.1
add interface=ether2 transport-address=100.127.0.1
/routing ospf network
add area=backbone network=100.64.0.0/29
add area=backbone network=100.127.0.1/32
add area=backbone network=100.64.0.8/29
/system identity
set name=MikroTik-P-Core-1
/tool romon
set enabled=yes

MikroTik P-CORE-2

/interface bridge
add name=Lo0
/interface wireless security-profiles
set [ find default=yes ] supplicant-identity=MikroTik
/routing ospf instance
set [ find default=yes ] router-id=100.127.0.2
/ip address
add address=100.64.0.10/29 interface=ether1 network=100.64.0.8
add address=100.127.0.2 interface=Lo0 network=100.127.0.2
add address=100.64.0.17/29 interface=ether2 network=100.64.0.16
/ip dhcp-client
add dhcp-options=hostname,clientid disabled=no interface=ether1
/mpls interface
set [ find default=yes ] mpls-mtu=1534
/mpls ldp
set enabled=yes lsr-id=100.127.0.2 transport-address=100.127.0.2
/mpls ldp interface
add interface=ether1 transport-address=100.127.0.2
add interface=ether2 transport-address=100.127.0.2
/routing ospf network
add area=backbone network=100.64.0.8/29
add area=backbone network=100.127.0.2/32
add area=backbone network=100.64.0.16/29
/system identity
set name=MikroTik-P-Core-2
/tool romon

 

 

One thought to “MikroTik ISP Design: Building an 802.1q trunk between sites using VPLS and S-tag”

  1. Nice post! Very interesting architecture.

    Should interface “ether2” and “vlan777” be added to bridge vpls-bridge-vlan-777 instead of Lo0?

Leave a Reply

Your email address will not be published. Required fields are marked *