IP Infusion: EVPN-MPLS first look on GA 6.0

IP Infusion just released OcNOS version 6.0 and the release notes, as well as press release, show a focus on EVPN with an MPLS data plane. Don’t forget EVPN and VxLAN aren’t mutually exclusive, EVPN runs on and was originally designed for a MPLS data plane. I recently discussed this on a podcast EVPN doesn’t need VxLAN if you want to know more on that topic.

Lets take a look at basic EVPN-VPWS and EVPN-VPLS deployment. Since we’re looking at an MPLS data plane we will utilize ISIS-SR for MPLS. We’re utilizing ISIS-SR as it is increasingly replacing LDP and RSVP-TE for label distribution.

IGP and Label Distribution

First let’s look at the IGP setup and label distribution as everything else will be built on top of this.

ipi-1.lab.jan1.us.ipa.net#show run int lo
interface lo
 ip address
 ip address secondary
 ipv6 address ::1/128
 ipv6 address 2001:db8::1/128
 prefix-sid index 101
 ip router isis UNDERLAY
 ipv6 router isis UNDERLAY

We have to set an index to create the node-sid for this device. In this case we use 101.

ipi-1.lab.jan1.us.ipa.net#show run segment-routing
 mpls sr-prefer
 global block 16000 23999

Since our segment routing global block starts at 16000 the node-sid becomes 16101 as the index + the start of the SRGB defines the sid. Additionally, we run mpls sr-prefer as this will prefer SR labels over LDP or RSVP-TE labels.

ipi-1.lab.jan1.us.ipa.net#show run isis
router isis UNDERLAY
 is-type level-1-2
 metric-style wide
 mpls traffic-eng router-id
 mpls traffic-eng level-1
 mpls traffic-eng level-2
 capability cspf
 fast-reroute ti-lfa level-1 proto ipv4
 fast-reroute ti-lfa level-2 proto ipv4
 net 49.0015.1001.2700.0001.00
 segment-routing mpls

Finally, we have to enable ISIS for segment routing.

ipi-1.lab.jan1.us.ipa.net#show clns neighbors

Total number of L1 adjacencies: 1
Total number of L2 adjacencies: 1
Total number of adjacencies: 2
Tag UNDERLAY:  VRF : default
System Id      Interface   SNPA                State  Holdtime  Type Protocol
ipi-2.lab.jan1.us.ipa.net xe48        3c2c.99c0.00aa      Up     26        L1L2 IS-IS
ipi-1.lab.jan1.us.ipa.net#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
        K - CLI ILM, T - MPLS-TP, s - Stitched ILM
       S - SNMP, L - LDP, R - RSVP, C - CRLDP
       B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
       O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
       P - SR Policy, U - unknown

Code    FEC/VRF/L2CKT    ILM-ID      In-Label    Out-Label   In-Intf    Out-Intf/VRF       Nexthop
   i>     4           16101       Nolabel     N/A        N/A    
   B>   evpn:1             3           17          Nolabel     N/A        N/A    
   B>   evpn:100           1           16          Nolabel     N/A        N/A    
   B>   evpn:1             2           640         Nolabel     N/A        N/A    
   P>     7           20          3           N/A        xe48   
   i>     5           26240       3           N/A        xe48   
   i>     6           16102       3           N/A        xe48   

Now we can see that we have a clns/isis neighbor with ipi-2 as well as learned labels. We can see both device’s node-sids in the label table on ipi-1.

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg


Next we can build EVPN on top of the underlay to begin delivering services. First we have to build an EVPN BGP session between the two routers.

ipi-1.lab.jan1.us.ipa.net#show run bgp
router bgp 65000
 neighbor remote-as 65000
 neighbor update-source lo
 address-family l2vpn evpn
 neighbor activate
ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary
BGP router identifier, local AS number 65000
BGP table version is 32
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP
MCAST    ESI  PREFIX-ROUTE              4 65000 22856      22856      32      0      0  6d18h34m               2      1      0
     1      0      0


Next we can start build services on top. First we’ll build an EVPN-VPWS service.

evpn mpls enable
evpn mpls vtep-ip-global
mac vrf BLUE
 route-target both evpn-auto-rt
evpn mpls id 100 xconnect target-mpls-id 2
 host-reachability-protocol evpn-bgp BLUE
interface xe46.10 switchport
 encapsulation dot1q 10
  map vpn-id 100

EVPN MPLS has to be enabled. *IMPORTANT* This requires a reboot. Next the vtep id needs to be set. These are global settings for the environment.

For the creation of the service we’ll start by making a mac vrf to generate the information needed to create a EVPN type 2 route (mac-ip).

Since this is VPWS it is considered a cross connect xconnect and a target is defined. This is the remote PE vpn-id, in this case 2.

Finally it is assigned to a switchport. It has to be a switchport with a type of access-if-evpn. This maps back to the EVPN mac-vrf via the xconnect. Anything arriving on xe46.10 with a dot1q tag of 10 is placed into this tunnel.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect
EVPN Xconnect Info
AC-AC: Local-Cross-connect
AC-NW: Cross-connect to Network
AC-UP: Access-port is up
AC-DN: Access-port is down
NW-UP: Network is up
NW-DN: Network is down
NW-SET: Network and AC both are up

Local                            Remote       Connection-Details

================================ ============ ==========================================================================
VPN-ID       EVI-Name      MTU   VPN-ID       Source       Destination                   PE-IP           MTU   Type   NW
================================ ============ ==========================================================================
100          ----          1500  2            xe46.10      --- Single Homed Port ---     1500  AC-NW  NW

Total number of entries are 1
ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect tunnel
EVPN-MPLS Network tunnel Entries
Source           Destination      Status        Up/Down       Update        local-evpn-id remote-evpn-id
========================================================================================================      Installed     01:31:06      01:31:06      100           2

Total number of entries are 1

The tunnels are up, installed, and ready for forwarding. We can see the CE macs as mac-ip routes in evpn.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn vrf BLUE
BGP table version is 1, local router ID is
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

[EVPN route type]:[ESI]:[VNID]:[relevent route informantion]
1 - Ethernet Auto-discovery Route
2 - MAC/IP Route
3 - Inclusive Multicast Route
4 - Ethernet Segment Route
5 - Prefix Route

    Network          Next Hop            Metric    LocPrf	Weight     Path  Peer          Encap
* i  [1]:[0]:[2]:[16]          0        100       0    i     MPLS
*>   [1]:[0]:[100]:[16]
                       0        100       32768  i  ----------      MPLS

Total number of prefixes 2

The mac addresses are sent via an EVPN type-2 route between PEs.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
                                                     EVPN MPLS MAC Entries
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc

Total number of entries are : 0

Since this is VPWS there are no macs learned on the device.

[email protected]# run ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=21.531 ms
64 bytes from icmp_seq=1 ttl=64 time=22.124 ms

Success! The CEs can reach each other over the EVPN-VPWS circuit.


Now we’ll build an EVPN-VPLS service. The BGP setup is the same so we’ll focus solely on the differences. The first one being the vpn-id creation.

mac vrf ORANGE
 route-target both evpn-auto-rt
evpn mpls id 1
 host-reachability-protocol evpn-bgp ORANGE

There is no end point defined as a xconnect. All that is necessary is to bind the mac vrf to the evpn vpn id.

interface xe46.100 switchport
 encapsulation dot1q 100
  map vpn-id 1

Again, a switchport defined as an access-if-evpn is necessary. This is then mapped to the vpn-id for the VPLS service. In this case anything coming in with a dot1q tag of 100 will be placed into vpn-id 1.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
                                                     EVPN MPLS MAC Entries
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc

1          xe46.100  ----      ----      84c1.c132.5031                    Dynamic Local   -------    0
1          ----      ----      ----      84c1.c132.5032                    Dynamic Remote  -------    0

Total number of entries are : 2

Since this is a VPLS service MACs are learned both locally and remotely. The remote MAC is the MAC of the remote CE. This was learned via EVPN and from the VTEP

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn mac-ip vrf ORANGE
ESI                            Eth-Tag     Mac-Address    IP-Address                              VNID/LABEL     L3VNID
   Nexthop         GW-Type         Encap
0                              1           84c1:c132:5031 --                                      17             0     --              MPLS
0                              1           84c1:c132:5032 --                                      17             0     --              MPLS

The type-2 routes are populated in the BGP table.

[email protected]# run ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=21.894 ms
64 bytes from icmp_seq=1 ttl=64 time=22.159 ms

Success! We have reachability across the service.


IP Infusion is continuing to build their evpn/mpls deployment as well as segment routing. It is exciting to see these feature sets continue to mature as traditional LDP/VPLS deployments move to EVPN/MPLS. If you need assistance on the transition from LDP to segment routing or VPLS to EVPN reach out to IP Architechs.

Interop IP infusion and MikroTik: part 2 – VPLS

VPLS is a pretty common technology in ISPs to either sell layer 2 services or backhaul traffic to a centralized aggregation point to conserve IPv4 space; check out more on that here.

How can I take advantage of segment routing but still deliver the same services? We already looked at how to setup the label switched paths utilizing IP Infusion as a segment routing mapping server in this interop post. Now let’s see how we can deliver a VPLS service over this with mikrotik as a provider edge router.

Delivering a service with a L2VPN

After setting up the IGP and label distribution between the PEs we will start building the L2VPN.

Why a BGP signaled VPLS session instead of LDP signaled VPLS?

In the segment-routing domain there is no LDP running. However, some vendors support static pseudowires or other methods to bring up a targeted LDP session for VPLS. I did some basic testing here couldn’t easily identify the right combination of knobs to make this work. Don’t worry; I’ll come back to it.

BGP signaled VPLS is a standards based technology that both vendors support.

First thing we need to do after having loopback reachability is to build the BGP sessions.


router bgp 65000
 neighbor remote-as 65000
 neighbor update-source lo
 address-family l2vpn vpls
 neighbor activate

/routing bgp peer
add address-families=l2vpn name=OCNOS1 nexthop-choice=force-self \
    remote-address= remote-as=65000 update-source=Lo0

After establishing the BGP peering we need to build the vpls instance.


mpls vpls TEST 10
 signaling bgp
  ve-id 10

/interface vpls bgp-vpls
add bridge=VPLS-TEST export-route-targets=65000:10 import-route-targets=\
    65000:10 name=TEST route-distinguisher= site-id=2 \

On OcNOS the route-target and route distinguisher is automatically derived from the vpls instance creation. While you explicitly set it on MikroTik.

ipi-1.lab.jan1.us.ipa.net# show mpls vpls TEST
Virtual Private LAN Service Instance: TEST, ID: 10
 SIG-Protocol: BGP
  Route-Distinguisher :65000:10
  Route-Target :65000:10
  VE-ID :10

The site-id (Tik) or VE-ID (OcNOS) are the vpls endpoint identifiers which uniquely identify each PE.

The final step before we can start to test is to setup the attachment circuit. On OcNOS we will have to setup a service template to pair with the vpls instance on attachment circuit. Then you can assign the vpls instance and the service template to a switchport. We will match vlan tag 10 on our service template. Anything coming in with dot1q 10 from the CCR2004 will enter the VPLS tunnel.


service-template TEST
 match outer-vlan 10
interface xe2
 mpls-vpls TEST service-template TEST

On the MikroTik side we need to setup a bridge for the autocreation of the tunnels and assign appropriate interfaces to it. Again, we will match on dot1q 10.


/interface bridge
add name=Lo0
add name=LoVOICE
add mtu=1500 name=VPLS-TEST protocol-mode=none

/interface bridge port
add bridge=VPLS-TEST interface=ether10 pvid=10

/interface bridge vlan
add bridge=VPLS-TEST tagged=VPLS-TEST untagged=vlan10,ether10 vlan-ids=10

Finally, we can do testing and verification. We want to look for the mesh peer which is the autodiscovered PE also in this vpls instance.

ipi-1.lab.jan1.us.ipa.net#show mpls vpls TEST
Virtual Private LAN Service Instance: TEST, ID: 10
 SIG-Protocol: BGP
  Route-Distinguisher :65000:10
  Route-Target :65000:10
  VE-ID :10
 Attachment-Circuit :UP
 Learning: Enabled
 Control-Word: Disabled
 Group ID: 0, Configured MTU: 1500
 Description: none
 service-tpid: dot1.q
 Operating mode: Raw
 Configured interfaces:
  Interface: xe2
 Service-template : TEST
  Match criteria : 10

 Mesh Peers: (Up)

Then we want to make sure we are learning mac-addresses. As you can see the MACs not learned from the attached device are learned from the BGP peer at

ipi-1.lab.jan1.us.ipa.net#show mpls vpls mac-address
VPLS-ID     MAC address       Learned from   Vlan-Id          Peer address   Tim
10          027b.6866.7831    xe48           -          300

10          488f.5a00.4f87    xe2            10               -              300

10          4c5e.0c6f.4476    xe48           -          300

Next we can test reachability. I did have a strange issue where if the AC went down on the MikroTik-1 the mesh wouldn’t form on reattachment. This was resolvable with the following command on IPI-1: clear bgp l2vpn vpls

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg

I’ll be doing more testing on interop here. Let us know in the comments what you’d like to see next.