IP Infusion: EVPN-MPLS first look on GA 6.0

IP Infusion just released OcNOS version 6.0 and the release notes, as well as press release, show a focus on EVPN with an MPLS data plane. Don’t forget EVPN and VxLAN aren’t mutually exclusive, EVPN runs on and was originally designed for a MPLS data plane. I recently discussed this on a podcast EVPN doesn’t need VxLAN if you want to know more on that topic.

Lets take a look at basic EVPN-VPWS and EVPN-VPLS deployment. Since we’re looking at an MPLS data plane we will utilize ISIS-SR for MPLS. We’re utilizing ISIS-SR as it is increasingly replacing LDP and RSVP-TE for label distribution.

IGP and Label Distribution

First let’s look at the IGP setup and label distribution as everything else will be built on top of this.

ipi-1.lab.jan1.us.ipa.net#show run int lo
interface lo
 ip address
 ip address secondary
 ipv6 address ::1/128
 ipv6 address 2001:db8::1/128
 prefix-sid index 101
 ip router isis UNDERLAY
 ipv6 router isis UNDERLAY

We have to set an index to create the node-sid for this device. In this case we use 101.

ipi-1.lab.jan1.us.ipa.net#show run segment-routing
 mpls sr-prefer
 global block 16000 23999

Since our segment routing global block starts at 16000 the node-sid becomes 16101 as the index + the start of the SRGB defines the sid. Additionally, we run mpls sr-prefer as this will prefer SR labels over LDP or RSVP-TE labels.

ipi-1.lab.jan1.us.ipa.net#show run isis
router isis UNDERLAY
 is-type level-1-2
 metric-style wide
 mpls traffic-eng router-id
 mpls traffic-eng level-1
 mpls traffic-eng level-2
 capability cspf
 fast-reroute ti-lfa level-1 proto ipv4
 fast-reroute ti-lfa level-2 proto ipv4
 net 49.0015.1001.2700.0001.00
 segment-routing mpls

Finally, we have to enable ISIS for segment routing.

ipi-1.lab.jan1.us.ipa.net#show clns neighbors

Total number of L1 adjacencies: 1
Total number of L2 adjacencies: 1
Total number of adjacencies: 2
Tag UNDERLAY:  VRF : default
System Id      Interface   SNPA                State  Holdtime  Type Protocol
ipi-2.lab.jan1.us.ipa.net xe48        3c2c.99c0.00aa      Up     26        L1L2 IS-IS
ipi-1.lab.jan1.us.ipa.net#show mpls ilm-table
Codes: > - installed ILM, * - selected ILM, p - stale ILM
        K - CLI ILM, T - MPLS-TP, s - Stitched ILM
       S - SNMP, L - LDP, R - RSVP, C - CRLDP
       B - BGP , K - CLI , V - LDP_VC, I - IGP_SHORTCUT
       O - OSPF/OSPF6 SR, i - ISIS SR, k - SR CLI
       P - SR Policy, U - unknown

Code    FEC/VRF/L2CKT    ILM-ID      In-Label    Out-Label   In-Intf    Out-Intf/VRF       Nexthop
   i>     4           16101       Nolabel     N/A        N/A    
   B>   evpn:1             3           17          Nolabel     N/A        N/A    
   B>   evpn:100           1           16          Nolabel     N/A        N/A    
   B>   evpn:1             2           640         Nolabel     N/A        N/A    
   P>     7           20          3           N/A        xe48   
   i>     5           26240       3           N/A        xe48   
   i>     6           16102       3           N/A        xe48   

Now we can see that we have a clns/isis neighbor with ipi-2 as well as learned labels. We can see both device’s node-sids in the label table on ipi-1.

This image has an empty alt attribute; its file name is IPA-Blog-ad-template-network.jpg


Next we can build EVPN on top of the underlay to begin delivering services. First we have to build an EVPN BGP session between the two routers.

ipi-1.lab.jan1.us.ipa.net#show run bgp
router bgp 65000
 neighbor remote-as 65000
 neighbor update-source lo
 address-family l2vpn evpn
 neighbor activate
ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn summary
BGP router identifier, local AS number 65000
BGP table version is 32
1 BGP AS-PATH entries
0 BGP community entries

Neighbor                 V   AS   MsgRcv    MsgSen TblVer   InQ   OutQ    Up/Down   State/PfxRcd     AD  MACIP
MCAST    ESI  PREFIX-ROUTE              4 65000 22856      22856      32      0      0  6d18h34m               2      1      0
     1      0      0


Next we can start build services on top. First we’ll build an EVPN-VPWS service.

evpn mpls enable
evpn mpls vtep-ip-global
mac vrf BLUE
 route-target both evpn-auto-rt
evpn mpls id 100 xconnect target-mpls-id 2
 host-reachability-protocol evpn-bgp BLUE
interface xe46.10 switchport
 encapsulation dot1q 10
  map vpn-id 100

EVPN MPLS has to be enabled. *IMPORTANT* This requires a reboot. Next the vtep id needs to be set. These are global settings for the environment.

For the creation of the service we’ll start by making a mac vrf to generate the information needed to create a EVPN type 2 route (mac-ip).

Since this is VPWS it is considered a cross connect xconnect and a target is defined. This is the remote PE vpn-id, in this case 2.

Finally it is assigned to a switchport. It has to be a switchport with a type of access-if-evpn. This maps back to the EVPN mac-vrf via the xconnect. Anything arriving on xe46.10 with a dot1q tag of 10 is placed into this tunnel.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect
EVPN Xconnect Info
AC-AC: Local-Cross-connect
AC-NW: Cross-connect to Network
AC-UP: Access-port is up
AC-DN: Access-port is down
NW-UP: Network is up
NW-DN: Network is down
NW-SET: Network and AC both are up

Local                            Remote       Connection-Details

================================ ============ ==========================================================================
VPN-ID       EVI-Name      MTU   VPN-ID       Source       Destination                   PE-IP           MTU   Type   NW
================================ ============ ==========================================================================
100          ----          1500  2            xe46.10      --- Single Homed Port ---     1500  AC-NW  NW

Total number of entries are 1
ipi-1.lab.jan1.us.ipa.net#show evpn mpls xconnect tunnel
EVPN-MPLS Network tunnel Entries
Source           Destination      Status        Up/Down       Update        local-evpn-id remote-evpn-id
========================================================================================================      Installed     01:31:06      01:31:06      100           2

Total number of entries are 1

The tunnels are up, installed, and ready for forwarding. We can see the CE macs as mac-ip routes in evpn.

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn vrf BLUE
BGP table version is 1, local router ID is
Status codes: s suppressed, d damped, h history, a add-path, * valid, > best, i - internal,
              l - labeled, S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

[EVPN route type]:[ESI]:[VNID]:[relevent route informantion]
1 - Ethernet Auto-discovery Route
2 - MAC/IP Route
3 - Inclusive Multicast Route
4 - Ethernet Segment Route
5 - Prefix Route

    Network          Next Hop            Metric    LocPrf	Weight     Path  Peer          Encap
* i  [1]:[0]:[2]:[16]          0        100       0    i     MPLS
*>   [1]:[0]:[100]:[16]
                       0        100       32768  i  ----------      MPLS

Total number of prefixes 2

The mac addresses are sent via an EVPN type-2 route between PEs.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
                                                     EVPN MPLS MAC Entries
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc

Total number of entries are : 0

Since this is VPWS there are no macs learned on the device.

[email protected]# run ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=21.531 ms
64 bytes from icmp_seq=1 ttl=64 time=22.124 ms

Success! The CEs can reach each other over the EVPN-VPWS circuit.


Now we’ll build an EVPN-VPLS service. The BGP setup is the same so we’ll focus solely on the differences. The first one being the vpn-id creation.

mac vrf ORANGE
 route-target both evpn-auto-rt
evpn mpls id 1
 host-reachability-protocol evpn-bgp ORANGE

There is no end point defined as a xconnect. All that is necessary is to bind the mac vrf to the evpn vpn id.

interface xe46.100 switchport
 encapsulation dot1q 100
  map vpn-id 1

Again, a switchport defined as an access-if-evpn is necessary. This is then mapped to the vpn-id for the VPLS service. In this case anything coming in with a dot1q tag of 100 will be placed into vpn-id 1.

ipi-1.lab.jan1.us.ipa.net#show evpn mpls mac-table
                                                     EVPN MPLS MAC Entries
VNID       Interface VlanId    In-VlanId Mac-Addr       VTEP-Ip/ESI                    Type            Status     MAC mo
ve AccessPortDesc

1          xe46.100  ----      ----      84c1.c132.5031                    Dynamic Local   -------    0
1          ----      ----      ----      84c1.c132.5032                    Dynamic Remote  -------    0

Total number of entries are : 2

Since this is a VPLS service MACs are learned both locally and remotely. The remote MAC is the MAC of the remote CE. This was learned via EVPN and from the VTEP

ipi-1.lab.jan1.us.ipa.net#show bgp l2vpn evpn mac-ip vrf ORANGE
ESI                            Eth-Tag     Mac-Address    IP-Address                              VNID/LABEL     L3VNID
   Nexthop         GW-Type         Encap
0                              1           84c1:c132:5031 --                                      17             0     --              MPLS
0                              1           84c1:c132:5032 --                                      17             0     --              MPLS

The type-2 routes are populated in the BGP table.

[email protected]# run ping
PING ( 56 data bytes
64 bytes from icmp_seq=0 ttl=64 time=21.894 ms
64 bytes from icmp_seq=1 ttl=64 time=22.159 ms

Success! We have reachability across the service.


IP Infusion is continuing to build their evpn/mpls deployment as well as segment routing. It is exciting to see these feature sets continue to mature as traditional LDP/VPLS deployments move to EVPN/MPLS. If you need assistance on the transition from LDP to segment routing or VPLS to EVPN reach out to IP Architechs.

WEBINAR: ISP Design – Separation of Network Functions

Recently, we recorded a webinar to explain a design concept frequently used by iparchitechs.com to build and migrate WISP, FISP and Telco networks – separation of network functions. It centers around simplification of roles within an ISP network. It also explores the use of lower-cost commodity network equipment to maximize the service area for a given ISP footprint while meeting key requirements like scale, redundancy and capacity.

Topics that were covered include:

  • What are network functions?
  • Design examples for WISP/FISP and Telco
  • Equipment and budget considerations

Here is an example of solving design/operational issues with network function separation:

Link to the webinar and slides:

Webinar: Webinar Recording
Slides: Slide Deck