Hello my friend,
I haven’t been posting for a while, but since now I hope that I’ll be able to blogging further. Due to some reason, I’m no more Nokia’s employee. But now I’ve got amazing chance to take part in development of Vodafone’s network, which I find a very good step further for me. But back to article. Today I’ll speak about Inter-AS BGP/MPLS IP VPN Option C between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR
1 2 3 4 5 | No part of this blogpost could be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical or photocopying, recording, or otherwise, for commercial purposes without the prior permission of the author. |
What is it about?
This type of inter-AS VPN is the most scalable on the one hand, and the most complex on the another. In the previous article I’ve provided the brief comparison of all Inter-AS BGP/MPLS IP VPNs, so you can refer the for more information, but here I’ll remind, how Option C looks like:
The complexity arises due to deep mutual penetration of both ASes (autonomous systems) into each other. In Option-B we were just making BGP-peering between ASBRs (autonomous system border routers) for VPNv4/VPNv6 address family (AFI/SAFI 1/128 and 2/128 respectively) and didn’t care about alignment of IP addresses. In Option-C we establish BGP peering between route reflectors (RR), which are typically far away from the network border. Moreover, you need to activate MPLS at RR in order to build MPLS LSP from RR. At the ASBR level we establish BGP IPv4 labeled unicast (BGP-LU) sessions and announce loopbacks of RRs and PEs (provider edge) routers to each other. If we are speaking about big ISPs, usually it isn’t the problem, because they take part in Internet transit and therefore have often public IPv4 addresses assigned to their loopbacks (or even interconnects). In case of private addressing, it’s necessary to check, if there are no overlapping IP addresses for PE/RR, otherwise you will have to forget BGP/MPLS IP VPN with Option C and will have to look for something else.
What are we going to test?
In this lab we’ll built Inter-AS BGP/MPLS IP VPN (simply, MPLS L3VPN) across two routing domains and will discover the BGP topology and MPLS Label processing.
Topology
Below you can find the physical topology of our lab:
You might spot that it’s a little bit modified, though you have seen similar topology in one of the articles previously (link).
The reason for new topology as I’ve managed to launch 4 virtual routers simultaneously: 2 Nokia (Alcatel-Lucent) VSRs and 2 Cisco IOS XRv, what I find totally amazing as it’s broaden much my possibilities for labbing anywhere and anytime.
The logical topology with IP addressing and some basic routing is just the following pic:
As you can see, we have two routing domains. Each routing domain will represent separate AS. Right now only system interfaces (or loopbacks 0) are advertised inside each routing domain. In one routing domain (Nokia (Alcatel-Lucent) SR OS based) ISIS is used as IGP (in level 2 mode), whereas in another domain (Cisco IOS XR based) OSPF is used (just backbone area 0). All other configuration will be done and explained as usual in the article below. In order to shorten your time, you can use these configuration files that include necessary commands to build the provided physical and logical topologies.
Intra-AS MPLS LSP / Transport layer
The first step to build any kind of MPLS VPN is for sure the establishment of MPLS tunnels (MPLS LSPs), which forms transport layer for all other type of traffic. In case of intra-AS MPLS L3VPN Option C there will be two levels of transport, as we span two separate IGP domains with its own MPLS LSPs in one hierarchical MPLS LSP. So, let’s first focus on the very basic MPLS LSPs:
As you can see from the image, we will use RSVP-TE (link) based MPLS LSP in Nokia domain and LDP (link) in Cisco domain. To turn the service above into reality we need the following configuration:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3#show run |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4#show run |
I won’t go into details for the verification part, please, use the links above.
Just let’s briefly check LFIB at our routers in order see if we have necessary LSPs to system interfaces / loopback 0 IPv4 addresses of peering routers inside IGP domains. Let’s start with Nokia (Alcatel-Lucent) SR OS part first:
|
A:SR1# show router tunnel-table |
So far it looks good. What are about Cisco IOS XR routers?
|
RP/0/0/CPU0:XR3#show mpls forwarding |
Both intra-AS MPLS LSPs are established, so the first milestone in launching BGP/MPLS IP VPN Option C is achieved. So it’s time for us to move further!
Inter-AS MPLS LSP based on BGP-LU / Transport layer
BGP Labeled Unicast (or shortly BGP-LU representing AFI/SAFI 1/4) isn’t something new for us, because we have already used it for IGP-free Data Center (link) and Seamless MPLS (link) configuration in our series. The particular logical topology for this lab from BGP-LU prospective is as follows:
The main thing you should take care of is using of proper IPv4 addresses for BGP sessions establishment and proper next-hop changes at ASBRs in order to get end-to-end reachability from SR1 to XR4. The configuration is as follows:
Pay attention to static route that is necessary to add at Cisco IOS XR ASBR XR3 at BGP/MPLS enabled interface in direction of Nokia SR OS router SR2.
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3#show run |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4#show run |
Again, we have already done the comprehensive explanation of BGP Labeled Unicast (BGP-LU) configuration and verification. There are two things that we are going to check now. The first one is the content of BGP Local-RIB:
|
A:SR1# show router bgp routes label-ipv4 |
The long output above provides you information about MPLS labels assigned for system interface at Nokia SR1 and loopback 0 at Cisco XR4 distributers. Another thing I want to show you is test of end-to-end reachability between SR1 and XR4. Here is the basic ping for that:
|
A:SR1# ping 10.2.255.4 source 10.1.255.1 count 1 |
And here is the packet capture:
The difference in number of packets at the link XR3-XR4 is result of PHP (penultimate hop popping) behavior of MPLS LSP at Cisco (link).
Now we have established end-to-end hierarchical MPLS LSP that spans two separate network domains (Nokia/ISIS/RSVP-TE and Cisco/OSPF/LDP). The latest building block for this solution is MPLS L3VPN itself, which will finalize inter-AS BGP/MPLS IP VPN Option-C.
Inter-AS MPLS L3 VPN / Service layer
As I usually do in my series, I’ll provide you the working solution both for IPv4 and IPv6 addresses, as we have finally reach the time, when IPv6 is starting coming into play. From the service configuration point of view, we have VPRN/VRF instances only at Nokia (Alcatel-Lucent) SR OS SR1 router and Cisco IOS XR XR4 router, and these two routers are interconnected by BGP VPNv4/VPNv6 session to exchange that routing information.
During making configuration don’t forget to enable eBGP-multihop function for your VPNv4/VPNv6 session, as you make external BGP peering, not internal, what we were mainly doing in previous labs (link). Here is the configuration itself:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR4 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3#show run |
Main discussion about configuration of BGP/MPLS IP VPNs was already done, so for the most of details I invite you to read that article (link).
The most interesting thing here is the filtering of prefixes, which are announced between ASes. Typicaly we don’t do it inside one AS, but you will definitely want to do it, when you peer with someone else. We have already done similar filtering previously for standard communities (link). The only difference in extended communities that you need to define community type and explicitely stress in Cisco IOS XR that communities are extended. Everything other should be familiar to you, so we go for checks.
First of all, make sure that you have established BGP for the correct address-family between correct neighbors. At Nokia (Alcatel-Lucent) SR OS side you do it with very basic command that shows you all configured BGP peering:
|
A:SR1# show router bgp summary |
In Cisco IOS XR you have to issue the similar command several times, once per corresponding address family:
|
RP/0/0/CPU0:XR4#show bgp vpnv4 unicast summary |
Now let’s check, if we have routes installed in VPRN/VRF routing tables. For IPv4 address for both vendors being tested (Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR) it looks like as follows:
|
A:SR1# show router 100 route-table ipv4 |
For IPv6 address you just add “ipv6” as a key word:
|
A:SR1# show router 100 route-table ipv6 |
As the routing tables are populated in all the VRFs, the last thing we need to do is to check, what labels are used for this VPN. At Nokia (Alcatel-Lucent) SR OS part we do it by reviewing the information in BGP RIB for VPNv4/VPNv6 address family:
|
A:SR1# show router bgp routes vpn-ipv4 |
What about Cisco IOS XR? The answer is straightforward; we should do the same:
|
RP/0/0/CPU0:XR4#show bgp vpnv4 unicast vrf CUST100 labels |
Now we are sure that inter-AS BGP/MPLS IP VPN option C is configured between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR routes. Now we can make some traffic simulation to see the life of the packet on the wire. Let’s do it! IPv4 comes first:
|
A:SR1# ping router 100 192.168.2.1 source 192.168.1.1 count 1 |
This command results in the following trace:
Though we have discussed similar outputs in previous articles (link), let’s briefly review it again, as it’s vital for understanding of the overall process. Forward path SR1->XR4:
- The first packet is the one that leaves SR1 and approaches SR2. It has 3 labels: the first one (the topmost) is RSVP-TE delivered and points to the egress PE inside first IGP domain, which is BGP next-hop for XR4 loopback’s IP. The second label points to XR4 in global routing table itself and is learned from BGP-LU, whereas the third label (the bottommost) is the service label learned from BGP-VPNv4 and point to customer VRF at XR4.
- SR2 terminates intra-area LSP (RSVP-TE) and pops that label. Then it swaps BGP-LU label to the new one, learned from XR3. Service label remains untouched.
- XR3 is Cisco IOS XR router, so it has PHP behavior that’s why it removes BGP-LU label as XR4 is the directly connected MPLS hop for it. Service label remains the same, as well as at the previous hop.
Backward path XR4->SR1 also show differences between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR in terms of PHP and label processing:
- When the packet is transmitted by XR4 in the direction of XR3 it has 2 labels. There is no LDP labels at the topmost position due to PHP. The topmost label here is BGP-LU label, advertised by XR3, and the bottommost label is service label learned from SR1.
- XR3 swaps BGP-LU label to the new one and sends the packet further to SR2 over BGP/MPLS enabled link.
- SR2 in its turn swaps BGP-LU label and adds RSVP-TE label in order to reach SR1 inside its IGP domain, what results in 3 labels in total
Here is the example for IPv6:
|
A:SR1# ping router 100 fc00::192:168:2:1 source fc00::192:168:1:1 count 1 |
And its trace:
That’s it. The final configuration you can find in the following links:
Lessons learned
In the beginning I’ve tried to use SPRING (segment routing) as a MPLS transport technology in one of the domains, but it was unsuccessful. I’ve received the following syslog message in Cisco IOS XRv:
|
ROUTING-RIB-3-LABEL_ERR_ADD |
One of the first links, which I have found, led me to the description of the bug at Cisco’s website (link). Here is the explanation:
If the router has learned a label from BGP Labeled-Unicast in addition to the ISIS segment-routing label a “local-label conflict” can occur leaving the forwarding table out of sync with the routing table for the tunnel destination prefix.
Well, I’ve started to thinking about the root cause of the issue. After having spent some time, I’ve realized, why it so. Take a look at LDP binding for prefix of XR4 and BGP label announced to SR2 from XR3 (Cisco IOS XR router) prospective:
|
RP/0/0/CPU0:XR3#show mpls forwarding |
Both labels have the same value, or more precisely, it’s just the same label. Both Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR implements per platform address space, what means that independently of the ingress interface the MPLS label ultimately is mapped to one egress interface (and destination prefix). In case of SPRING (Segment routing) we separate MPLS label space for the prefixes to be used solely for segment routing. In Nokia (Alcatel-Lucent) SR OS there is no preconfigure address space and we define it manually ourselves, whereas in Cisco IOS XR this value lays in the range from 16000 till 23999. In Cisco IOS XR the labels allocated by BGP starts from the value of 24000. It results in two separated values for the same prefix (destination IP address). XR4 announces its loopback 0 via OSPF (including Segment Routing SID) and internal BGP-LU (including MPLS label). Segment routing has higher preference than BGP-LU in terms of installing label in LFIB, what results in SPRIN label being installed there. Meanwhile the MPLS label learned from BGP is announced further to SR2 effectively breaking end-to-end MPLS LSP between SR2 and XR4, due to the fact that XR3 doesn’t know what to do with incoming from SR2 packet, as there is no such label in XR3’s LFIB
In Nokia (Alcatel-Lucent) SR OS we can modify the preference for the SPRING labels, so that iBGP MPLS label is installed in the LFIB. But it doesn’t solve the problem, rather moves it from PE to P router, which doesn’t BGP and has only SPRING labels
Conclusion
We have covered all the possibly options for BGP/MPLS IP VPNs in terms of using technologies. There were: intra-AS L3VPN (link), inter-AS option B L3VPN (link) and now inter-AS option C L3VPN. I hope you enjoy these articles and they are useful bookmarks for the references.
Meanwhile I’m moving further and will cover another topics regarding joint operation of Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR in the communication service provider (CSP) networks.
Support us
BR,
Anton Karneliuk
