Hello my friend,
After we have reviewed Seamless MPLS concept, it might be not so interesting to speak about BGP/MPLS IP VPNs again. Nevertheless, Inter-AS MPLS VPNs is a separate topic, which is very interesting and can add useful solutions to your network. Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR have these options for you.
1 2 3 4 5 | No part of this blogpost could be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical or photocopying, recording, or otherwise, for commercial purposes without the prior permission of the author. |
Brief overview
There are three options for inter-AS VPNs:
Inter-AS MPLS VPN option A is the easiest and the least scalable. There is no MPLS labels’ exchange between different autonomous systems. Instead of this, ASBR routers establish eBGP sessions on per VRF basis. If there are 100 clients that have inter-AS VPNs, then 100 eBGP sessions must be established between a pair of ASBR routers. So users traffic passes link between ASBRs as normal IP packets, which are completely unlabeled:
Inter-AS MPLS VPN option B has balance of scalability and integration of the networks. On the one hand, ASBRs in different autonomous system exchange MPLS labels, so careful consideration regarding policies and route filtering at ASBRs must be done. On the other hand, we have end-to-end MPLS LSP. In my opinion, it’s the best option for Inter-AS VPNs and I prefer to use it in the most of cases. It looks like as follows:
Inter-AS MPLS VPN option C is the most scalable solution. But it also requires the deepest integration of two ASes, because not only MPLS labels are exchanged between two services. There are also IP addresses (usually loopback or system) of PE routers, which build VPN, and BGP route reflectors. Take a look at the picture below:
Option A isn’t something specific and you can read article about usual BGP/MPLS IP VPN, where we were configuring MPLS VPN and BGP as PE-CE routing protocol. I won’t write separate article about it. Option B will be covered in this article and option C will be covered in the next article.
What are we going to test?
There are two options how we inter-AS MPLS VPN option B can be implemented. The first one is just it’s drawn at the image above. The second option is a little bit similar to Seamless MPLS and is used inside big network of one company, not between different companies. So we are going to test both of the solutions.
Topology
We are reverting back to the topology, which we have used previously:
As I’m going to cover two cases, there will be two topologies. In general, they will be the same, but there still be some differences. That’s why I’m going to provide both of the separately. The first topology is as follows:
On the left side there is we have two Nokia (Alcatel-Lucent) SR OS routers SR1 and SR2, which represent the first service provider (ISP A) with autonomous system (AS) 65100. OSPF is configured as IGP there with announcement of loopbacks. SPRING (Segment Routing) is used for establishment of MPLS transport LSPs. Cisco IOS XR router XR3 solely represents another service provider (ISP B) with autonomous system (AS) 65200 and is connected to SR2. There is no dynamic routing between them.
Here you can find initial configuration files: linux_initial sr1_initial sr2_initial xr3_initial
Case 1 – BGP configuration for Inter-AS MPLS VPN option B
The main focus of the configuration in this article is BGP for service layer and partially for transport layer. Here is the scheme, how BGP will be configured between Nokia (Alcatel-Lucent SR OS routers in Cisco IOS XR router in order to establish Inter-AS MPLS VPN option B:
Here is the configuration for this topology:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | |
|
A:SR2>edit-cfg# candidate view |
Our main focus in the current configuration are routers SR2 and XR, because at SR1 we have simple configuration for BGP/MPLS IP VPN, which we have reviewed previously (link). At SR2 the main difference from intra-AS VPN is “enable-inter-as-vpn”, which activates possibility for inter-AS VPNs.
In Cisco IOS XR the main difference from intra-AS is activation of MPLS into BGP so that BGP VPNv4/VPNv6 labels (actually, service labels) are used for traffic forwarding as transport labels at this particular link. Static route at XR3 is needed for proper label allocation. (link)
Also what is worth to be mentioned is next-hop processing. By default in BGP, routers changes the next hop its IP address, when they send routes for eBGP neighbors, but retain original next-hop, when they send updates for iBGP neighbors. Don’t forget to change this behavior, so that next-hop is changed, when BGP updates are sent to iBGP neighbors.
Pay attention that we activate both VPNv4 and VPNv6 contexts in our inter-AS MPLS VPN option B.
The first check that we should perform is to assure that BGP is established in the necessary address family at Nokia (Alcatel-Lucent) SR OS router SR2 and Cisco IOS XR router XR3:
|
A:SR2# show router bgp summary |
The next step is to check MPLS forwarding plane between SR2 and XR3:
|
A:SR2# show router tunnel-table |
You see that at Nokia (Alcatel-Lucent) SR OS router SR2 you see only built MPLS LSP to SR1 because we have it based on SPRING information. There is no built MPLS LSP towards XR3, because there is no any LSP so far. Let’s change this.
The next step would be configuration of VPRN/VRF contexts for VPN service itself. This configuration is to be done at SR1 and XR3. At SR2 we aren’t going to configure anything.
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
For more information about BGP/MPLS IP VPN configuration refer to the corresponding article.
As usual we start our checks from Nokia (Alcatel-Lucent) SR OS router, which in this case is SR1. The first check is to see, what we have in the routing table in VPRN 65300 for IPv4 and IPv6:
|
A:SR1# show router 65300 route-table ipv4 |
You see that SR1 knows about prefixes learned from XR3, and they point to the next hop via SPRING MPLS LSP. The same next-hop is used both for IPv4 and IPv6. Here are the labels that are used at SR1 for those prefixes:
|
A:SR1# show router bgp routes vpn-ipv4 | match expression “Label|None|100” |
Now let’s shift to SR2. Here we don’t have any VPRN configured, but still we have VPNv4/VPNv6 routes in BGP RIB:
|
A:SR2# show router bgp routes vpn-ipv4 | match expression “Label|None” |
The last point of check is Cisco IOS XR router XR3, where we can see its labels for MPLS service:
|
RP/0/0/CPU0:XR3#show bgp vpnv4 uni vrf INTER_AS_VPN labels |
Let’s make just a simple test, how traffic is delivered between SR1 and XR3:
|
A:SR1# ping router 65300 192.168.2.1 source 192.168.1.1 count 1 |
Let’s see what is happening behind the scene at the packet level. Here is the trace from WireShark:
Here is the explanation of the packet life for forward paths:
- Initially packet is labeled with two labels: MPLS transport label delivered from SPRING and MPLS service label delivered from BGP.
- At the next hop (SR2) transport label is popped and internal service label is swapped.
- When XR3 receives this packet, it pops the only MPLS label and put the packet to the correct context.
At the backward path the actions are just opposite:
- XR3 pushes the single label at the packet, when it sends it towards VRF at SR1.
- SR2 swaps that label and adds additional MPLS label in order to reach SR1.
- SR1 receives the packet with two MPLS labels, pops both of them and puts the packet into correct VPRN.
Here is the trace for IPv6:
|
A:SR1# ping router 65300 fc00::192:168:2:1 source fc00::192:168:1:1 count 1 |
And Wireshark overview of that process:
Configuration files for the first case of Inter AS option B between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR are here: sr1_case1_final sr2_case1_final xr3_case1_final
Case 2 – BGP configuration for Inter-AS MPLS VPN option B with MPLS LSPs in both domains
This variant of inter AS MPLS VPN Option B I’ve seen recently in one service provider’s network. Take a look at the image below:
The main difference here is that BGP is used in service layer only. All the packets have separate transport label derived from LDP / RSVP-TE / SPRING or BGP-LU. What we are doing in our topology is some modification between SR2 and XR3:
- link between SR2 and XR3, as well as loopback0 at XR3 is added to area 0.0.0.1;
- SPRING is configured for this area as well;
- eBGP between SR2 and XR3 is built using system/loopback0 interfaces (corresponding modification to BGP is done);
Let’s see, which configuration should be done to achieve this:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR2 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
What we are doing here at our Nokia (Alcatel-Lucent) SR OS router SR is just enabling OSPF and changing BGP, as SPRING is already activated there.
At Cisco IOS XR router XR3 we configure OSPF, enable SPRING and change BGP configuration. Also we remove previous static route and MPLS operation at interface under BGP context.
Don’t forget to add multihop option for eBGP session built between loopbacks.
Let’s check what we have in LFIB at our Nokia (Alcatel-Lucent) SR OS router SR2:
|
A:SR2# show router tunnel-table detail |
You see that we have new MPLS LSP towards 172.16.255.33/32, what is IPv4 address of loopback0 interface at XR3. You might spot that label value is 3, what means “implicit-null” label. We have discussed this behavior previously during configuration of LDP, so we won’t discuss it again.
In Cisco IOS XR router XR3 we have the following LFIB:
|
RP/0/0/CPU0:XR3#show mpls forwarding |
You see that MPLS labels, which are learned from SPRING, are installed there, so it means that data plane is established. Now we need to look at the next hop in BGP VPNv4/VPNv6 routes to be sure that is announced with IPv4 addresses, for which we have MPLS labels. There are no changes at SR1 or at SR2 in the direction of SR1, that’s why we need look at this information only at SR2 and XR3:
|
A:SR2# show router bgp routes vpn-ipv4 |
Well, it looks like we have enough topology information to test built VPN:
|
A:SR1# ping router 65300 192.168.2.1 source 192.168.1.1 count 1 |
It works, and it’s very good. But what is the difference comparing to the first case? The answer you can see, if you launch Wireshark to discover packet structure:
Due to PHP operation at XR3 you see no labels at the second line, but it’s this “implicit null” label. Nevertheless, on the backward path you see MPLS transport labels at both links. In the same time MPLS service label is also changed at SR2.
The basic rule described in “MPLS in the SDN era” says that new label is generated each time, when next-hop is changed. Remember it and you won’t have problems with understanding of MPLS label processing.
For IPv6 we have similar trace:
|
A:SR1# ping router 65300 fc00::192:168:2:1 source fc00::192:168:1:1 count 1 |
Final configuration for the second case of inter AS MPLS VPN option B is here (as there were changes only at SR2 and XR3, I provide configuration files only for them): xr3_case2_final sr2_case2_final
Lessons learned
I’m was a little bit surprised, when I met the second case implemented in the network. On the one hand it looks nice and works in general well. But there is one drawback in its default operation. If you have all routers from AS 65200 being connected to ASBR router in AS 65100 (let’s assume that OSPF area 0.0.0.1 is access network or mobile backhaul and are 0.0.0.0 is core), by default you won’t have BGP routes from one router in AS 65200 at another router in the same AS due to AS_PATH path attribute. So you have to configure “as-override” and “next-hop-unchanged” at ASBR to fix it. But you should do it conditionally, because you still need to change next-hop for routes learned from AS 65100.
Conclusion
MPLS world is really huge and you have virtually endless possibilities to delivery service. There is no single right choice for all cases. Our job as network engineers and architects is to find the most suitable network solution that is able to deliver all necessary services, whereas being scalable, modular and flexible to cover upcoming challenges. MPLS is definitely platform for such solutions. And inter AS MPLS VPNs helps you to extend services beyond your network. Take care and good bye!
Support us
BR,
Anton Karneliuk