Hello my friend,
We continue our journey across technologies, which form SDN in Service Provider networks. Today we are going to review new extension to BGP that is called BGP link state (BGP-LS).
1 2 3 4 5 | No part of this blogpost could be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical or photocopying, recording, or otherwise, for commercial purposes without the prior permission of the author. |
Brief overview
The main advantage of links-state routing protocols over distance-vector routing protocols is that they distribute topology information rather than routes. It results in the fact that each router in certain domain (area for OSPF and level for ISIS) has the same set of topology information and therefore build its own routing table based on SPF graph. TE (traffic engineering) extension to OSPF/ISIS adds more information about links (utilization, affinity, SRLG, etc), which is used for MPLS-TE based on RSVP or Segment Routing. But one of the main limitations of current MPLS traffic engineering implementations is that MPLS-TE information is distributed only within certain domain (again, area for OSPF and level for ISIS). Inter-area/inter-level communication could be achieved, but with classical RSVP/MPLS-TE approach we need to define loose explicit path for MPLS LSP. It means that we don’t have any flexibility provided TE attributes. To overcome this issue, we need some central instance (“overmind”) that will have visibility in each and every routing domain and will be able to calculate end-to-end MPLS LSP taking into account all constraints.
Such overmind is SDN controller that collects information from all routing domains, so that it can calculate the necessary MPLS LSP and instructs network elements, which are typically PE routers, how to get to the necessary destination by confining there MPLS TE tunnel. This tunnel can be build using either RSVP or Segment Routing.
The last piece in this puzzle is the technology, how SDN controller receives topology information from peers. There are different options, but the most scalable is by using BGP-LS. Typically, all the routers in service provide network have BGP so that they need just to add new address family and make so small configurations.
What are we going to test?
In this lab we are going to configure BGP-LS session between Nokia (Alcatel-Lucent) SR OS router and Cisco IOS XR router and distribute ISIS links-state database (LSDB) through this session.
Software version
For tests in this lab I use the following versions of software for routers:
- Nokia (Alcatel-Lucent) SR OS 14.0.R4 (on SR2)
- Nokia (Alcatel-Lucent) SR OS 15.0.R2 (on SR1)
- Cisco IOS XRv 6.1.2
BGP-LS address family for BGP isn’t supported in Nokia (Alcatel-Lucent) SR OS 14.0.R4 therefore I use the newer version at the SR1, because it has to speak this new address family.
Topology
Physical topology doesn’t change at all and we continue to use 2 Nokia (Alcatel-Lucent) VSR and 2 Cisco IOS XRv routers:
For more details about building lab I advise you to read the corresponding article.
Logical topology is a bit different than it was in a couple of previous labs:
Actually we have only 3 routers, which forms data plane. They speak with each other ISIS for IPv4 and for IPv6. The 4th router emulates SDN controller and he should receive all link-state topology information from data plane router.
Initial router configuration is here: 086_config_initial_linux_p1 086_config_initial_sr1_p1 086_config_initial_sr2_p1 086_config_initial_xr3_p1 086_config_initial_xr4_p1
Configuration of BGP-LS with distribution of LS information at Cisco IOS XR
Actually we’ll have two cases in this article. The first one is exactly what we have shown at the image above, where XR3 distributes its ISIS LSDB over BGP-LS to SR1. In this case SR1 emulates SDN controller. Let’s check the routing table at Cisco IOS XR router XR3:
|
RP/0/0/CPU0:XR3#show route ipv4 isis |
Ok, we see that routing table is filled in with proper information. But as I’ve said previously, the main advantage of the link-state routing protocols is that each router has full topology information rather than just routing table from neighbors. Let’s check the ISIS LSDB at the same Cisco IOS XR router XR3:
|
RP/0/0/CPU0:XR3#show isis database verbose |
We see that we have two topologies in ISIS (one for IPv4 and one for IPv6) and each topology has its own IS adjacency and prefixes. Everything is ready to configure BGP-LS itself.
From configuration point of view, we need to configure new address family (BGP link-state) for BGP session between Nokia SR1 and Cisco XR3 and to configure XR3 to distribute its ISIS LSDB to SR1 through this session. We form BGP session between loopbacks so some static routes are necessary as I don’t have any IGP running between Nokia (Alcatel-Lucent) SR OS router SR1 and Cisco IOS XR router XR3:
If you need to refresh static routing configuration in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR, I advise you to read the corresponding article.
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
As you see, the configuration is quite simple. XR3 distributes its full LSDP to SDN controller, so you don’t need to configure BGP-LS address family (and in this case BGP in general) at SR2 and XR4. To distinguish different IGP domains we use “instance-id” during configuration of BGP-LS under ISIS. It isn’t necessary for our case but it’s good to have it.
BGP-LS address-family isn’t available in Nokia (Alcatel-Lucent) SR OS 14.0.R4 so I use 15.0.R2 at SR1.
First of all, let’s check if BGP-LS peering is established between Nokia (Alcatel-Lucent) VSR router SR1 and Cisco IOS XRv router XR3:
|
A:SR1# show router bgp summary |
As you see, XR3 sends 21 BGP-LS prefixes to SR1 and SR1 doesn’t send anything to XR3, what is fully correct. The question that might arise in your mind now is “why we send 21 prefix?”
The question is fully legitimate, because we have only 3 LSP in ISIS LSDB (one per each router in ISIS process). Let’s analyse, whet we have in BGP RIB for BGP link-state address family. Let’s start with Cisco IOS XR router XR3, as it grabs ISIS information:
|
RP/0/0/CPU0:XR3#show bgp link-state link-state |
You see, we have 21 prefix in XR3 BGP RIB, so this number is correct. But why we have so much prefixes? Let’s analyse them. The first important point is that we have 3 different route types:
- [V] represents the node in ISIS topology. We have 3 such routes.
- [E] represents the link between nodes. We have 12 such routes: each router is connected to two another in 2 topologies (IPv4 and IPv6) therefore we have 2*2 = 4 links per router.
- [T] represents prefixes. We have 6 such routes, as each router announces one its IPv4 and one IPv6 prefix.
Basically what router do is extracts different TLVs from ISIS LSP and generate different BGP-LS routes based on TLV number. Though you can read and guess the prefix meaning just by its name, we’ll see the details of each route.
We start with [V] that is node route (I just copy the route how its show in BGP RIB):
|
RP/0/0/CPU0:XR3#show bgp link-state link-state [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]]/328 |
The most important information is in the end, where we see which ISIS topology are configured (0 for IPv4 unicast and 2 for IPv6 unicast), node hostname, TE-ID (traffic engineering) and ISIS area.
The next route we check is [E], which is link route:
|
P/0/0/CPU0:XR3#show bgp link-state link-state [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]]/568 |
We don’t have any RSVP, MPLS-TE or Segment Routing parameters configured, that’s why we have only IGP metric as parameters for this link.
And the last type of BGP-LS routes is [T], which represents the prefixes sent originated by certain router:
|
RP/0/0/CPU0:XR3#show bgp link-state link-state [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][P[p10.0.0.44/32]]/400 |
Here we in general don’t have much interesting information.
Now let’s verify, how we see these prefixes in Nokia (Alcatel-Lucent) SR OS routers:
|
A:SR1# show router bgp routes bgp-ls |
The very first point is that Nokia (Alcatel-Lucent) SR OS has lower amount of routes than Cisco (18 vs 21). Careful review show of SR1 BGP RIB for BGP-LS shows that Nokia VSR doesn’t IPv6 prefixes routes (3 of 6 [T] routes in Cisco). If I chose details, I can’t see IPv6 prefix there, only IPv4. I believe it would be fixed (or already fixed) in newer version of Nokia (Alcatel-Lucent) SR OS. In the rest information is much the same and even user friendly, as BGP-LS routes are splitted into 3 categories and are named as Node, Link and IPv4.
To see details of BGP-LS routes in Nokia (Alcatel-Lucent) SR OS use “show router bgp routes bgp-ls hunt” command.
Adding more value to BGP-LS
BGP-LS is one of the building block of SDN that’s why pure routing metrics are neither interesting nor very useful. Let’s enable Segment Routing in our ISIS network and then analyze changes in BGP-LS prefixes:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
Now we have much more information in our ISIS LSDB, which is related to segment routing. You see, we haven’t configured anything additional to commands for distributing ISIS to BGP-LS. But if we see into BGP-LS NLRI (network layer reachability information – prefix, actually), we see new information there. For link prefix [E] we have:
|
P/0/0/CPU0:XR3#show bgp link-state link-state [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[i10.22.33.22][n10.22.33.33]]/696 |
We see new attribute in this BGP NLRI, which is ADJ-SID label generated by Nokia (Alcatel-Lucent) SR OS router SR2.
Now we check [T] prefix:
|
P/0/0/CPU0:XR3#show bgp link-state link-state [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][P[p10.0.0.22/32]]/400 |
Here we have prefix-SID of the router, which is used for generating of the label for the node itself. As a brief summary, now the value of BGP-LS has increased because SDN controller has more useful information that can be used for building MPLS LSP between PEs later.
Final configuration files for this case are here: 086_config_final_xr3_p1 086_config_final_xr4_p1 086_config_final_sr1_p1 086_config_final_sr2_p1
Configuration of BGP-LS with distribution of LS information at Nokia SR OS
The second case is to change SR1 and XR3 with their functions. Nokia (Alcatel-Lucent) VSR (SR 7750) will distribute its ISIS LSDB through BGP-LS to XR3. So the topology now looks a bit different:
The initial configuration of the devices is here: 086_config_initial_linux_p2 086_config_initial_sr1_p2 086_config_initial_sr2_p2 086_config_initial_xr3_p2 086_config_initial_xr4_p2
It’s just the same as it was in the very beginning for another case from functionality point of view. Now we do the same configuration as we’ve done before:
- Static routing between SR1 and XR3
- BGP-LS between SR1 and XR3
- Distribution of ISIS LSDB to BGP-LS at SR1
- Segment routing in SR1, SR2, XR4
Here we go:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
BGP-LS-ID 167772171 is decimal for IPv4 address 10.0.0.11.
It took me enormous time in order to configure Nokia (Alcatel-Lucent) SR OS to start sending BGP-LS NLRI to Cisco. Let’s check the BGP peering for BGP link-state address family:
|
A:SR1# show router bgp summary |
Well, now we have more than 2 times lower amount of BGP-LS prefixes than we have had previously, when we exported them from Cisco IOS XR router. We can assume Nokia (Alcate-Lucent) SR OS doesn’t export IPv6 prefixes as it doesn’t understand them. But in that case we should have at least 18 prefixes…
As you remember, we can’t see local routes in BGP RIB in Nokia (Alcatel-Lucent) SR OS, so we have to analyze BGP-LS NLRI at Cisco IOS XR router XR3:
|
RP/0/0/CPU0:XR3# show bgp link-state link-state |
First of all, we see full absence of any IPv6-related information, both for address prefixes [T] and links [E]. We still have all node [V] prefixes and IPv4 address prefixes [T]. What is really strange for me is that links [E] prefixes aren’t fully distributed: SR1 exports links from SR1 and SR2 (both are Nokia (Alcatel-Lucent) SR OS routers) and doesn’t export links from XR4 (Cisco IOS XR router). I’ve compared IS-neighbor fields in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR and the only difference I’ve found is that in Cisco we have two Segment Routing MPLS labels, whereas in Nokia we have only one. For example, here we have parts of ISIS LSP from SR2 and XR4, which describes link SR2-XR4:
|
RP/0/0/CPU0:XR4#show isis database level 2 SR2.00-00 verbose |
When I was writing this part I’ve spotted interesting difference that I haven’t paid attention previously. As you might see, Nokia (Alcatel-Lucent) VSR (SR 7750) announces all IP addresses of its interfaces, whereas Cisco IOS XRv 9000 (ASR 9000) announces only IP addreses of passive interfaces. I’m speaking now not about extended IP reachability TLVs, where information is similar. I’m speaking about interfaces TLV. Let’s change a bout configuration of our ISIS so that all IP addresses are announced:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
| A:SR2>edit-cfg# candidate view ========================= configure router isis no advertise-passive-only exit exit exit ========================= |
RP/0/0/CPU0:XR4(config)#show conf |
What do you think? Does it help to solve our issue? The only way is to check BGP-LS RIB at Cisco IOS XR router XR3 and to see, what Nokia (Alcatel-Lucent) SR OS router SR1 has announced to it:
|
RP/0/0/CPU0:XR3#show bgp link-state link-state |
We have 8 prefixes more than earlier:
- 6 of them are address prefixes [T] from links between routers.
- 2 of them are link prefixes [E] generated by XR4.
Now our SDN controller has full topology information from our network.
The final configuration files for the second case are here: 086_config_final_sr1_p2 086_config_final_sr2_p2 086_config_final_xr3_p2 086_config_final_xr4_p2
Lessons learned
Documentation is always a key, but in this article I was like a blind cat, because Nokia documentation for BGP-LS is very bad, at least official “Unicast Routing Protocols Guide” available on Nokia website. It doesn’t provide example hot configure distribution of TE-DB through BGP-LS, only separated commands not related to each other. So I had to trying all the commands I can find in this guide and CLI to make it working and though I made it, I took me enormous amount of time.
Dear Nokia colleagues, please, make your documentation consistent and clear. I understand that you want to sell your services for hundreds of thousands of dollars, but respect your customers.
Conclusion
BGP-LS provides you possibility to distribute your routing topology with all traffic engineering attributes (LSDB and TED) to SDN controller so that It can build in his brain full view of all your network including all inter-area cases. It’s the first step in provisioning automatic MPLS LSP within the routing domain (area for OSPF and level for ISIS) or between them. BGP-LS is quite new technology and I’m pretty sure, its use cases will be further expanded. Take care and good bye!
Support us
BR,
Anton Karneliuk
Hi Anton, superb post. Kudos to your efforts. I have a doubt here that, is the NOKIA SR.20.7 has the capability of advertising BGP-LS. Because I’m facing BGP session establishment issue between NOKIA.vSIM (IPV4 family) and ODL(BGP-LS family)controller. Do you have any idea?
Hello Vignesh,
Yes, Nokia SR OS 20.7.R1 has the BGP-LS. There might be issues in terms of building the session with ODL from the interoperability perspective, but generally that is possible.
Cheers,
Anton