SDN sandbox 4. BGP-LS in Nokia SR OS and Cisco IOS XR.

Anton Karneliuk

7 years ago

Hello my friend,

We continue our journey across technologies, which form SDN in Service Provider networks. Today we are going to review new extension to BGP that is called BGP link state (BGP-LS).

1
2
3
4
5

No part of this blogpost could be reproduced, stored in a
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.

Brief overview

The main advantage of links-state routing protocols over distance-vector routing protocols is that they distribute topology information rather than routes. It results in the fact that each router in certain domain (area for OSPF and level for ISIS) has the same set of topology information and therefore build its own routing table based on SPF graph. TE (traffic engineering) extension to OSPF/ISIS adds more information about links (utilization, affinity, SRLG, etc), which is used for MPLS-TE based on RSVP or Segment Routing. But one of the main limitations of current MPLS traffic engineering implementations is that MPLS-TE information is distributed only within certain domain (again, area for OSPF and level for ISIS). Inter-area/inter-level communication could be achieved, but with classical RSVP/MPLS-TE approach we need to define loose explicit path for MPLS LSP. It means that we don’t have any flexibility provided TE attributes. To overcome this issue, we need some central instance (“overmind”) that will have visibility in each and every routing domain and will be able to calculate end-to-end MPLS LSP taking into account all constraints.

Such overmind is SDN controller that collects information from all routing domains, so that it can calculate the necessary MPLS LSP and instructs network elements, which are typically PE routers, how to get to the necessary destination by confining there MPLS TE tunnel. This tunnel can be build using either RSVP or Segment Routing.

The last piece in this puzzle is the technology, how SDN controller receives topology information from peers. There are different options, but the most scalable is by using BGP-LS. Typically, all the routers in service provide network have BGP so that they need just to add new address family and make so small configurations.

What are we going to test?

In this lab we are going to configure BGP-LS session between Nokia (Alcatel-Lucent) SR OS router and Cisco IOS XR router and distribute ISIS links-state database (LSDB) through this session.

Software version

For tests in this lab I use the following versions of software for routers:

Nokia (Alcatel-Lucent) SR OS 14.0.R4 (on SR2)
Nokia (Alcatel-Lucent) SR OS 15.0.R2 (on SR1)
Cisco IOS XRv 6.1.2

BGP-LS address family for BGP isn’t supported in Nokia (Alcatel-Lucent) SR OS 14.0.R4 therefore I use the newer version at the SR1, because it has to speak this new address family.

Topology

Physical topology doesn’t change at all and we continue to use 2 Nokia (Alcatel-Lucent) VSR and 2 Cisco IOS XRv routers:

For more details about building lab I advise you to read the corresponding article.

Logical topology is a bit different than it was in a couple of previous labs:

Actually we have only 3 routers, which forms data plane. They speak with each other ISIS for IPv4 and for IPv6. The 4^th router emulates SDN controller and he should receive all link-state topology information from data plane router.

Initial router configuration is here: 086_config_initial_linux_p1 086_config_initial_sr1_p1 086_config_initial_sr2_p1 086_config_initial_xr3_p1 086_config_initial_xr4_p1

Configuration of BGP-LS with distribution of LS information at Cisco IOS XR

Actually we’ll have two cases in this article. The first one is exactly what we have shown at the image above, where XR3 distributes its ISIS LSDB over BGP-LS to SR1. In this case SR1 emulates SDN controller. Let’s check the routing table at Cisco IOS XR router XR3:

RP/0/0/CPU0:XR3#show route ipv4 isis
i L2 10.0.0.22/32 [115/10] via 10.22.33.22, 02:02:05, GigabitEthernet0/0/0/0.23
i L2 10.0.0.44/32 [115/10] via 10.33.44.44, 01:59:51, GigabitEthernet0/0/0/0.34
!
!
RP/0/0/CPU0:XR3#show route ipv6 isis
i L2 fc00::10:0:0:22/128
. [115/10] via fe80::22, 02:02:09, GigabitEthernet0/0/0/0.23
i L2 fc00::10:0:0:44/128
. [115/10] via fe80::44, 01:59:55, GigabitEthernet0/0/0/0.34

Ok, we see that routing table is filled in with proper information. But as I’ve said previously, the main advantage of the link-state routing protocols is that each router has full topology information rather than just routing table from neighbors. Let’s check the ISIS LSDB at the same Cisco IOS XR router XR3:

RP/0/0/CPU0:XR3#show isis database verbose
IS-IS CORE (Level-2) Link State Database
LSPID                 LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL
SR2.00-00             0x0000001f   0xe564        901           0/0/0
. Area Address:   49.0000
. NLPID:          0xcc
. NLPID:          0x8e
. MT:             Standard (IPv4 Unicast)
. MT:             IPv6 Unicast                                 0/0/0
. Hostname: SR2
. Router ID:    10.0.0.22
. IP Address: 10.0.0.22
. IP Address: 10.22.33.22
. IP Address: 10.22.44.22
. IPv6 Address:   fc00::10:0:0:22
. Metric: 10         IS-Extended XR3.00
.   Interface IP Address: 10.22.33.22
.   Neighbor IP Address: 10.22.33.33
. Metric: 10         MT (IPv6 Unicast) IS-Extended XR3.00
. Metric: 10         IS-Extended XR4.00
.   Interface IP Address: 10.22.44.22
.   Neighbor IP Address: 10.22.44.44
. Metric: 10         MT (IPv6 Unicast) IS-Extended XR4.00
. Metric: 0          IP-Extended 10.0.0.22/32
. Metric: 0    MT (IPv6 Unicast) IPv6 fc00::10:0:0:22/128
XR3.00-00 *           0x00000018   0x8df3        1098          0/0/0
. Area Address: 49.0000
. NLPID:          0xcc
. NLPID:          0x8e
. MT:             Standard (IPv4 Unicast)
. MT:             IPv6 Unicast                                 0/0/0
. Hostname: XR3
. IP Address:     10.0.0.33
. IPv6 Address:   fc00::10:0:0:33
. Metric: 10         IS-Extended SR2.00
. Metric: 10         IS-Extended XR4.00
. Metric: 0          IP-Extended 10.0.0.33/32
. Metric: 10 MT (IPv6 Unicast) IS-Extended SR2.00
. Metric: 10 MT (IPv6 Unicast) IS-Extended XR4.00
. Metric: 0    MT (IPv6 Unicast) IPv6 fc00::10:0:0:33/128
XR4.00-00 0x00000014   0x97c5        1132          0/0/0
. Area Address:   49.0000
. NLPID:          0xcc
. NLPID:          0x8e
. MT:             Standard (IPv4 Unicast)
. MT:             IPv6 Unicast                                 0/0/0
. Hostname:       XR4
. IP Address:     10.0.0.44
. IPv6 Address:   fc00::10:0:0:44
. Metric: 10         IS-Extended SR2.00
. Metric: 10 IS-Extended XR3.00
. Metric: 0          IP-Extended 10.0.0.44/32
. Metric: 10 MT (IPv6 Unicast) IS-Extended SR2.00
. Metric: 10 MT (IPv6 Unicast) IS-Extended XR3.00
. Metric: 0          MT (IPv6 Unicast) IPv6 fc00::10:0:0:44/128

We see that we have two topologies in ISIS (one for IPv4 and one for IPv6) and each topology has its own IS adjacency and prefixes. Everything is ready to configure BGP-LS itself.

From configuration point of view, we need to configure new address family (BGP link-state) for BGP session between Nokia SR1 and Cisco XR3 and to configure XR3 to distribute its ISIS LSDB to SR1 through this session. We form BGP session between loopbacks so some static routes are necessary as I don’t have any IGP running between Nokia (Alcatel-Lucent) SR OS router SR1 and Cisco IOS XR router XR3:

If you need to refresh static routing configuration in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR, I advise you to read the corresponding article.

Nokia (Alcatel-Lucent) SR OS	Cisco IOS XR
SR1	XR3
A:SR1>edit-cfg# candidate view ========================= configure router static-route-entry 10.0.0.33/32 next-hop 10.11.33.33 no shutdown exit exit autonomous-system 65000 bgp group “SDN” family bgp-ls peer-as 65000 local-address 10.0.0.11 neighbor 10.0.0.33 exit exit no shutdown exit exit exit =========================	RP/0/0/CPU0:XR3(config)#show conf ! router static address-family ipv4 unicast 10.0.0.11/32 GigabitEthernet0/0/0/0.13 10.11.33.11 ! ! router isis CORE distribute bgp-ls instance-id 10 ! router bgp 65000 bgp router-id 10.0.0.33 bgp log neighbor changes detail address-family link-state link-state domain-distinguisher 65000:10.0.0.33 ! neighbor 10.0.0.11 remote-as 65000 update-source Loopback0 address-family link-state link-state ! ! ! end

As you see, the configuration is quite simple. XR3 distributes its full LSDP to SDN controller, so you don’t need to configure BGP-LS address family (and in this case BGP in general) at SR2 and XR4. To distinguish different IGP domains we use “instance-id” during configuration of BGP-LS under ISIS. It isn’t necessary for our case but it’s good to have it.

BGP-LS address-family isn’t available in Nokia (Alcatel-Lucent) SR OS 14.0.R4 so I use 15.0.R2 at SR1.

First of all, let’s check if BGP-LS peering is established between Nokia (Alcatel-Lucent) VSR router SR1 and Cisco IOS XRv router XR3:

A:SR1# show router bgp summary
===============================================================================
BGP Router ID:10.0.0.11        AS:65000       Local AS:65000
===============================================================================
BGP Summary
===============================================================================
Legend : D – Dynamic Neighbor
===============================================================================
Neighbor
Description
.                 AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
.                    PktSent OutQ
——————————————————————————-
10.0.0.33
.              65000      54    0 00h17m20s 21/0/0 (LinkState)
.                         37    0
——————————————————————————-
!
!
RP/0/0/CPU0:XR3#show bgp link-state link-state summary
BGP router identifier 10.0.0.33, local AS number 65000
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 65
BGP main routing table version 65
BGP NSR Initial initsync version 22 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
!
Neighbor    Spk    AS MsgRcvd MsgSent   TblVer InQ OutQ Up/Down St/PfxRcd
10.0.0.11         0 65000 526 627       65    0 0    00:17:59       0

As you see, XR3 sends 21 BGP-LS prefixes to SR1 and SR1 doesn’t send anything to XR3, what is fully correct. The question that might arise in your mind now is “why we send 21 prefix?”

The question is fully legitimate, because we have only 3 LSP in ISIS LSDB (one per each router in ISIS process). Let’s analyse, whet we have in BGP RIB for BGP link-state address family. Let’s start with Cisco IOS XR router XR3, as it grabs ISIS information:

RP/0/0/CPU0:XR3#show bgp link-state link-state
BGP router identifier 10.0.0.33, local AS number 65000
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0 RD version: 65
BGP main routing table version 65
BGP NSR Initial initsync version 22 (Reached)
BGP NSR/ISSU Sync-Group versions 0/0
BGP scan interval 60 secs
!
Status codes: s suppressed, d damped, h history, * valid, > best
.             i – internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i – IGP, e – EGP, ? – incomplete
Prefix codes: E link, V node, T IP reacheable route, u/U unknown
.             I Identifier, N local node, R remote node, L link, P prefix
.             L1/L2 ISIS level-1/level-2, O OSPF, D direct, S static/peer-node
.             a area-ID, l link-ID, t topology-ID, s ISO-ID,
.             c confed-ID/ASN, b bgp-identifier, r router-ID,
.             i if-address, n nbr-address, o OSPF Route-type, p IP-prefix
.             d designated router address
. Network             Next Hop            Metric LocPrf Weight Path
*> [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]]/328
.                     0.0.0.0                                0 i
*> [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]]/328
.                     0.0.0.0                                0 i
*> [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]]/328
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[i10.22.33.22][n10.22.33.33]]/696
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0044.00]][L[i10.22.44.22][n10.22.44.44]]/696
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0044.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][R[c65000][b10.0.0.33][s0100.0000.0022.00]]/568
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][R[c65000][b10.0.0.33][s0100.0000.0022.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][R[c65000][b10.0.0.33][s0100.0000.0044.00]]/568
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][R[c65000][b10.0.0.33][s0100.0000.0044.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0022.00]]/568
.                     0.0.0.0                                0 i
> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0022.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]]/568
.                     0.0.0.0                                0 i
*> [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[t0x0002]]/616
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][P[p10.0.0.22/32]]/400
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][P[p10.0.0.33/32]]/400
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][P[p10.0.0.44/32]]/400
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][P[t0x0002][pfc00::10:0:0:22/128]]/544
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0033.00]][P[t0x0002][pfc00::10:0:0:33/128]]/544
.                     0.0.0.0                                0 i
*> [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][P[t0x0002][pfc00::10:0:0:44/128]]/544
.                     0.0.0.0                                0 i
Processed 21 prefixes, 21 paths

You see, we have 21 prefix in XR3 BGP RIB, so this number is correct. But why we have so much prefixes? Let’s analyse them. The first important point is that we have 3 different route types:

[V] represents the node in ISIS topology. We have 3 such routes.
[E] represents the link between nodes. We have 12 such routes: each router is connected to two another in 2 topologies (IPv4 and IPv6) therefore we have 2*2 = 4 links per router.
[T] represents prefixes. We have 6 such routes, as each router announces one its IPv4 and one IPv6 prefix.

Basically what router do is extracts different TLVs from ISIS LSP and generate different BGP-LS routes based on TLV number. Though you can read and guess the prefix meaning just by its name, we’ll see the details of each route.

We start with [V] that is node route (I just copy the route how its show in BGP RIB):

RP/0/0/CPU0:XR3#show bgp link-state link-state [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]]/328
BGP routing table entry for [V][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022. 00]]/328
Versions:
. Process bRIB/RIB SendTblVer
. Speaker 2 2
Last Modified: Aug 5 02:49:15.280 for 05:36:07
Paths: (1 available, best #1)
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Path #1: Received by speaker 0
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Local
.   0.0.0.0 from 0.0.0.0 (10.0.0.33)
.     Origin IGP, localpref 100, valid, redistributed, best, group-best
.     Received Path ID 0, Local Path ID 0, version 2
.     Link-state: MT-ID: 0 2, Node-name: SR2, ISIS area: 49.00.00
.                  Local TE Router-ID: 10.0.0.22

The most important information is in the end, where we see which ISIS topology are configured (0 for IPv4 unicast and 2 for IPv6 unicast), node hostname, TE-ID (traffic engineering) and ISIS area.

The next route we check is [E], which is link route:

P/0/0/CPU0:XR3#show bgp link-state link-state [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]]/568
BGP routing table entry for [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]]/568
Versions:
. Process bRIB/RIB SendTblVer
. Speaker 63 63
Last Modified: Aug 5 07:21:45.280 for 01:07:46
. Paths: (1 available, best #1)
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Path #1: Received by speaker 0
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Local
.   0.0.0.0 from 0.0.0.0 (10.0.0.33)
.     Origin IGP, localpref 100, valid, redistributed, best, group-best
.     Received Path ID 0, Local Path ID 0, version 63
.     Link-state: metric: 10

We don’t have any RSVP, MPLS-TE or Segment Routing parameters configured, that’s why we have only IGP metric as parameters for this link.

And the last type of BGP-LS routes is [T], which represents the prefixes sent originated by certain router:

RP/0/0/CPU0:XR3#show bgp link-state link-state [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][P[p10.0.0.44/32]]/400
BGP routing table entry for [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0044.00]][P[p10.0.0.44/32]]/400
Versions:
. Process bRIB/RIB SendTblVer
. Speaker 19 19
Last Modified: Aug 5 02:49:15.280 for 05:44:51
. Paths: (1 available, best #1)
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Path #1: Received by speaker 0
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Local
.   0.0.0.0 from 0.0.0.0 (10.0.0.33)
.     Origin IGP, localpref 100, valid, redistributed, best, group-best
.     Received Path ID 0, Local Path ID 0, version 19
.     Link-state: Metric: 0

Here we in general don’t have much interesting information.

Now let’s verify, how we see these prefixes in Nokia (Alcatel-Lucent) SR OS routers:

A:SR1# show router bgp routes bgp-ls
===============================================================================
BGP Router ID:10.0.0.11        AS:65000       Local AS:65000
===============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
.              l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete
===============================================================================
BGP-LS Node NLRIs
===============================================================================
Flag Prot/Id                                  Nexthop LocalPref MED
.     Local Node:AS/LsID/OSPF Ar Id
.      IGP Rt Id
——————————————————————————-
*>i ISIS Level-2/a                           10.0.0.33       100       None
.     0.0.253.232/167772193/-
.      0x10000000044
*>i   ISIS Level-2/a                           10.0.0.33       100       None
.     0.0.253.232/167772193/-
.      0x10000000033
*>i   ISIS Level-2/a                           10.0.0.33       100       None
.     0.0.253.232/167772193/-
.      0x10000000022
——————————————————————————-
Routes : 3
===============================================================================
BGP-LS Link NLRIs
===============================================================================
Flag Prot/Id                                  Nexthop LocalPref MED
.     Local Node:AS/LsID/OSPF Ar Id
.      IGP Rt Id
.     Remote Node:AS/LsID/OSPF Ar Id
.      IGP Rt Id
.     Link: MT ID/Local Id/Remote Id
.      IPv4 Inf/Neighbor
.      IPv6 Inf/Neighbor
——————————————————————————-
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.      0x10000000022
.     0.0.253.232/167772193/-
.      0x10000000044
.     2/-/-
.      -/-
*> i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.      0x10000000044
.     0.0.253.232/167772193/-
.      0x10000000022
.      None
*>i   ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.    0x10000000033
.     0.0.253.232/167772193/-
. 0x10000000022
.     2/-/-
.      -/-
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.    0x10000000044
.     0.0.253.232/167772193/-
. 0x10000000033
.     2/-/-
.      -/-
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.     0x10000000022
.     0.0.253.232/167772193/-
.    0x10000000044
.     -/-/-
. 10.22.44.22/10.22.44.44
*>i   ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
. 0x10000000022
.     0.0.253.232/167772193/-
. 0x10000000033
.     2/-/-
.      -/-
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.     0x10000000022
.     0.0.253.232/167772193/-
.    0x10000000033
.     -/-/-
. 10.22.33.22/10.22.33.33
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.    0x10000000044
.     0.0.253.232/167772193/-
. 0x10000000033
.     None
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.     0x10000000033
.     0.0.253.232/167772193/-
.    0x10000000044
.     None
*>i   ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
. 0x10000000033
.     0.0.253.232/167772193/-
.      0x10000000022
.     None
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.    0x10000000044
.     0.0.253.232/167772193/-
.      0x10000000022
.     2/-/-
.      -/-
*>i ISIS Level-2/a 10.0.0.33       100       None
.     0.0.253.232/167772193/-
.     0x10000000033
.     0.0.253.232/167772193/-
.    0x10000000044
.     2/-/-
. -/-
——————————————————————————-
Routes : 12
===============================================================================
BGP-LS Ipv4 NLRIs
===============================================================================
Flag Prot/Id                                  Nexthop LocalPref MED
.     Local Node:AS/LsID/OSPF Ar Id
.      IGP Rt Id
.     Prefix Desc:MT ID/OSPF Rt Type
.      IP Reachability Addr/Prefix Len
——————————————————————————-
*>i ISIS Level-2/a                           10.0.0.33       100 None
.     0.0.253.232/167772193/-
.      0x10000000044
.     -/-
.      10.0.0.44/32
*>i ISIS Level-2/a                           10.0.0.33       100 None
.     0.0.253.232/167772193/-
.    0x10000000022
.     -/-
.      10.0.0.22/32
*>i ISIS Level-2/a                           10.0.0.33       100 None
.     0.0.253.232/167772193/-
.      0x10000000033
.     -/-
.      10.0.0.33/32
——————————————————————————-
Routes : 3
——————————————————————————-
Total Routes : 18
===============================================================================

The very first point is that Nokia (Alcatel-Lucent) SR OS has lower amount of routes than Cisco (18 vs 21). Careful review show of SR1 BGP RIB for BGP-LS shows that Nokia VSR doesn’t IPv6 prefixes routes (3 of 6 [T] routes in Cisco). If I chose details, I can’t see IPv6 prefix there, only IPv4. I believe it would be fixed (or already fixed) in newer version of Nokia (Alcatel-Lucent) SR OS. In the rest information is much the same and even user friendly, as BGP-LS routes are splitted into 3 categories and are named as Node, Link and IPv4.

To see details of BGP-LS routes in Nokia (Alcatel-Lucent) SR OS use “show router bgp routes bgp-ls hunt” command.

Adding more value to BGP-LS

BGP-LS is one of the building block of SDN that’s why pure routing metrics are neither interesting nor very useful. Let’s enable Segment Routing in our ISIS network and then analyze changes in BGP-LS prefixes:

Nokia (Alcatel-Lucent) SR OS	Cisco IOS XR
SR1	XR3
A:SR1>edit-cfg# candidate view ========================= =========================	RP/0/0/CPU0:XR3(config)#show conf ! router isis CORE address-family ipv4 unicast segment-routing mpls sr-prefer ! address-family ipv6 unicast segment-routing mpls sr-prefer ! interface Loopback0 address-family ipv4 unicast prefix-sid index 33 ! address-family ipv6 unicast prefix-sid index 1033 ! ! ! end
SR2	XR4
A:SR2>edit-cfg# candidate view ========================= configure router mpls-labels sr-labels start 500000 end 510000 exit isis advertise-router-capability as segment-routing prefix-sid-range global tunnel-table-pref 8 no shutdown exit interface “system” ipv4-node-sid index 22 ipv6-node-sid index 1022 exit exit exit exit =========================	RP/0/0/CPU0:XR4(config)#show conf ! router isis CORE address-family ipv4 unicast segment-routing mpls sr-prefer ! address-family ipv6 unicast segment-routing mpls sr-prefer ! interface Loopback0 address-family ipv4 unicast prefix-sid index 44 ! address-family ipv6 unicast prefix-sid index 1044 ! ! ! end

Now we have much more information in our ISIS LSDB, which is related to segment routing. You see, we haven’t configured anything additional to commands for distributing ISIS to BGP-LS. But if we see into BGP-LS NLRI (network layer reachability information – prefix, actually), we see new information there. For link prefix [E] we have:

P/0/0/CPU0:XR3#show bgp link-state link-state [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[i10.22.33.22][n10.22.33.33]]/696
BGP routing table entry for [E][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][R[c65000][b10.0.0.33][s0100.0000.0033.00]][L[i10.22.33.22][n10.22.33.33]]/696
Versions:
. Process bRIB/RIB SendTblVer
. Speaker 80 80
Last Modified: Aug 5 10:27:11.280 for 00:30:49
. Paths: (1 available, best #1)
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Path #1: Received by speaker 0
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Local
.   0.0.0.0 from 0.0.0.0 (10.0.0.33)
.     Origin IGP, localpref 100, valid, redistributed, best, group-best
.     Received Path ID 0, Local Path ID 0, version 80
.     Link-state: Local TE Router-ID: 10.0.0.22, TE-default-metric: 10
.     metric: 10, ADJ-SID: 262143(30)

We see new attribute in this BGP NLRI, which is ADJ-SID label generated by Nokia (Alcatel-Lucent) SR OS router SR2.

Now we check [T] prefix:

P/0/0/CPU0:XR3#show bgp link-state link-state [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][P[p10.0.0.22/32]]/400
BGP routing table entry for [T][L2][I0xa][N[c65000][b10.0.0.33][s0100.0000.0022.00]][P[p10.0.0.22/32]]/400
Versions:
. Process bRIB/RIB SendTblVer
. Speaker 82 82
Last Modified: Aug 5 10:30:00.280 for 00:29:32
. Paths: (1 available, best #1)
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Path #1: Received by speaker 0
. Advertised to peers (in unique update groups):
.   10.0.0.11
. Local
.   0.0.0.0 from 0.0.0.0 (10.0.0.33)
.     Origin IGP, localpref 100, valid, redistributed, best, group-best
.     Received Path ID 0, Local Path ID 0, version 82
.     Link-state: Metric: 0, PFX-SID: 22(60/0)

Here we have prefix-SID of the router, which is used for generating of the label for the node itself. As a brief summary, now the value of BGP-LS has increased because SDN controller has more useful information that can be used for building MPLS LSP between PEs later.

Final configuration files for this case are here: 086_config_final_xr3_p1 086_config_final_xr4_p1 086_config_final_sr1_p1 086_config_final_sr2_p1

Configuration of BGP-LS with distribution of LS information at Nokia SR OS

The second case is to change SR1 and XR3 with their functions. Nokia (Alcatel-Lucent) VSR (SR 7750) will distribute its ISIS LSDB through BGP-LS to XR3. So the topology now looks a bit different:

The initial configuration of the devices is here: 086_config_initial_linux_p2 086_config_initial_sr1_p2 086_config_initial_sr2_p2 086_config_initial_xr3_p2 086_config_initial_xr4_p2

It’s just the same as it was in the very beginning for another case from functionality point of view. Now we do the same configuration as we’ve done before:

Static routing between SR1 and XR3
BGP-LS between SR1 and XR3
Distribution of ISIS LSDB to BGP-LS at SR1
Segment routing in SR1, SR2, XR4

Here we go:

Nokia (Alcatel-Lucent) SR OS	Cisco IOS XR
SR1	XR3
A:SR1>edit-cfg# candidate view ========================= configure router static-route-entry 10.0.0.33/32 next-hop 10.11.33.33 no shutdown exit exit autonomous-system 65000 bgp link-state-import-enable link-state-export-enable group “SDN” family bgp-ls peer-as 65000 local-address 10.0.0.11 neighbor 10.0.0.33 exit exit no shutdown exit mpls-labels sr-labels start 500000 end 510000 exit isis database-export identifier 10 bgp-ls-identifier 167772171 advertise-router-capability as segment-routing prefix-sid-range global tunnel-table-pref 8 no shutdown exit interface “system” ipv4-node-sid index 11 ipv6-node-sid index 1011 exit exit exit exit =========================	RP/0/0/CPU0:XR3(config)#show conf ! router static address-family ipv4 unicast 10.0.0.11/32 GigabitEthernet0/0/0/0.13 10.11.33.11 ! ! router bgp 65000 bgp router-id 10.0.0.33 bgp log neighbor changes detail address-family link-state link-state domain-distinguisher 65000:10.0.0.33 ! neighbor 10.0.0.11 remote-as 65000 update-source Loopback0 address-family link-state link-state ! ! ! end
SR2	XR4
A:SR2>edit-cfg# candidate view ========================= configure router mpls-labels sr-labels start 500000 end 510000 exit isis advertise-router-capability as segment-routing prefix-sid-range global tunnel-table-pref 8 no shutdown exit interface “system” ipv4-node-sid index 22 ipv6-node-sid index 1022 exit exit exit exit =========================	RP/0/0/CPU0:XR4(config)#show conf ! router isis CORE address-family ipv4 unicast segment-routing mpls sr-prefer ! address-family ipv6 unicast segment-routing mpls sr-prefer ! interface Loopback0 address-family ipv4 unicast prefix-sid index 44 ! address-family ipv6 unicast prefix-sid index 1044 ! ! ! end

BGP-LS-ID 167772171 is decimal for IPv4 address 10.0.0.11.

It took me enormous time in order to configure Nokia (Alcatel-Lucent) SR OS to start sending BGP-LS NLRI to Cisco. Let’s check the BGP peering for BGP link-state address family:

Well, now we have more than 2 times lower amount of BGP-LS prefixes than we have had previously, when we exported them from Cisco IOS XR router. We can assume Nokia (Alcate-Lucent) SR OS doesn’t export IPv6 prefixes as it doesn’t understand them. But in that case we should have at least 18 prefixes…

As you remember, we can’t see local routes in BGP RIB in Nokia (Alcatel-Lucent) SR OS, so we have to analyze BGP-LS NLRI at Cisco IOS XR router XR3:

First of all, we see full absence of any IPv6-related information, both for address prefixes [T] and links [E]. We still have all node [V] prefixes and IPv4 address prefixes [T]. What is really strange for me is that links [E] prefixes aren’t fully distributed: SR1 exports links from SR1 and SR2 (both are Nokia (Alcatel-Lucent) SR OS routers) and doesn’t export links from XR4 (Cisco IOS XR router). I’ve compared IS-neighbor fields in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR and the only difference I’ve found is that in Cisco we have two Segment Routing MPLS labels, whereas in Nokia we have only one. For example, here we have parts of ISIS LSP from SR2 and XR4, which describes link SR2-XR4:

RP/0/0/CPU0:XR4#show isis database level 2 SR2.00-00 verbose
IS-IS CORE (Level-2) Link State Database
LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL
SR2.00-00 0x00000073 0x0e43 1068 0/0/0
Area Address: 49.0000
NLPID: 0xcc
NLPID: 0x8e
MT: Standard (IPv4 Unicast)
MT: IPv6 Unicast 0/0/0
Hostname: SR2
Router ID: 10.0.0.22
Router Cap: 10.0.0.22, D:0, S:0
TE PCE Discovery (Old):
PCE sub TLV 232: Length 2 <Unknown TLV>
Segment Routing: I:1 V:1, SRGB Base: 500000 Range: 10001
SubTLV 19Length: 1
IP Address: 10.0.0.22
IP Address: 10.11.22.22
IP Address: 10.22.44.22
IPv6 Address: fc00::10:0:0:22
Metric: 10 IS-Extended XR4.00
Interface IP Address: 10.22.44.22
Neighbor IP Address: 10.22.44.44
ADJ-SID: F:0 B:0 V:1 L:1 S:0 weight:0 Adjacency-sid:262142
!
!
RP/0/0/CPU0:XR4#show isis database level 2 XR4.00-00 verbose
IS-IS CORE (Level-2) Link State Database
LSPID LSP Seq Num LSP Checksum LSP Holdtime ATT/P/OL
XR4.00-00 * 0x0000005a 0x4c17 1056 0/0/0
Area Address: 49.0000
NLPID: 0xcc
NLPID: 0x8e
MT: Standard (IPv4 Unicast)
MT: IPv6 Unicast 0/0/0
Hostname: XR4
IP Address: 10.0.0.44
IPv6 Address: fc00::10:0:0:44
Router Cap: 10.0.0.44, D:0, S:0
Segment Routing: I:1 V:1, SRGB Base: 16000 Range: 8000
Metric: 10 IS-Extended SR2.00
Interface IP Address: 10.22.44.44
Neighbor IP Address: 10.22.44.22
ADJ-SID: F:0 B:1 V:1 L:1 S:0 weight:0 Adjacency-sid:24010
ADJ-SID: F:0 B:0 V:1 L:1 S:0 weight:0 Adjacency-sid:24011

When I was writing this part I’ve spotted interesting difference that I haven’t paid attention previously. As you might see, Nokia (Alcatel-Lucent) VSR (SR 7750) announces all IP addresses of its interfaces, whereas Cisco IOS XRv 9000 (ASR 9000) announces only IP addreses of passive interfaces. I’m speaking now not about extended IP reachability TLVs, where information is similar. I’m speaking about interfaces TLV. Let’s change a bout configuration of our ISIS so that all IP addresses are announced:

Nokia (Alcatel-Lucent) SR OS	Cisco IOS XR
SR1	XR3
A:SR1>edit-cfg# candidate view ========================= configure router isis no advertise-passive-only exit exit exit =========================	RP/0/0/CPU0:XR3(config)#show conf ! end
SR2	XR4
A:SR2>edit-cfg# candidate view ========================= configure router isis no advertise-passive-only exit exit exit =========================	RP/0/0/CPU0:XR4(config)#show conf ! router isis CORE address-family ipv4 unicast no advertise passive-only ! address-family ipv6 unicast no advertise passive-only ! ! end

What do you think? Does it help to solve our issue? The only way is to check BGP-LS RIB at Cisco IOS XR router XR3 and to see, what Nokia (Alcatel-Lucent) SR OS router SR1 has announced to it:

RP/0/0/CPU0:XR3#show bgp link-state link-state
Status codes: s suppressed, d damped, h history, * valid, > best
i – internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i – IGP, e – EGP, ? – incomplete
Prefix codes: E link, V node, T IP reacheable route, u/U unknown
I Identifier, N local node, R remote node, L link, P prefix
L1/L2 ISIS level-1/level-2, O OSPF, D direct, S static/peer-node
a area-ID, l link-ID, t topology-ID, s ISO-ID,
c confed-ID/ASN, b bgp-identifier, r router-ID,
i if-address, n nbr-address, o OSPF Route-type, p IP-prefix
d designated router address
Network Next Hop Metric LocPrf Weight Path
*>i[V][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]]/328
10.0.0.11 100 0 i
*>i[V][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]]/328
10.0.0.11 100 0 i
*>i[V][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]]/328
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]][R[c65000][b10.0.0.11][s0100.0000.0022.00]][L[i10.11.22.11][n10.11.22.22]]/696
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]][R[c65000][b10.0.0.11][s0100.0000.0044.00]][L[i10.11.44.11][n10.11.44.44]]/696
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]][R[c65000][b10.0.0.11][s0100.0000.0011.00]][L[i10.11.22.22][n10.11.22.11]]/696
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]][R[c65000][b10.0.0.11][s0100.0000.0044.00]][L[i10.22.44.22][n10.22.44.44]]/696
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]][R[c65000][b10.0.0.11][s0100.0000.0011.00]][L[i10.11.44.44][n10.11.44.11]]/696
10.0.0.11 100 0 i
*>i[E][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]][R[c65000][b10.0.0.11][s0100.0000.0022.00]][L[i10.22.44.44][n10.22.44.22]]/696
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]][P[p10.11.22.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]][P[p10.11.44.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0011.00]][P[p10.0.0.11/32]]/400
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]][P[p10.11.22.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]][P[p10.22.44.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0022.00]][P[p10.0.0.22/32]]/400
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]][P[p10.11.44.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]][P[p10.22.44.0/24]]/392
10.0.0.11 100 0 i
*>i[T][L2][I0xa][N[c65000][b10.0.0.11][s0100.0000.0044.00]][P[p10.0.0.44/32]]/400
10.0.0.11 100 0 i
Processed 18 prefixes, 18 paths

We have 8 prefixes more than earlier:

6 of them are address prefixes [T] from links between routers.
2 of them are link prefixes [E] generated by XR4.

Now our SDN controller has full topology information from our network.

The final configuration files for the second case are here: 086_config_final_sr1_p2 086_config_final_sr2_p2 086_config_final_xr3_p2 086_config_final_xr4_p2

Lessons learned

Documentation is always a key, but in this article I was like a blind cat, because Nokia documentation for BGP-LS is very bad, at least official “Unicast Routing Protocols Guide” available on Nokia website. It doesn’t provide example hot configure distribution of TE-DB through BGP-LS, only separated commands not related to each other. So I had to trying all the commands I can find in this guide and CLI to make it working and though I made it, I took me enormous amount of time.

Dear Nokia colleagues, please, make your documentation consistent and clear. I understand that you want to sell your services for hundreds of thousands of dollars, but respect your customers.

Conclusion

BGP-LS provides you possibility to distribute your routing topology with all traffic engineering attributes (LSDB and TED) to SDN controller so that It can build in his brain full view of all your network including all inter-area cases. It’s the first step in provisioning automatic MPLS LSP within the routing domain (area for OSPF and level for ISIS) or between them. BGP-LS is quite new technology and I’m pretty sure, its use cases will be further expanded. Take care and good bye!

Support us

Support new interop and automation articles at karneliuk.com
EUR

I want to support with:

BR,

Anton Karneliuk