BGP Labeled Unicast in Nokia (Alcatel-Lucent) SR and Cisco IOS XR
Anton Karneliuk
Hello my friend,
As I’m told in the first article about BGP, I’ll be returning to this protocol many times in future. Today I’ll cover very interesting topic (at least in my opinion) that is called BGP Label Unicast (BGP-LU). Let’s see what is it about and how it can be configured.
Disclaimer
Thanks to colleague of mine, Greg Hankins, I’ve updated my lab to the latest SR OS version – 14.0.R4. And I’ve spotted there are several major differences in configuration of BGP labeled unicast (BGP-LU) comparing to 13.0.R1 that I’ve used before. The difference is almost the same in terms of configuration as between Cisco IOS and Cisco IOS XR. Refer to this book for examples of configuration of BGP-LU prior to version 14.
Before we begin
You might expect that we’ll talk about BGP/MPLS VPNs in the article, because I’ve mentioned labeled, what definitely means MPLS. No, we won’t. BGP-LU is used to distribute labels via BGP for address family IPv4/IPv6 unicast (AFI/SAFI 1/4 and 2/4) and to build MPLS forwarding plane (i.E. BGP-LU distributes transport labels). BGP/MPLS VPNs deal with address family VPNv4/VPNv6 unicast/multicast (AFI/SAFI 1/128 and 2/128) and is used to distribute labels for certain L3 VPNs (VPRN/VRF). There are four most widely used cases for BGP labeled unicast:
Inter-AS MPLS VPN Option C;
Seamless MPLS (Unified MPLS) in service provider network;
Carrier Supporting Carrier (CSC) VPN;
IGP Free-data center.
In the first three cases, BGP-LU is built on top of ready MPLS network (LDP, RSVP or SR) in order to create a hierarchical LSP (Labeled Switched Path) across different IGP domains. This is necessary to provide scalability and resiliency in big service providers’ networks. More information you can find in this IETF draft.
The latest scenario implies using only BGP for routing inside the data center due to its high level of scalability and flexibility. More information on that is this IETF draft .
In our lab we’ll use the latter case, so we’ll build an IGP-free datacenter using BGP labeled unicast.
Topology
The physical topology doesn’t change and we continue to use the previous one:
The logical topology is built with extensive use of VPRN (for Nokia (Alcatel-Lucent) SR OS) and VRF (for Cisco IOS XR). The reason for that is that we’ll have 6 virtual routers will functional MPLS data plane and therefore we’ll have a 5-stage fabric for data center:
In order to make it working, special configuration for VPRNs at Nokia (Alcatel-Lucent) VSR (SR 7750) is needed. Nevertheless if you have enough computing resources, you can make a simple configuration without any VPRNs/VRFs. The initial topology for our lab you can find here: linuxSR1_initialSR3_initialXR1_initial
If you carefully review the initial configs, you can find significant difference comparing to first BGP article for virtual routers (VPRNs) in Nokia (Alcatel-Lucent) SR OS. Here we have to enable MPLS in VPRN, that’s why the configuration is different.
Overall architecture
On top of the IPv4/IPv6 addressing it’s necessary to define, which prefix should be transferred with labeled and which without. The picture above helps you to understand, how BGP sessions should be configured:
In order to achieve good level of flexibility and scalability, I’m going to use BGP community. Extensive use of reg-exps (UNIX regular expressions) in route policies ease the application of certain actions to prefixes:
All prefixes that should be announced via BGP labeled unicast meaning the prefixes that have MPLS label (AFI/SAFI 1/4 in our case) have community ending with “:1”.
All prefixes that should be announced via BGP unlabeled unicast (AFI/SAFI 1/1 for IPv4 and 2/1 for IPv6) have community ending with “:5”.
Based on these communities we establish route policies that allow or deny prefixes.
BGP Labeled Unicast (BGP-LU)
As we’ve said in the very beginning, BGP is used in this article in order to build MPLS forwarding plane and BGP is used to deliver MPLS transport label.
Don’t be afraid of big amount of configuration below. It’s the same across all devices.
Nokia (Alcatel-Lucent) VSR (SR 7750)
Cisco IOS XR (ASR 9000)
VPRN 65001 (SPINE1)
VRF SPINE2
A:SR1>config>router>policy-options# info
—————————
prefix-list PL_AS65001_Lo1
prefix 10.0.255.1/32
exit
community CL_BGP_LU_NH_AS65001 member “65001:1”
community CL_BGP_LU_NH_ALL member “.*:1$”
policy-statement “RP_BGP_LU_IN”
entry 10
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
policy-statement “RP_BGP_AS65001_OUT”
entry 10
from
protocol direct
prefix-list “PL_AS65001_Lo1”
exit
action accept
community add “CL_BGP_LU_NH_AS65001”
exit
exit
entry 20
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
—————————
A:SR1>config>service>vprn# info
—————————
bgp
advertise-inactive
rapid-withdrawal
group “eBGP_LU”
family label-ipv4
import “RP_BGP_LU_IN”
export “RP_BGP_AS65001_OUT”
neighbor 10.0.0.4
peer-as 65010
exit
neighbor 10.0.0.8
peer-as 65020
exit
exit
no shutdown
exit
no shutdown
—————————
A:SR3>config>router>policy-options# info
—————————
prefix-list “PL_AS65010_Lo1”
prefix 10.0.255.10/32 exact
exit
community “CL_BGP_LU_NH_ALL” members “.*:1$”
community “CL_BGP_LU_NH_AS65010” members “65010:1”
policy-statement “RP_BGP_LU_IN”
entry 10
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
policy-statement “RP_BGP_AS65010_OUT”
entry 10
from
protocol direct
prefix-list “PL_AS65010_Lo1”
exit
action accept
community add “CL_BGP_LU_NH_AS65010”
exit
exit
entry 20
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
—————————
A:SR3>config>service>vprn# info
—————————
bgp
advertise-inactive
rapid-withdrawal
group “eBGP_LU”
family label-ipv4
import “RP_BGP_LU_IN”
export “RP_BGP_AS65010_OUT”
neighbor 10.0.0.0
peer-as 65100
exit
neighbor 10.0.0.5
peer-as 65001
exit
neighbor 10.0.0.7
peer-as 65002
exit
exit
no shutdown
exit
no shutdown
—————————
VPRN 65020 (LEAF2)
A:SR3>config>router>policy-options# info
—————————
prefix-list “PL_AS65020_Lo1”
prefix 10.0.255.20/32 exact
exit
community “CL_BGP_LU_NH_ALL” members “.*:1$”
community “CL_BGP_LU_NH_AS65020” members “65020:1”
policy-statement “RP_BGP_LU_IN”
entry 10
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
policy-statement “RP_BGP_AS65020_OUT”
entry 10
from
protocol direct
prefix-list “PL_AS65020_Lo1”
exit
action accept
community add “CL_BGP_LU_NH_AS65020”
exit
exit
entry 20
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
—————————
A:SR3>config>service>vprn# info
—————————
bgp
advertise-inactive
rapid-withdrawal
group “eBGP_LU”
family label-ipv4
import “RP_BGP_LU_IN”
export “RP_BGP_AS65010_OUT”
neighbor 10.0.0.2
peer-as 65200
exit
neighbor 10.0.0.9
peer-as 65001
exit
neighbor 10.0.0.11
peer-as 65002
exit
exit
no shutdown
exit
no shutdown
—————————
VPRN 65100 (TOR1)
VRF TOR2
A:SR1>config>router>policy-options# info
—————————
prefix-list PL_AS65100_Lo100
prefix 10.0.255.100/32
exit
community CL_BGP_LU_NH_AS65100 member “65100:1”
community CL_BGP_LU_NH_ALL member “.*:1$”
policy-statement “RP_BGP_LU_IN”
entry 10
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
policy-statement “RP_BGP_AS65100_OUT”
entry 10
from
protocol direct
prefix-list “PL_AS65100_Lo100”
exit
action accept
community add “CL_BGP_LU_NH_AS65100”
exit
exit
entry 20
from
community “CL_BGP_LU_NH_ALL”
exit
action accept
exit
exit
default-action drop
exit
exit
—————————
A:SR1>config>service>vprn# info
—————————
bgp
advertise-inactive
rapid-withdrawal
group “eBGP_LU”
family label-ipv4
import “RP_BGP_LU_IN”
export “RP_BGP_AS65100_OUT”
neighbor 10.0.0.3
peer-as 65020
exit
exit
no shutdown
exit
no shutdown
—————————
RP/0/0/CPU0:XR1(config)#show conf
route-policy RP_IPV4_AS65200_PREFIX_REDISTRIBUTION
if destination in (10.0.255.200/32) then
set community (65100:1)
elseif destination in (172.16.200.0/24) then
set community (65100:5)
endif
end-policy
!
route-policy RP_IPV4_BGP_LU_OUT
if community matches-any (ios-regex ‘_.*:1_’) then
pass
endif
end-policy
!
route-policy RP_PASSALL
pass
end-policy
!
router static
vrf TOR2
address-family ipv4 unicast
10.0.0.3/32 GigabitEthernet0/0/0/0.11
!
!
!
router bgp 65000
bgp router-id 1.1.1.1
bgp log neigh chan det
add vpnv4 uni
!
add vpnv6 uni
!
vrf TOR2
mpls activate
interface GigabitEthernet0/0/0/0.11
!
bgp router-id 10.0.255.200
rd 10.0.255.200:65200
bgp bestpath as-path multipath-relax
add ipv4 uni
redistribute connected route-policy RP_IPV4_AS65200_PREFIX_REDISTRIBUTION
allocate-label route-policy RP_IPV4_BGP_LU_OUT
!
neighbor 10.0.0.3
remote-as 65020
local-as 65200 no-prepend replace-as
add ipv4 labeled-unicast
send-community-ebgp
route-policy RP_IPV4_BGP_LU_OUT out
route-policy RP_PASSALL in
!
!
!
!
Despite the long configuration, everything we do is just announce looback of each router with community “AS:1” and establish BGP neighboring for AFI/SAFI 1/4, what corresponds address-family IPv4 labeled unicast. Also we make filtering in order to send and receive only prefixes with community “AS:1” through such BGP-LU peering.
Static routes in Cisco IOS XR is necessary. Refer to lessons learned for explanation.
If you have configured the part above correctly, you’ll see corresponding messages in the log:
SR OS:
41 2016/10/05 12:06:40.72 UTC MINOR: BGP #2038 vprn65010 Peer 2: 10.0.0.7
“VR 2: Group eBGP_LU: Peer 10.0.0.7: moved into established state”
!
!
IOS XR:
RP/0/0/CPU0:XR1#RP/0/0/CPU0:2016 Sep 28 12:10:56.168 : bgp[1053]: %ROUTING-BGP-5-ADJCHANGE_DETAIL : neighbor 10.0.0.6 Up (VRF: SPINE2; AFI/SAFI: 1/4) (AS: 65010)
The BGP sessions are established for address family IPv4 labeled-unicast. Nokia (Alcatel-Lucent) SR OS tells about it in the following way:
You can see that each router advertises all routes and labels but in LFIB is installed only label for the best route.
A:SR1# show router 65001 fib 1
============================================================================
FIB Display
============================================================================
Prefix [Flags] Protocol
NextHop
—————————————————————————-
10.0.0.4/31 LOCAL
10.0.0.4 (toLEAF1)
10.0.0.8/31 LOCAL
10.0.0.8 (toLEAF2)
10.0.255.1/32 LOCAL
10.0.255.1 (Lo1)
10.0.255.2/32 BGP_LABEL
10.0.0.4 (toLEAF1)
10.0.255.10/32 BGP_LABEL
10.0.0.4 (toLEAF1)
10.0.255.20/32 BGP_LABEL
10.0.0.8 (toLEAF2)
10.0.255.100/32 BGP_LABEL
10.0.0.4 (toLEAF1)
10.0.255.200/32 BGP_LABEL
10.0.0.8 (toLEAF2)
—————————————————————————-
Total Entries : 8
—————————————————————————-
!
!
A:SR1# show router 65001 bgp inter-as-label
============================================================================
BGP Inter-AS labels
Flags: B – entry has backup, P – entry is promoted
============================================================================
NextHop Received Advertised Label
Label Label Origin
—————————————————————————-
10.0.255.1 0 262140 Edge
10.0.0.4 262131 262130 External
10.0.0.4 262133 262134 External
10.0.0.4 262141 262139 External
10.0.0.8 262135 262135 External
10.0.0.8 262140 262138 External
—————————————————————————-
Total Labels allocated: 6
In Cisco IOS XR you will check labels received by BGP and LFIB with the following commands:
show bgp vrf SPINE2 ipv4 labeled-unicast labels
!
!
show mpls forwarding vrf SPINE2
If you want, you can make some ping/traceroute tests to check connectivity. In general all routers know about all “transport” loopbacks, so such tests must be successful.
Vanilla BGP (unlabeled BGP)
Let’s recall BGP topology:
As routers for address-family IPv4 unicast don’t connect to each other directly, we built ebgp-multihop session. Source IPv4 addresses for peering are loobacks, which have announced with community “AS:1”. So the main Idea is to send traffic from Lo999 at VPRN 65100 to Lo999 at VRF TOR2 through MPLS data plane, built between Lo100 addresses. We announce Lo999 prefixes with community “AS:5” in order to make proper filtering.
Let’s create this configuration:
Nokia (Alcatel-Lucent) VSR (SR 7750)
Cisco IOS XR (ASR 9000)
VPRN 65001 (SPINE1)
VRF SPINE2
A:SR1>config>router>policy-options# info
—————————
community “CL_BGP_ULU” member “[0-9]*:5$”
policy-statement RP_BGP_ULU
entry 10
from
community “CL_BGP_ULU”
exit
action accept
exit
exit
default-action drop
exit
exit
—————————
A:SR1>config>service>vprn# info
—————————
bgp
group “eBGP_ULU”
family ipv4
multihop 255
import “RP_BGP_ULU”
export “RP_BGP_ULU”
local-address 10.0.255.1
neighbor 10.0.255.100
peer-as 65100
exit
neighbor 10.0.255.200
peer-as 65200
exit
exit
exit
—————————
RP/0/0/CPU0:XR1(config)#show conf
route-policy RP_BGP_ONLY_ULU
if community matches-any (ios-regex ‘_[0-9]*:5$’) then
pass
else
drop
endif
end-policy
!
router bgp 65000
vrf SPINE2
neighbor 10.0.255.100
remote-as 65100
ebgp-multihop 255
local-as 65002 no-prepend replace-as
update-source Loopback1
address-family ipv4 unicast
send-community-ebgp
route-policy RP_BGP_ONLY_ULU in
route-policy RP_BGP_ONLY_ULU out
!
!
neighbor 10.0.255.200
remote-as 65200
ebgp-multihop 255
local-as 65002 no-prepend replace-as
update-source Loopback1
address-family ipv4 unicast
send-community-ebgp
route-policy RP_BGP_ONLY_ULU in
route-policy RP_BGP_ONLY_ULU out
!
!
!
!
end
VPRN 65100 (TOR1)
VRF TOR2
A:SR1>config>router>policy-options# info
—————————
prefix-list “PL_AS65100_DC”
prefix 172.16.100.0/24
exit
community “CL_BGP_AS_65100_ULU” member “65100:5”
policy-statement RP_BGP_ULU
entry 10
from
community “CL_BGP_ULU”
exit
action accept
exit
exit
default-action drop
exit
exit
policy-statement “RP_BGP_AS65100_ULU_OUT”
entry 10
from
prefix-list “PL_AS65100_DC”
protocol direct
exit
action accept
community add “CL_BGP_AS_65100_ULU”
exit
exit
default-action drop
exit
exit
—————————
A:SR1>config>service>vprn# info
—————————
bgp
group “eBGP_ULU”
family ipv4
multihop 255
import “RP_BGP_ULU”
export “RP_BGP_AS65100_ULU_OUT”
local-address 10.0.255.100
neighbor 10.0.255.1
peer-as 65001
exit
neighbor 10.0.255.2
peer-as 65002
exit
exit
exit
—————————
RP/0/0/CPU0:XR1(config)#show conf
route-policy RP_BGP_AS65200_OUT
if destination in (172.16.200.0/24) then
set community (65200:5)
endif
end-policy
!
router bgp 65000
vrf TOR2
neighbor 10.0.255.1
remote-as 65001
local-as 65200 no-prepend replace-as
update-source Loopback100
ebgp-multihop 255
address-family ipv4 unicast
send-community-ebgp
route-policy RP_BGP_ONLY_ULU in
route-policy RP_BGP_AS65200_OUT out
!
!
neighbor 10.0.255.2
remote-as 65002
update-source Loopback100
local-as 65200 no-prepend replace-as
ebgp-multihop 255
address-family ipv4 unicast
send-community-ebgp
route-policy RP_BGP_ONLY_ULU in
route-policy RP_BGP_AS65200_OUT out
!
!
!
!
end
The configuration above is quite simple and follow the task. The good part of the story is that we have established BGPv4 unlabeled unicast session and even announced prefixes:
A:SR1# show router 65001 bgp summary
============================================================================
BGP Summary
============================================================================
Legend : D – Dynamic Neighbor
============================================================================
Neighbor
Description
. AS PktRcvd InQ Up/Down State|Rcv/Act/Sent (Addr Family)
. PktSent OutQ
—————————————————————————-
10.0.0.4
. 65010 155 0 01h11m30s 6/3/6 (Lbl-IPv4)
. 156 0
10.0.0.8
. 65020 155 0 01h11m30s 6/2/6 (Lbl-IPv4)
. 156 0 10.0.255.100 . 65100 75 0 00h35m59s 1/0/0 (IPv4) . 100 0 10.0.255.200 . 65200 27 0 00h07m03s 1/0/0 (IPv4) . 21 0
—————————————————————————-
A:SR1# show router 65001 bgp routes ipv4
============================================================================
BGP Router ID:10.0.255.1 AS:65001 Local AS:65001
============================================================================
Legend –
Status codes : u – used, s – suppressed, h – history, d – decayed, * – valid
l – leaked, x – stale, > – best, b – backup, p – purge
Origin codes : i – IGP, e – EGP, ? – incomplete
============================================================================
BGP IPv4 Routes
============================================================================
Flag Network LocalPref MED
. Nexthop (Router) Path-Id Label
. As-Path
——————————————————————————-
i 172.16.100.0/24 None None
. 10.0.255.100 None –
. 65100
? 172.16.200.0/24 None 0
. 10.0.255.200 None –
. 65200
—————————————————————————-
The bad part of the story is that I have to use VPRN/VRF, what leads to a numerous limitations. In Nokia (Alcatel-Lucent) SR OS it’s necessary to activate additional feature in BGP in order to provide possibility of resolving BGP next-hop through another BGP route. This feature isn’t available in VPRN; it’s allowed only in global configuration. Absence of this feature leads to the following problem:
A:SR1# show router 65100 bgp routes ipv4 detail | match Ne
Network : 172.16.200.0/24
Nexthop : 10.0.255.2 Res. Nexthop : Unresolved
Flags : Invalid Incomplete Nexthop-Unresolved
Cisco IOS XR by default use reverse lookup for BGP routes as well so you can see this new routes at TOR2:
So I can’t show you bidirectional communication. Nevertheless our MPLS dataplane works. If we enable debug of ICMP packets and logging at VPRN 65100 and make ping from VRF TOR2, you see that packets:
A:SR1# configure log
A:SR1>config>log# log-id 10
*A:SR1>config>log>log-id$ from debug-trace
*A:SR1>config>log>log-id$ to console
*A:SR1>config>log>log-id$ no shutdown
*A:SR1>config>log>log-id$
*A:SR1# debug router 65100 ip icmp
!
!
RP/0/0/CPU0:XR1#ping vrf TOR2 172.16.100.1 source 172.16.200.1
Wed Sep 28 16:38:03.280 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.100.1, timeout is 2 seconds:
.
!
!
*A:SR1#
1 2016/10/05 19:32:57.36 UTC MINOR: DEBUG #2001 vprn65100 PIP
“PIP: ICMP
instance 3 (65100), interface index 6 (toLEAF1),
ICMP ingressing on toLEAF1:
172.16.200.1 -> 172.16.100.1
type: Echo (8) code: No Code (0)
“
As you can imagine, there is no 172.16.x.0/24 routes at SPINE devices:
A:SR3# show router 65010 route-table
============================================================================
Route Table (Service: 65010)
============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
—————————————————————————-
10.0.0.0/31 Local Local 01h34m55s 0
toTOR1 0
10.0.0.4/31 Local Local 01h34m55s 0
toSPINE1 0
10.0.0.6/31 Local Local 01h34m55s 0
toSPINE2 0
10.0.255.1/32 Remote BGP_LABEL 01h32m53s 170
10.0.0.5 0
10.0.255.2/32 Remote BGP_LABEL 00h16m40s 170
10.0.0.7 0
10.0.255.10/32 Local Local 01h35m08s 0
Lo1 0
10.0.255.20/32 Remote BGP_LABEL 01h34m08s 170
10.0.0.5 0
10.0.255.100/32 Remote BGP_LABEL 01h32m53s 170
10.0.0.0 0
10.0.255.200/32 Remote BGP_LABEL 00h15m26s 170
10.0.0.5 0
—————————————————————————-
No. of Routes: 9
Cisco IOS XR has a certain problem relating to MPLS. All next-hops for labels must be configured as host-routes (with prefix /32). As we have interconnected subnet with prefix /31, the MPLS forwarding plane doesn’t work from the very beginning:
RP/0/0/CPU0:XR1#show mpls forwarding vrf TOR2
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
—— ———- —————- ———- ———– ———-
24000 262140 10.0.255.20/32[V] 10.0.0.3 0
24002 262139 10.0.255.1/32[V] 10.0.0.3 0
24003 262138 10.0.255.10/32[V] 10.0.0.3 0 24007 Aggregate 10.0.0.2/31[V] TOR2 0
24012 262134 10.0.255.2/32[V] 10.0.0.3 0
In order to get BGP label working as transport labels as well, we need to manually configure such host-routes to BGP next-hop from the router update. Actually it’s just another IP address from the interconnected subnet. To do so we need to add static routes:
After such modification if RIB we get host-route to BGP NH and our MPLS forwarding plane starts working:
RP/0/0/CPU0:XR1(config-static-vrf-afi)#do show mpls forwarding vrf TOR2
Local Outgoing Prefix Outgoing Next Hop Bytes
Label Label or ID Interface Switched
—— ———- —————- ———- ———– ———-
24000 262140 10.0.255.20/32[V] 10.0.0.3 0
24002 262139 10.0.255.1/32[V] 10.0.0.3 0
24003 262138 10.0.255.10/32[V] 10.0.0.3 0
24012 262134 10.0.255.2/32[V] 10.0.0.3 0 24013 Pop 10.0.0.3/32[V] Gi0/0/0/0.11 10.0.0.3 0
Conclusion
BGP is wonderful protocol. I don’t know, which protocol else has so many different options and flexibility. It can be very easy and straightforward, or it can be really complex. It depends only on your design and skills. BGP labeled unicast is widely used in Service provider world and in high-scale Data Centers as well. For sure there are other options, like VXLAN, which also can be used as transport technology for DC (another option is MPLS). It’s good that we have so many options. And BGP-LU is definitely is one of the best one. Take care.
