SDN sandbox 3. Unnumbered Ethernet interface in Nokia SR OS and Cisco IOS XR. Part 2.
Anton Karneliuk
Hello my friend,
The last time we have started interesting discussion regarding configuration of unnumbered Ethernet interfaces. We have figured out that Cisco IOS XR has strange behavior and we’ll try to solve it in this part.
1 2 3 4 5
No part of this blogpost could be reproduced, stored in a
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
Instead of introduction
This article isn’t standalone. It’s continuation of the previous one, so we use the topology and existing configuration from that one. I advise you to read it beforehand, so you will be synchronized with the actual state, what we are talking about.
Change of IGP from ISIS to OSPF
As we have figured out in the end of the previous article, there Cisco IOS XR doesn’t generate information in MPLS TED (Traffic Engineering Database). Probably, it’s a bug or limitation of XRv itself or IOS XR version 6.1.2, I don’t know. I’ve started thinking, what I could do. I’ve tried a lot of different options on my Cisco IOS XRv routers like enabling/disabling RSVP on interfaces, playing with different MPLS-TE and Segment Routing parameters, but unfortunately it hasn’t brought anything that solves this issue. Then I decided to change routing protocol in my network, which my small service provide core actually, to OSPF. Let’s see how to do it:
Nokia (Alcatel-Lucent) SR OS
Cisco IOS XR
SR1
XR3
A:SR1>edit-cfg# candidate view
=========================
configure
router
isis shutdwon
no isis
ospf
advertise-router-capability area
traffic-engineering
segment-routing
prefix-sid-range global
tunnel-table-pref 8
no shutdown
exit
area 0.0.0.0
interface “system”
node-sid index 11
passive
no shutdown
exit
interface “toSR2”
interface-type point-to-point
no shutdown
exit
interface “toXR3”
interface-type point-to-point
no shutdown
exit
interface “toXR4”
interface-type point-to-point
no shutdown
exit
exit
no shutdown
exit
exit
exit
=========================
RP/0/0/CPU0:XR3(config)#show conf
no router isis CORE
router ospf 1
router-id 10.0.0.33
segment-routing mpls
area 0.0.0.0
network point-to-point
mpls traffic-eng
interface Loopback0
prefix-sid index 33
!
interface GigabitEthernet0/0/0/0.13
!
interface GigabitEthernet0/0/0/0.34
!
!
mpls traffic-eng router-id Loopback0
!
end
SR2
XR4
A:SR2>edit-cfg# candidate view
=========================
configure
router
isis shutdown
no isis
ospf
advertise-router-capability area
traffic-engineering
segment-routing
prefix-sid-range global
tunnel-table-pref 8
no shutdown
exit
area 0.0.0.0
interface “system”
node-sid index 22
passive
no shutdown
exit
interface “toSR1”
interface-type point-to-point
no shutdown
exit
interface “toXR4”
interface-type point-to-point
no shutdown
exit
exit
no shutdown
exit
exit
exit
=========================
After the network has reconverged, we briefly check the routing table to confirm that:
RP/0/0/CPU0:XR3#show route ipv4
O 10.0.0.11/32 [110/200] via 10.0.0.44, 00:06:01, GigabitEthernet0/0/0/0.34
O 10.0.0.22/32 [110/200] via 10.0.0.44, 00:06:01, GigabitEthernet0/0/0/0.34
L 10.0.0.33/32 is directly connected, 03:56:18, Loopback0
O 10.0.0.44/32 [110/101] via 10.0.0.44, 00:06:01, GigabitEthernet0/0/0/0.34
The routing table looks so far good. Actually, in ISIS pure IPv4 routing was also working for unnumbered Ethernet interfaces without problems. What about packet forwarding? Let’s see:
A:SR1# ping 10.0.0.11 count 1
PING 10.0.0.11 56 data bytes
64 bytes from 10.0.0.11: icmp_seq=1 ttl=64 time=0.448ms.
—- 10.0.0.11 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 0.448ms, avg = 0.448ms, max = 0.448ms, stddev = 0.000ms
!
!
A:SR1# ping 10.0.0.22 count 1
PING 10.0.0.22 56 data bytes
64 bytes from 10.0.0.22: icmp_seq=1 ttl=64 time=16.6ms.
—- 10.0.0.22 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 16.6ms, avg = 16.6ms, max = 16.6ms, stddev = 0.000ms
!
!
A:SR1# ping 10.0.0.33 count 1
PING 10.0.0.33 56 data bytes
64 bytes from 10.0.0.33: icmp_seq=1 ttl=255 time=43.3ms.
—- 10.0.0.33 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 43.3ms, avg = 43.3ms, max = 43.3ms, stddev = 0.000ms
!
!
A:SR1# ping 10.0.0.44 count 1
PING 10.0.0.44 56 data bytes
64 bytes from 10.0.0.44: icmp_seq=1 ttl=255 time=20.4ms.
—- 10.0.0.44 PING Statistics —-
1 packet transmitted, 1 packet received, 0.00% packet loss
round-trip min = 20.4ms, avg = 20.4ms, max = 20.4ms, stddev = 0.000ms
I won’t make the same configuration for OSPF to check LDP and Segment Routing, because it was working in ISIS. Maybe something changes here, but it will be again large amount of text. So I assume that that both these MPLS data plane technologies are working in OSPF as well, so I can focus solely on Traffic Engineering (first of all, Segment Routing Traffic Engineering).
For ISIS there were missing entries in MPLS TED from Cisco IOS XR side. Let’s see how it looks like now:
At a glance we see the improvement with the new configuration. Now we have full MPLS TED and theoretically should be able to build a Segment Routing Traffic Engineering LSP. Just of curiosity point of view, pay attention to IGP metric both in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR routers. It has the maximum possible value, whereas TE metric is correct and is related to actual bandwidth.
Read this article to get understanding, how to make SR TE in general.
According to IGP calculation, the shortest path between SR1 and XR3 is SR1 <-> XR4 <-> XR3. Let’s configure SR-TE tunnel so that traffic takes path SR1 <-> SR2 <-> XR4 <-> XR3.
The following configuration we put on our service provide PE routers represented by Nokia (Alcatel-Lucent) VSR 7750 and Cisco IOS XRv 9000 (ASR 9000):
Nokia (Alcatel-Lucent) SR OS
Cisco IOS XR
SR1
XR3
A:SR1>edit-cfg# candidate view
=========================
configure
router
mpls
path “EP_SR1_SR2_XR4_XR3”
hop 10 10.0.0.22 strict
hop 20 10.0.0.44 strict
hop 30 10.0.0.33 strict
no shutdown
exit
lsp “SR_TE_LSP_SR1_TO_XR4” sr-te
to 10.0.0.33
cspf
primary “EP_SR1_SR2_XR4_XR3”
exit
no shutdown
exit
exit
exit
exit
=========================
RP/0/0/CPU0:XR3(config)#show conf
explicit-path name EP_SR_TE_XR3_XR4_SR2_SR1
index 10 next-label 16022
index 20 next-label 500011
!
interface tunnel-te1031
ipv4 unnumbered Loopback0
signalled-name SR_TE_XR3_TO_SR1
autoroute announce
!
destination 10.0.0.11
record-route
path-option 10 explicit name EP_SR_TE_XR3_XR4_SR2_SR1 segment-routing
!
end
The configuration is pretty much the same as it was in the mentioned article. The only difference is that I have specified exact label stack rather than next-hop IPv4 addresses. We start checking configured MPLS SR TE LSP from Nokia (Alcatel-Lucent) SR OS router SR1:
A:SR1# show router mpls sr-te-lsp path detail
===============================================================================
MPLS SR-TE LSP Path (Detail)
===============================================================================
Legend :
S – Strict L – Loose
===============================================================================
——————————————————————————-
SR-TE LSP SR_TE_LSP_SR1_TO_XR4 Path EP_SR1_SR2_XR4_XR3
——————————————————————————-
LSP Name : SR_TE_LSP_SR1_TO_XR4
Path LSP ID : 11264
From : 10.0.0.11 To : 10.0.0.33
Admin State : Up Oper State : Up
Path Name : EP_SR1_SR2_XR4_XR3 Path Type : Primary
Path Admin : Up Path Oper : Up
Path Up Time : 0d 00:01:20 Path Down Time : 0d 00:00:00
Retry Limit : 0 Retry Timer : 30 sec
Retry Attempt : 1 Next Retry In : 0 sec
!
CSPF : Enabled Oper CSPF : Enabled
!
Bandwidth : No Reservation Oper Bandwidth : 0 Mbps
Hop Limit : 255 Oper HopLimit : 255
Setup Priority : 7 Oper Setup Priority : 7
Hold Priority : 0 Oper Hold Priority : 0
Inter-area : N/A
!
PCE Updt ID : 0 PCE Updt State : None
PCE Upd Fail Code: noError
!
PCE Report : Inherited Oper PCE Report : Disabled
PCE Control : Disabled Oper PCE Control : Disabled
PCE Compute : Disabled
!
Include Groups : Oper Include Groups :
None None
Exclude Groups : Oper Exclude Groups :
None None
!
IGP/TE Metric : 16777215 Oper Metric : 16777215
Oper MTU : 1488 Path Trans : 1
Failure Code : noError
Failure Node : n/a
Explicit Hops :
. 10.0.0.22(S) -> 10.0.0.44(S) -> 10.0.0.33(S)
Actual Hops :
. 10.0.0.22, If Index : 2 Record Label : 262142
.-> 10.0.0.44, If Index : 7 Record Label : 262142
.-> 10.0.0.33, If Index : 9 Record Label : 24002
===============================================================================
The LSP is shown as up and running. As we figured out earlier, it is impossible to use mpls oam toolkit (at least now) to trace SR TE tunnel. So we’ll see later on in MPLS VPN output, how the label stack looks like. Cisco IOS XR router XR3 also should build LSP:
RP/0/0/CPU0:XR3#show mpls traffic-eng tunnels
Name: tunnel-te1031 Destination: 10.0.0.11 Ifhandle:0xf0
. Signalled-Name: SR_TE_XR3_TO_SR1
. Status:
. Admin: up Oper: down Path: not valid Signalling: Down
.
. path option 10, (Segment-Routing) type explicit EP_SR_TE_XR3_XR4_SR2_SR1
. Last PCALC Error: Fri Aug 4 23:13:17 2017
. Info: No path to destination, 10.0.0.22 (ipaddr) . G-PID: 0x0800 (derived from egress interface properties)
. Bandwidth Requested: 0 kbps CT0
. Creation Time: Fri Aug 4 23:13:17 2017 (00:14:25 ago)
. Config Parameters:
. Bandwidth: 0 kbps (CT0) Priority: 7 7 Affinity: 0x0/0x0
. Metric Type: TE (global)
. Path Selection:
. Tiebreaker: Min-fill (default)
. Hop-limit: disabled
. Cost-limit: disabled
. Path-invalidation timeout: 10000 msec (default), Action: Tear (default)
. AutoRoute: enabled LockDown: disabled Policy class: not set
. Forward class: 0 (default)
. Forwarding-Adjacency: disabled
. Autoroute Destinations: 0
. Loadshare: 0 equal loadshares
. Auto-bw: disabled
. Fast Reroute: Disabled, Protection Desired: None
. Path Protection: Not Enabled
. BFD Fast Detection: Disabled
. Reoptimization after affinity failure: Enabled
. Soft Preemption: Disabled
It should, but unfortunately not. I omit outputs of all cases, though I tried to configure all possible options of explicit-path in Cisco IOS XR (next-labels node/adjacent, next-hop, IP/unnumbered). I figured out the following interesting moment. Let’s briefly recall MPLS TED:
In my lab (with Cisco IOS XRv 6.1.2), Cisco router neither generate nor take into account Intf ID of neighbour in order to build SR TE LSP. Nokia (Alcatel-Lucent) SR OS does it automatically, and I assume that is the reason why it works there. I’ve tried to configure another type of explicit-path in Cisco IOS XR:
RP/0/0/CPU0:XR3#show conf
!
explicit-path name EP_SR_TE_XR3_XR4_SR2_SR1
index 10 next-address strict ipv4 unicast unnumbered 10.0.0.44 2
index 20 next-address strict ipv4 unicast unnumbered 10.0.0.22 7
index 30 next-address strict ipv4 unicast unnumbered 10.0.0.11 2
!
end
As I don’t see Intf ID explicitly in MPLS TED, I’ve copied them from OSPF LSDB:
RP/0/0/CPU0:XR4#show ospf database router 10.0.0.33
. OSPF Router with ID (10.0.0.44) (Process ID 1)
.
. Router Link States (Area 0.0.0.0)
.
. Routing Bit Set on this LSA
. LS age: 355
. Options: (No TOS-capability, DC)
. LS Type: Router Links
. Link State ID: 10.0.0.33
. Advertising Router: 10.0.0.33
. LS Seq Number: 80000005
. Checksum: 0x8f2
. Length: 48
. Number of Links: 2
.
. Link connected to: a Stub Network
. (Link ID) Network/subnet number: 10.0.0.33
. (Link Data) Network Mask: 255.255.255.255
. Number of TOS metrics: 0
. TOS 0 Metrics: 1
.
. Link connected to: another Router (point-to-point)
. (Link ID) Neighboring Router ID: 10.0.0.44
. (Link Data) Router Interface address: 0.0.0.9
. Number of TOS metrics: 0
. TOS 0 Metrics: 100
Probably it’s wrong, but I haven’t found any other similar indicators. Anyway, SR TE LSP is down in Cisco IOS XR, but now for another reason:
So, despite the fact I have configured next-hop unnumbered addresses (actually, router-id), it still can’t be used.
Well, Nokia (Alcatel-Lucent) SR OS router has shown that it’s MPLS SR TE LSP is up and running. Let’s make a check by using ping in L3 VPN created in the previous part of this article. But before we need slightly modify the VPN configuration at SR1 so that next-hop is resolved to SR-TE tunnel rather than to “ordinary” Segment Routing label:
Now we see that next-hop at SR1 is resolved to SR TE tunnel:
A:SR1# show router 10 route-table
===============================================================================
Route Table (Service: 10)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
. Next Hop[Interface Name] Metric
——————————————————————————-
192.168.1.10/32 Local Local 00h15m24s 0
. TEST 0
192.168.4.10/32 Remote BGP VPN 00h03m16s 170 . 10.0.0.33 (tunneled:SR-TE:655362) 0
——————————————————————————-
No. of Routes: 2
Flags: n = Number of times nexthop is repeated
. B = BGP backup route available
. L = LFA nexthop available
. S = Sticky ECMP requested
===============================================================================
My assumption is that traffic from SR1 to XR3 will follow this SR TE tunnel and therefore it will have corresponding MPLS label stack. The backward traffic from XR3 to SR1 will be forwarded based on best-effort routing and therefore will have only on transport label (and one service on top for sure). What do you think? Is my assumption correct?
A:SR1# ping router 10 192.168.4.10
PING 192.168.4.10 56 data bytes
64 bytes from 192.168.4.10: icmp_seq=1 ttl=255 time=11.2ms.
64 bytes from 192.168.4.10: icmp_seq=2 ttl=255 time=9.17ms.
64 bytes from 192.168.4.10: icmp_seq=3 ttl=255 time=16.4ms.
64 bytes from 192.168.4.10: icmp_seq=4 ttl=255 time=9.08ms.
64 bytes from 192.168.4.10: icmp_seq=5 ttl=255 time=12.4ms.
—- 192.168.4.10 PING Statistics —-
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min = 9.08ms, avg = 11.6ms, max = 16.4ms, stddev = 2.67ms
Ping between sites of our customer emulated by two interfaces is working, what is definitely good. But now it’s more important to understand details, how it’s working:
The output from Wireshark fully confirms my assumption. The following explanation is related to forward path from SR1 to XR3:
You see that ICMP packet has 3 labels (2 transport + 1 service), when it leaves SR1.
Then SR2 pops one label and forwards this packet to XR4 with only 2 labels (1 transport + 1 service).
XR4 pops the only left transport label and sends this ICMP packet to XR3 with a single service label.
At the backward path from XR3 to SR1 we have “ordinary” Segment Routing operation:
XR3 pushes 2 labels (1 transport + 1 service) on the packet and sends it to XR4.
XR4 swaps topmost label, which is the transport one, and sends the packet according to its LFIB directly toSR1.
As usual, the devil is in detail. When something doesn’t work you have to dig much deeper than if everything is OK. It’s very time consuming, but in reality it’s very useful for us, engineers. Otherwise we won’t develop further,
Conclusion
Based on two parts of this article I can say that Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR has different implementations of SR TE. Probably it’s caused by virtual environment, as VSR is fully commercial product with full IP/MPLS stack support, whereas Cisco IOS XRv (at least my version) has numerous limitations. In general I have achieved all desired results in terms of usage of unnumbered Ethernet interfaces in Nokia (Alcatel-Lucent) SR OS routers and moderate results in Cisco IOS XR. I believe, colleagues from Cisco will solve it (or probably it’s already solved in hardware solution and/or another version of IOS XR). Take care and good bye!
