Hello my friend,
One of my friends said that he has started implementing on Cisco routers one interesting feature that is subject of the article. Then I have found its description in slides from Cisco Live 2017. I decided to dig into it deeper and see how interoperable it is. Some results are unexpected.
1 2 3 4 5 | No part of this blogpost could be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical or photocopying, recording, or otherwise, for commercial purposes without the prior permission of the author. |
Disclaimer
Information in this article is provided on “as-is” basis. I haven’t managed to achieve 100% interoperability between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR. What was unexpected for me is that Cisco has shown lower level of support their own features in router itself. I can imagine two possible causes for that. Either I haven’t configured something properly, or Cisco solution demands centralized controller for that. That will be checked in next articles.
Brief overview
The presentation that I mention is called BRKSPG-2518, you can search for it at CiscoLive website (link). It explains Cisco’s vision of network evolution beyond seamless MPLS deployment. I want to talk here about one particular feature that is usage of unnumbered Ethernet interfaces. In nutshell unnumbered interfaces aren’t something new and they were used for decades in serial links with HLDC/PPP encapsulation, termination of clients at BNG via PPPoE and so on. The main idea of unnumbered interfaces is that you don’t configure any specific IPv4/IPv6 address at interface but you rather say interface to use IP address from different interface, which is typically system for Nokia (Alcatel-Lucent) SR OS or loopback 0 in Cisco IOS XR. As loopback interface usually have 32-bit-length prefix for IPv4 and 128-bit-length for IPv6, usage of such address at interface changes the rules for routing protocols (IGP). Both OSPF and ISIS demands that IPv4 addresses at common link between two routers belong to the same subnet.
Well, let’s think for a moment about communication flow at point-to-point links. Initially the 30-bit-length prefix was used to provide such connectivity. Such prefix provide in total 4 IPv4 addresses, whereas only two of them (2nd and 3rd) can be used for addressing of the interface. 1st and 4th IPv4 addresses are network address and broadcast for this subnet and they can’t be used for interface addressing. It means that effectivity of IP address usage at point-to-point links were as low as 50%. Then in xxxx RFCXXXX (link) was issued that allowed to use 31-bit-length prefixes at point-to-point links, as there are only two hosts and there is no necessity to send broadcast neither have subnet address. It allowed ISPs across all the world to increase usage of IPv4 addresses at interconnects links up to 100% and provided a lot of free IPv4 addresses that can’t be used for customers or services.
As I mentioned earlier, for point-to-point encapsulations that is HLDC, PPP or point-to-point sub interfaces in Frame-Relay it was possible even to go further and make use of its loopback for interconnect interfaces. It means that you don’t need to lose ANY IPv4 address for interconnect links. For Ethernet it wasn’t possible as Ethernet isn’t considered to be point-to-point link, as it’s rather multi-access. Now it’s possible.
What does it mean? It means that at P routers you technically need ONLY ONE IP address. Well, from management prospective you might have more, but one IP address will be enough to make traffic forwarding between other P/PE devices. At PE/ASBR routers you might have still 30/31 prefix to the customers, but it also isn’t 100% needed. So, in time when we have lack of available public IPv4 addresses, we can reuse then from interconnect links as they aren’t needed there anymore.
What are we going to test?
I’m going to start with initial configuration from previous lab (link) and to convert that network to use unnumbered Ethernet interfaces. We’ll test IPv4/IPv6 forwarding and then we’ll test MPLS forwarding with LDP, RSVP/MPLS-TE and segment routing. To make use of MPLS data plane, we’ll create simple BGP/MPLS IP VPN (for IPv4/IPv6) and test connectivity of redesigned network.
Software version
For tests in this lab I use the following versions of software for routers:
- Nokia (Alcatel-Lucent) SR OS 14.0.R4
- Cisco IOS XRv 6.1.2
I haven’t used to mention it previously, though it’s very important to understand, which particular SW allows us to achieve the interoperability and which not.
Topology
Our physical topology will be formed from 4 routers, 2 of them are Nokia (Alcatel-Lucent) VSR and two of them are Cisco IOS XRv:
If you have questions, how to establish the lab, I advise you to read the very first article in series.
As I’ve already mentioned before, we’ll use the topology from the previous lab as initial basis:
I’m intend to use the existing topology because it’s easy to compare the results, changes and so on with the previous lab to assess the changes. The only difference, I always say that you can use routing protocol whatever you like. This time it makes sense for you to start with ISIS as it’s provided in my config. You will see later on why.
The initial configurations files are available here: 080_config_initial_linux 080_config_initial_xr3 080_config_initial_xr4 080_config_initial_sr2 080_config_initial_sr1
Configuration of unnumbered IPv4 interfaces
Before we start, just brief check that connectivity between our nodes is up and running:
|
RP/0/0/CPU0:XR3#ping 10.0.0.11 so 10.0.0.33 |
It looks so far good and now we can start. Today the core point is to change configuration of the Ethernet interfaces as unnumbered for IPv4:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
During the changes the configuration of the interfaces, the ISIS adjacency is continuously flapping. We haven’t configured logging at Nokia (Alcatel-Lucent) SR OS, that’s why we see the indication at Cisco IOS XR:
|
RP/0/0/CPU0:XR4(config)#RP/0/0/CPU0:Aug 4 18:16:09.471 : isis[1011]: %ROUTING-ISIS-6-INFO_STARTUP_FINISH : Cold controlled start completed |
If you want to know how to enable and fine tune logging in Nokia (Alcatel-Lucent) SR OS or Cisco IOS XR, read the corresponding article.
As you see, just after we changes the interface configuration on side, we receive the syslog message that IPv4 address of the peering router isn’t in the common range so that the adjacency went down. Later, when we change the peer configuration as well, the ISIS peering went up.
The first check relates to the routing and ARP table, because both of them are critically important for connectivity. At Nokia (Alcatel-Lucent) SR OS router SR1 you see the following situations:
|
A:SR1# show router interface |
Now we check the same tables at Cisco IOS XR router XR3:
|
RP/0/0/CPU0:XR3#show ipv4 interface brief |
This output shows very interesting difference between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR. Nokia (Alcatel-Lucent) VSR automatically generates prefix from IPv4 link-local range (169.254.0.0/16), whereas Cisco doesn’t do it. On the other hand, you can’t find this IPv4 address somewhere else.
The next check will be related the ISIS LSDP, where we see, what is distributed into LSP by different routers. The following output is grabbed from XR3:
|
RP/0/0/CPU0:XR4#show isis database SR1.00-00 verbose |
You might spot that the LSP are generated in a bit different manner. Nokia (Alcatel-Lucent) SR OS extends information of adjacent ISIS by adding IDs of local/remote interfaces, whereas Cisco IOS XR doesn’t do it.
Now we can make some tests to check basic connectivity:
|
A:SR1# ping 10.0.0.11 count 1 |
Configuration of unnumbered IPv6 interfaces
In IPv6 addressing we have something called link-local address (from range fe80::/13) which are automatically generates as soon as you enable any interface (including system/loopback) for IPv6 routing. It’s a must, because almost all protocols responsible for building IPv6 forwarding plane (OSPF/ISIS, PIM, ICMP and on) uses these IPv6 link-local address (LLA) for communication. Actually, it’s enough to configure IPv6 LLA manually or just say that interface is IPv6 enabled in Cisco and that’s it. As this is the specific range and it isn’t routable beyond this particular interface that’s why we can’t save any IP addresses as we don in IPv4.
Unnumbered Ethernet interfaces and MPLS data plane using LDP
Let’s see which implication on other technologies has such approach of interface configuration. First of we’ll configure LDP in our network and see how label propagation/mapping is done.
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
More information about LDP configuration you can find in the corresponding article.
MPLS LDP is up and running and we just briefly screen the labels in LFIB. Nokia (Alcatel-Lucent) SR OS router SR1 has the following information there:
|
A:SR1# show router fp-tunnel-table 1 |
You see that next-hops look very strange. I’ve checked, there is no mapping to interface IDs in ISIS LSP, so it might be related to something different or be just the form of the placeholder. Nevertheless, it works. Now it’s turn of Cisco IOS XR router XR3 to show its details of LFIB:
|
RP/0/0/CPU0:XR3#show mpls forwarding |
Here you see the next hop as it is meaning the unnumbered interface IPv4 address from another router.
BGP/MPLS IPv4/IPv6 VPN service on top
To check the established MPLS data plane we’ll create a simple L3 VPN to generate the traffic:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
More information about configuration of BGP/MPLS IP VPN in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR you can find in the dedicated article.
We check the routing tables in corresponding VRF/VPRN and see how the tables ae populated.
Nokia (Alcatel-Lucent) VSR (SR 7750) router SR1:
|
A:SR1# show router 10 route-table |
Cisco IOS XRv 9000 (ASR 9000) side is represented by XR3:
|
RP/0/0/CPU0:XR3#show route vrf TEST ipv4 |
To make more interesting Wireshark oouput between SR1 and XR3 we’ll break the direct links and use another path. According to our topology we should see the following paths:
- Forward path is SR1 -> XR4 -> XR3;
- Backward path is XR3 – > XR4 -> SR1;
We make a ping and capture traffic in Wireshark. For IPv4 it looks like the following:
|
A:SR1# ping router 10 192.168.4.10 |
Wireshark output for IPv4:
And for IPv6:
|
A:SR1# ping router 10 fc00::192:168:4:10 |
Wireshark output for IPv6:
You see it’s working without problems. So the solution with unnumbered Ethernet interfaces is looking very promising so far.
Unnumbered Ethernet interfaces and MPLS data plane using Segment Routing
I’ve found the information about such interfaces configuration in context of Cisco SDN with Segment Routing, so it definitely should work. Let’s convert our MPLS data plane from LDP-based to Segment Routing based:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
More information about Segment Routing configuration in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR you can find in the dedicated article.
Though it isn’t necessary I deconfigure LDP in order to have clear setup with one protocol that forms MPLS data plane. We won’t spend a lot of time on checking the data plane. Rather we’ll focus only on LFIB. First of all let’s check the information at SR1, which is Nokia (Alcatel-Lucent) VSR (SR 7750):
|
A:SR1# show router fp-tunnel-table 1 |
Here is another PE XR4, which is Cisco IOS XRv 9000 (ASR 9000):
|
RP/0/0/CPU0:XR3#show mpls forwarding |
We see the Segment Routing labels right now in LFIB in for both routers.
Now we check, how customer traffic is transmitted over the network. This time we do it from Cisco IOS XR router XR3:
|
RP/0/0/CPU0:XR3#ping vrf TEST 192.168.1.10 so 192.168.4.10 |
Wireshark output for IPv4:
For IPv6:
|
RP/0/0/CPU0:XR3#ping vrf TEST fc00::192:168:1:10 so fc00::192:168:4:10 |
Wireshark output for IPv6:
Such solution is also working, what is definitely good. Let’s move forward.
Unnumbered Ethernet interfaces and MPLS data plane using Segment Routing Traffic Engineering
No we have come to the very important part of the article. We have identified previously that best-efforts routing (we don’t influence path selection) works without problem. Now we want to add traffic engineering support:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
More information about Segment Routing Traffic Engineering in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR you can find in the corresponding article.
The configuration is pretty much the same as in the link above. But let’s check the MPLS TED (Traffic Engineering Database):
|
RP/0/0/CPU0:XR3#show mpls traffic-eng topology brief |
Critical point up to now. Cisco IOS XR (version 6.1.2) doesn’t populate TED if we use unnumbered Ethernet interface and ISIS as IGP. It means that we won’t be able to build SR TE LSP, because CSPF can’t be performed. In the same time Nokia (Alcatel-Lucent) SR OS populates TED without problems and/or additional configuration.
To be continued …
The configuration files are here: 082_config_final_sr1 082_config_final_sr2 082_config_final_xr3 082_config_final_xr4
Lessons learned
This time it’s important stick to a certain IGP, as I’ve said in the very beginning. In the next article we’ll play with additional staff to see what and how we can fix. I’ve just occasionally found that behavior of Cisco in ISIS and OSPF relating this case very different. And I spot it just because it didn’t work for me in ISIS. It means that probably some of my previous articles written on top of ISIS might not work for OSPF and vice versa. I If you find such caveats, please, let me know.
Conclusion
Initially I was thinking to cover everything in one article, but it’s already 24 pages in Word (where I’m writing it) so it would be definitely long to read. So I decided to split the article in two, and we’ll talk in the next chapter about SR-TE and RSVP/MPLS-TE in conjunction with unnumbered Ethernet interfaces. We speak about brand new technologies and concepts that are being developed just right now, so I assume there will be fixes for that. Probably I have missed something in configuration in Cisco IOS XR as there is still no clear documentation, even official configuration guide doesn’t provide answers. But in any case I’m very excited, because two cases are working and in case you don’t need MPLS TE, you can start profit from that approach right now. Take care and good bye!
Support us
BR,
Anton Karneliuk
