Hello my friend,
I haven’t been writing for a while due to necessity to finalize one very important project. Now I’m back to blogging and we’ll talk in this and a couple of next articles about some technologies that are intend to be used for SDN deployment in service provider networks.
1 2 3 4 5 | No part of this blogpost could be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical or photocopying, recording, or otherwise, for commercial purposes without the prior permission of the author. |
Disclaimer
I had to pause blogging because I was on a final run for CCIE SP lab exam preparation. Now I’ve done it and I’m 2x CCIE in Routing and Switching and in Service Providers. I was joking with my wife that now I officially know how Internet works. Besides jokes, it was really hard preparation and examination track, and for sure very useful for me, because a lot of my previous articles were inspired by this preparation. And I see much more stuff that I’d like to cover, let’s see what I’ll be able to realize.
I also used this time to refresh myself a bit from writing, as I’ve spotted that I use the same words and phrases a lot, what makes it definitely boring to read. I don’t know, if it changes in future, but both you and I have relaxed from it.
Brief overview of SDN
Each and every vendor speaks now about SDN (Software Defined Networking). This topic isn’t new actually and has been being discussed for last 5 years at least. Initially it comes with OpenFlow 1.0. There were a lot of limitations in the first release and it has evolved much and not it’s used in many modern data center SDN installations, like Nuage by Nokia for instance. For sure it isn’t the only solution, I’m just writing in train without internet access so I don’t have anything else now in mind. Definitely Cisco has something as well, though mainly Cisco focuses on its own product Cisco ACI (Application Center Infrastructure), which has OpFlex as a main communication protocol. Back to OpenFlow. It is a control plane protocol, which instructs devices what to do with certain network flows. By network flows we understand any combination of source/destination Ethernet, IP, UDP, TCP or other addresses. By actions we understand forward (by providing next hop for flow), drop or any QoS action (like police). It’s very generic definition, but it gives the overview of that technology.
But this is Data Center technologies. The main difference from Service Provider is an operational environment. In Data Center you have quite stable and well predictable environment. There is no cable cut caused by constriction works, there is no microwave links flaps caused by heavy rain and so on. Why am I talking about it? It’s important to understand context, which network technologies are working in. In Data Center the probability to loose connectivity to SDN controller that is actually brain of your network is very low and you can fully rely on it (especially, if it’s redundant). In typical Service Provider network, you need to have much more abilities for the network to self-heal and operate in stand-alone mode. That’s why approach for SDN in SP networks must be different.
Brief overview of SR TE (Segment Routing Traffic Engineering)
It’s possible to talk a lot about “cloud castles”, but let’s back to the core topic of this article. You might be a little bit confused that article is called SDN sandbox 2, whereas you can’t find any article at my site that is called somewhat similar to “SDN sandbox 1”. That’s true, but I consider the article about Segment Routing as the first part of this “SDN sandbox”. In that article I’ve described how to configure Segment Routing and use it for so called “best effort routing”. It means that we don’t want to influence data path somehow and allow IGP to define it, whereas Segment Routing is used later on only for adding corresponding label to destination, much like LDP.
SR TE is a traffic engineering based on Segment Routing labels. A lot of advanced staff is typically done at the controller level, but at the router level itself we can do a lot as well. The main difference in SR TE from RSVP TE is that tunnel is stateless. It means that all the routers along the LSP are unaware that there is any tunnel. Only head-end router has information how the tunnel should look like. In order such approach can work, the headend router (ingress LSR) pushes the label stack with different labels (from 2 to platform limit). This label is being decreased upon passing corresponding segments. The image that I’ve made for the 1st article about Segment Routing explains the concept of SR TE (Segment Routing Traffic Engineering) very well:
What are we going to test?
We’ll configure SR TE between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR router with almost the simplest option, which includes definition of explicit path. TE path will be different from the one proposed by IGP.
Topology
To show the real power of any kind of traffic engineering it’s good to have a lot of routers. Unfortunately, I don’t have it, but my laptop allows me to launch 4 virtual routers. Half of them are Nokia (Alcatel-Lucent) VSR and half of them are Cisco IOS XRv (XRv 9000) routers. The physical topology of virtual routers has the following structure:
For the logical topology I’ll deploy partial mesh, so that I have a lot of possibilities to send the traffic between PEs:
As a IGP in this network I’ll used ISIS, just because I like ISIS more than OPSF, but you can easily replace it, if you like. The most important point is that all links have the same cost, so you could easily identify routing patterns in the network.
The initial configurations files are available here:
080_config_initial_sr1 080_config_initial_sr2 080_config_initial_xr3 080_config_initial_xr4 080_config_initial_linux
Configuration of SR TE between Nokia SR OS and Cisco IOS XR
Before we configure any fashion of SR TE, we need to enable segment routing.
| v | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
We don’t stay here for a long check, just briefly screen that ISIS propagate label SR information that is label block, index and adjacency labels. For variety we’ll check how ISIS LSP look like at Nokia VSR side:
|
A:SR2# show router isis database SR1.00-00 detail |
And at Cisco IOS XR side:
|
RP/0/0/CPU0:XR4#show isis database XR3.00-00 verbose |
You might spot interesting difference, that Cisco IOS XR allocates 2 labels for interconnect link, whereas Nokia allocates only one.
More information about Segment Routing configuration and verification you can find in the corresponding article.
After Segment Routing is enabled and labels are propagated, we come to the most interesting part of this article, where we deploy SR TE. The scenario is the following:
- Traffic from SR1 (Nokia SR OS routers) to XR3 (Cisco IOS XR router) should follow path SR1 -> SR2 -> XR3 -> XR4.
- Traffic from XR3 to SR1 should go XR3 -> XR4 -> SR1
To better understand that scenario, see the scheme below:
First of all we need to enable MPLS TE support, as Segment Routing TE uses TED (traffic engineering database):
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR1 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
| SR2 | XR4 |
|
A:SR2>edit-cfg# candidate view |
RP/0/0/CPU0:XR4(config)#show conf |
The first interesting difference is that in Nokia (Alcatel-Lucent) SR OS you have to enable both MPLS TE and RSVP in order to get MPLS TE information generated (in order TED to be populated in other words). In Cisco IOS XR you need enable only MPLS TE. Let’s check briefly the content of newly build MPLS TED:
|
RP/0/0/CPU0:XR3#show mpls traffic-eng topology brief | inc IGP |
Though this output might seem to long, it shows that all our routers has links to each other in TED so that it’s full and can be used for LSP build.
Now we proceed the configuration of SR TE tunnels :
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR |
| SR2 | XR3 |
|
A:SR1>edit-cfg# candidate view |
RP/0/0/CPU0:XR3(config)#show conf |
Now we have 2 SR TE tunnels. From the configuration point of view, you see that we define next-hop routers in a strict fashion, and we say nothing about labels. Pay attention that in Nokia (Alcatel-Lucent) SR OS we define specific LSP type “sr-te” upon LSP creation, whereas in Cisco IOS XR we define segment routing in “path-option”. Let’s check which labels are used for this tunnels.
From Nokia SR OS router SR2 we have the following:
|
A:SR1# show router mpls sr-te-lsp “SR1_TO_XR3” detail path “EP_SR_SR1_SR2_XR4_XR3” |
So this output says, if we use this SR-TE LSP, we send traffic to next-hop 10.11.22.22 (SR2) with two labels in stack: topmost is 262142 and bottom transport label is 24008. Both of these labels differ from Segment Routing block (16000 – 23999 is default for for Cisco IOS XR and we have configured 500000 – 519999 for Nokia SR OS). Seems strange? Well, these are adjacency labels for interconnected links (the output from ISIS LSDP is highly reduced):
|
A:SR1# show router isis database SR2.00-00 detail |
Logic is quite simple:
- When SR2 receives the packet with the topmost label 262142, it knows that it must pop it and send packet out to next hop 10.22.44.44
- When XR4 receives the packer with the topmost label 24008, it knows that it must pop it and send packet out to next hop 10.33.44.33
- When XR3 receives the rest of the packet, it will look into the MPLS service label, so no additional IP/MPLS lookup is needed
Let’s check Cisco IOS XR side now:
|
RP/0/0/CPU0:XR3#show mpls traffic-eng tunnels 1031 |
Cisco unfortunately doesn’t show directly all the label stack if we have to deal with MPLS TE tunnels, so we need to combine these two ouputs.
Cisco IOS XR is much easier than Nokia SR OS in terms of checking MPLS TE as you can use TE tunnel for all traffic, not only to services.
Keeping in mind the tip above, let’s test TE tunnel:
|
RP/0/0/CPU0:XR3#traceroute 10.0.0.11 so 10.0.0.33 |
Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR (actually, IOS / IOS XE / NX-OS) behaves in different fashion regarding PHP. Cisco has it by default and Nokia doesn’t have. Based on that fact, we have 2 labels in label stack as well:
- When XR4 receives the packet with the topmost label 16011, it swaps this label and send packet to next hop 10.11.44.11
- When XR1 receives the packet with the topmost label 500011, it removes this label and make the next IP/MPLS lookup to understand, what to do with the traffic further.
Verification of Segment Routing Traffic Engineering
To show you these packets in live I’ll configure simple BGP/MPLS IP VPN between SR1 and XR3 to make use of these SR-TE tunnels. I don’t provide configuration for brevity, just one output per device. Nokia VSR:
|
A:SR1# show router 10 route-table |
Cisco XRv 9000:
|
RP/0/0/CPU0:XR3#show route vrf TEST ipv4 |
Refer to the dedicated article for L3 VPN configuration between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR.
Now let’s make ping test and make Wireshark snapshot.
|
RP/0/0/CPU0:XR3#ping vrf TEST 192.168.11.11 so 192.168.33.33 |
And here is the Wireshark packet capture:
The final configuration files are here:
080_config_final_sr1 080_config_final_sr2 080_config_final_xr3 080_config_final_xr4
Lessons learned
What is very interesting in the output above is that label stack in way from XR3 to SR1 has only 2 labels instead of 3. If we think a minute and see my explanation above, it should be 3 labels (16011 / 500011 / BGP Label). But when XR4 makes swap 16011 to 500011, it would then (500011 / 500011 / BGP Label), what is obviously not efficient. I see this packet dump for the first time myself, but I find it cool that there is such kind of label stack optimisation in place.
Conclusion
SDN is slowly coming in SP world and we need to understand not only how it work, but also how we can benefit from it. Evolution of Segment Routing to SR TE gives you possibility to have single protocol for all kind of the routing. BUT (hello, Frank :-)) it makes sense only if the network has such kind of demand or topology, where traffic engineering is necessary. Each technology is only as good as the business case it’s driven by. SR TE is evolving set right now and there are numerous limitations now (like absence of BW reservation of lack of MVPN support), but I’ m sure it’s only temporary issue and the solution will be developed soon. Take care and good bye!
Support us
BR,
Anton Karneliuk
Hi Anton, what versions of OSes did you use for this lab?
Hi Pat,
For Nokia (Alcatel-Lucent) SR OS I use 14.0.R4 and for Cisco IOS XR it’s 6.1.2
BR,
Anton