Hello my friend,
So far we have reviewed two technology of building MPLS data plane (transport tunnel in Nokia (Alcatel-Lucent) terminology) that are LDP and BGP labeled unicast. Today we continue our journey with the third option, which is called RSVP-TE and provides you wide possibilities for Traffic Engineering (TE).
Disclaimer
Nokia (Alcatel-Lucent) SR OS has option to build so called “non-TE” RSVP-based MPLS tunnels (LSP – label switched path). In this case you don’t have to enable IGP extensions (OSPF-TE or ISIS-TE), but you don’t have all fancy features like link coloring, SRLG, Fast Reroute and so on. Moreover this option isn’t interoperable with Cisco IOS XR, and to be frank, I don’t know another vendor, who supports it. So in this article we’ll configure “ordinary” TE MPLS tunnel that implies creation and usage of TED (Traffic Engineering Database).
Topology
The physical topology remains the same as always:
The logical topology was also used previously in LDP article:
The initial configuration we’ll use also from that article. It will be easier for you to compare the configuration and operation of two different MPLS-forming technologies in the same conditions:
Preparation for building LSP (MPLS tunnel)
First of all we need to enable support of MPLS traffic engineering and RSVP in our network. To achieve it we’ll activate TE extension to routing protocol (as IGP we use ISIS, but the configuration for OSPF is pretty similar). Afterwards we enable RSVP and MPLS at all interfaces as all of them take part in our MPLS forwarding plane:
| Nokia (Alcatel-Lucent) VSR (SR 7750) | Cisco IOS XR (ASR 9000) |
| SR1 | XR1 |
|
A:SR1>config>router# info |
RP/0/0/CPU0:XR1(config)#show conf |
| SR3 | |
|
A:SR3>config>router# info |
I assume that some explanation and verification is needed before we go further, as Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR behaves a little bit different:
| Nokia (Alcatel-Lucent) SR OS | Cisco IOS XR | |
| ISIS-TE extension | You simple enable TE and that’s it. Router-id is derived from global routing configuration | You chose for which level you want activate TE extension and chose interface for router-id |
| RSVP | Interfaces are automatically derived from MPLS context. Reservable BW is derived from capacity of physical port associated with interface. | Interfaces and reservable BW must be explicitly configured. |
| MPLS TE | MPLS LDP must be activated for proper operation. MPLS OAM must be explicitly activated. |
So these differences in behavior must be taken into account during configuration of MPLS/RSVP-TE in multivendor network based on Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR.
After we’ve applied the configuration above, we can check if our interfaces are operational in MPLS and RSVP contexts. In Nokia (Alcatel-Lucent) SR OS you can check it as follows:
|
A:SR1# show router mpls interface |
In Cisco IOS XR the commands are very similar:
|
RP/0/0/CPU0:XR1#show mpls interfaces |
In Cisco IOS XR the output of the command “show mpls interfaces” shows all type of protocols that can be used for build MPLS data plane, where “tunnel” means MPLS traffic engineering. In Nokia (Alcatel-Lucent) SR OS the output of “show router mpls interfaces” shows operation of interfaces for RSVP-TE based MPLS.
You can refer to article about LDP, where we haven’t activated MPLS context at Nokia (Alcatel-Lucent) VSR (SR 7750).
The next interesting point, what should be checked and what will be actively used, is TED (traffic engineering database). It’s information is distributed using ISIS updates in specific TLVs, so ISIS database is what we need. I’ll provide only part of this LSP (link-state packet in case of ISIS) that is related traffic engineering, because the whole LSP is really big:
|
A:SR1# show router isis database SR1.00-00 detail |
In Cisco IOS XR you can use the very similar command “show isis database SR1.00-00 verbose” to see the detailed structure of LSP. But there is another useful command available at Cisco IOS XR:
|
RP/0/0/CPU0:XR1#show mpls traffic-eng topology brief |
I’ve reduced the output just to one node, as basically you see all nodes and their interconnection. So this is the actual TED built using information extracted from ISIS. If you omit keyword “brief”, you’ll see detailed information including BW utilization per priority.
The last thing that I want to show you, before going to MPLS tunnels themselves, is the CSPF tool in Nokia (Alcatel-Lucent) SR OS. It’s really powerful tool that allow you to determine the actual path of LSP from point head-end to tail-end MPLS router based on the mentioned constrains. This provide you valuable input on possible path before you configure (or activate) LSP. Take a look at the basic example:
|
A:SR3# tools perform router mpls cspf from 10.0.255.1 to 10.0.255.2 bandwidth 567 |
As constrains for example we added that we want to avoid certain next-hop and reserved BW should 567 Mbps. Pay attention that we do this calculation at SR3, though we are determining path between SR1 and XR1. It’s possible because all the routers has the same TED across the RSVP-TE domain, otherwise MPLS-TE won’t work.
Basic RSVP-TE LSP
Let’s shutdown the direct links between SR1 and XR3 so that the only available path between them pass SR3.
Now we can proceed with the establishment of MPLS LSP (label switch path) or MPLS tunnel from Nokia (Alcatel-Lucent) SR OS router SR1 to Cisco IOS XR router XR1. As the first step the LSP will be quite simple without any constrains (though it will be CSPF enabled in terms of Nokia (Alcatel-Lucent).
| Nokia (Alcatel-Lucent) VSR (SR 7750) | Cisco IOS XR (ASR 9000) |
| SR1 | XR1 |
|
A:SR1>config>router# info |
RP/0/0/CPU0:XR1(config)#show conf |
We’ve configured appropriate logging at SR1 and SR3 in order see the RSVP-TE and MPLS LSP operation in the real time.
In Nokia (Alcatel-Lucent) SR OS you create a separate object called “path”, which is a key component of MPLS LSP. You must enable it before enabling LSP, otherwise LSP will be operationally down. The only parameter that can be configured inside path is explicit path (we’ll talk about it later), and if there are no hops configured the path is considered to be totally loose and TED will be consulted to build exact path to destination. Keyword “cspf” means that TED will be used. As a source of MPLS LSP the system interface of the Nokia (Alcatel-Lucent) VSR (SR 7750) is used.
In Cisco IOS XR you manually mentioned, which interface should be used as source for MPLS tunnel. “Autoroute destination” mean that route to destination IP will be installed in routing table and will be available for routing look up for all other protocols. Also in Cisco IOS XR you have to explicitly enable “record-route” in order RRO to be built. “Path-option dynamic” in IOS XR equals “primary loose” in SR OS.
Just after you bring up (enters “no shutdown”) at MPLS LSP SR1_to_XR1 configured at SR1, you see the following log messages:
|
SR OS: |
The similar messages you see, when you make commit at Cisco IOS XR and configuration of MPLS tunnel is being applied. LSP is up and running, but before checking it lets overview some useful commands for trouble shooting.
|
A:SR1# show router rsvp session |
The output above shows you that all the routers along the path participates in RSVP. For each MPLS LSP there is a separate RSVP session. The most important information here is triplet “To/destination address” – “Tunnel ID/DPort” – “From/ExtTunID”, what forms SESSION object in RSVP and uniquely identifies MPLS LSP. This triplets remains stable until restart in Nokia (Alcatel-Lucent) SR OS. In Cisco IOS XR it remains the same even after restart, because you manually configure tunnel ID. At the same time LSP ID increases each time the tunnel flaps or is reoptimized (resignaled in Nokia (Alcatel-Lucent) terms).
For sure we are interested in the details, and one of the most interesting one is MPLS label that is used for traffic forwarding. You can check it as follows:
|
A:SR1# show router rsvp session detail | match expression “LSP|Label|From|Hop|Interface” |
Remember, value “3” for Label means implicit NULL and calls penultimate hop popping (PHP) at pre-last hop router. By default Cisco IOS XR signals this label and Nokia (Alcatel-Lucent) SR OS not.
Output of RSVP-TE show commands contains the information from control plane (to be more precise – LIB (Label Information Base)). To check forwarding plane including MPLS labels for configured LSPs, we have another option:
|
A:SR1# show router mpls lsp detail terminate |
In Nokia (Alcatel-Lucent) SR OS you need add specific keyword “transit” or “detail” to see the state of MPLS tunnel, if LSP doesn’t start at the device.
In MPLS/RSVP-TE model the most interesting and complex work is done at the head-end router that is router that calculates and establishes LSP. There will be big output following. At Nokia (Alcatel-Lucent) VSR (SR 7750) the details of establishing LSP you can check in the following command:
|
A:SR1# show router mpls lsp path detail |
There are a lot information above, so I advise you to spend some time here to familiarize yourself, if you aren’t. For now the most interesting information is two fields: “Computed Hops” and “Actual hops”. In “Computed hops” you see the results of CSPF calculation upon MPLS LSP establishment. In RSVP PATH messages these hops are copied to ERO (Explicit Route Object), which is analyzed by each router along the path from head-end to tail-end router. “Actual hops” contains information from RRO (Record Route Object) extracted from RSVP RESV message. These hops are identical, what is correct. Now let’s take a look at information that Cisco IOS XR tells us regarding its tunnel:
|
RP/0/0/CPU0:XR1#show mpls traffic-eng tunnels detail |
The provided information is identical, as all the data is necessary for tunnel establishment and RFC defined. You can find ERO in PATH message and RRO in RESV.
You can spot the difference in interpretation of these fields between our vendors. Nokia (Alcatel-Lucent) SR OS show egress interfaces of the LSP head-end as the first hop in the ERO/RRO, though technically this hop won’t be included in the RSVP packet header in ERO/RRO field. It will be stripped and inserted in HOP field in PATH message. Cisco IOS XR doesn’t include its egress’s interface IP address in ERO/RRO.
What else is worth to be mentioned is the difference in default setup/hold priorities for LSP. Nokia (Alcatel-Lucent) SR OS has 7/0 default priorities respectively and Cisco IOS XR has 7/7
Let’s check LSP operation. Remember, we have activated MPLS OAM toolset at Cisco IOS XR in the initial configuration, because it’s necessary for MPLS tunnel’s check. Nokia (Alcatel-Lucent) SR OS has this feature activated by default. The following output provides checks from SR1 side:
|
A:SR1# oam lsp-ping “SR1_to_XR1” |
Let’s text XR1 side (remember, all LSP are unidirectional):
|
RP/0/0/CPU0:XR1#traceroute mpls traffic-eng tunnel-te 21 |
The same commands were used during LSP checks in LDP.
Constrains: explicit path link coloring, priority, bandwidth reservation and hops
One of the biggest advance of RSVP-TE over LDP is that RSVP-TE provides you very rich toolset for traffic engineering.
I hope, you know how RSVP constrains work in general. If no, please, refer to RFC or any networking courses that explains it. I can recommend MPLS course from Nokia (Alcatel-Lucent) that I’ve taken myself.
Before going to the configuration part, let’s review our topology from MPLS constrains point of view:
In this topology we’ll configure three tunnels with the following parameters:
- Tunnel with priority 4/4, requested bandwidth 123 Mbps, 2 hops maximum that don’t pass “green” links.
- Tunnel with priority 2/2, required bandwidth 222 Mbps that pass only “red” links.
- Tunnel with default priority, bandwidth 444 that must explicitly pass the link with IP addresses 10.0.0.0 or 10.0.0.1
Before configuring these LSPs we remove the previous configured LSP and bring back to operation all direct links between SR1 and XR1.
| Nokia (Alcatel-Lucent) VSR (SR 7750) | Cisco IOS XR (ASR 9000) |
| SR1 | XR1 |
|
A:SR1>config>router# info |
RP/0/0/CPU0:XR1(config)#show conf |
| SR3 | |
|
A:SR3>config>router# info |
For the sake of consistency, I’ve configured all Cisco IOS XR LSPs with meaningful names that correspond to the names at SR1. In general the configuration is self-explanatory and if know how one vendor works, you will understand how other vendor works as well. Some key point are here:
- In Nokia (Alcatel-Lucent) SR OS you configure path attributes inside of certain path in LSP. In Cisco IOS XR you configure attributes mainly for TE-tunnel in general (though signaled BW can be configured per path-option).
- In Nokia (Alcatel-Lucent) SR OS we configure “least-fill” option in order to make router to analyze actual utilization of links in TED and use the least-loaded possible path, otherwise the path is chosen randomly.
- In Cisco IOS XR you configure “soft-preemption” explicitly to allow the tunnels to be smoothly preempted by other tunnels with higher priority possibly without traffic drop.
Do you have ideas already, over which links the tunnels are built, how much BW is utilized and how much is available? Compare your ideas with outputs from our lab. Nokia (Alcatel-Lucent) SR OS router SR1:
|
A:SR1# show router rsvp session |
At Cisco IOS XR the same information is obtained by following command:
|
RP/0/0/CPU0:XR1#show rsvp session |
So two tunnels (tunnel 1 and tunnel 3) are established over link SR1 (toXR1_p1) — (gig0/0/0/0.10) XR1, due to configured constrains (exclude green links and explicit path). The tunnel 2 is established over SR3 due to constrain (include red links) as well. Due to different default hold priorities between Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR you see that bandwidth for the 3rd tunnel is reserved in different TE class depending on direction (hold priority 0 for SR1_to_XR1 tunnel and priority 7 for XR1_to_SR1 tunnel).
Let’s test if the tunnels work:
|
A:SR1# oam lsp-ping “SR1_to_XR1_1_NO_GREEN” |
There is one more constrain that I don’t review, which is called SLRG. It isn’t supported in Cisco IOS XR at sub interfaces, so I can’t show you its operation.
LSP Protection (secondary LSP)
The next point in our MPLS/RSVP-TE is protection. First of let’s we’ll configure so called “global protection”, what implies the configuration of the secondary path inside primary tunnel. There are two options: hot standby and cold standby. Hot standby means that the secondary tunnel is presignaled and the traffic will be immediately switched to it after detection of the failure on the first one. Cold standby means that the secondary tunnel starts to being signaled only after the failure at the primary is detected. Though the first option is seems to be more preferable, it consumes more resources, whereas the second option is sufficient if fast reroute is implemented (see the next point).
Both vendors support both option, but Cisco IOS XRv doesn’t support hot standby in VMWare image. So we’ll configure the secondary path as hot standby in Nokia (Alcatel-Lucent) SR OS and as cold standby in Cisco IOS XR for tunnel 2. We’ll allow only green links to be used as constrain for the path:
| Nokia (Alcatel-Lucent) VSR (SR 7750) | Cisco IOS XR (ASR 9000) |
| SR1 | XR1 |
|
A:SR1>config>router# info |
RP/0/0/CPU0:XR1(config)#show conf |
The keyword fur path to be standby is “standby” in Nokia (Alcatel-Lucent) SR OS. The same keyword in Cisco IOS XR is “path-protections”, but as I’ve already, it isn’t supported in XRv.
To check the status of the secondary path in Nokia (Alcatel-Lucent) SR OS you should use the following command:
|
A:SR1# show router mpls lsp “SR1_to_XR1_2_RED” path |
The standby tunnel has the same BW requirement as a primary one. As it’s already signaled, this BW must is reserved as well:
|
A:SR1# show router rsvp interface |
Now let’s check the secondary tunnel at Cisco IOS XR side:
|
RP/0/0/CPU0:XR1#show mpls traffic-eng tunnels name tunnel-te2 |
I’ve omitted the output a little bit in order to save the space. The most relevant information regarding protection is marked with bold. This tunnel isn’t presignaled, so there is no BW reservation for secondary path:
|
RP/0/0/CPU0:XR1#show rsvp interface |
Let’s check the operation of the protection. First of all we make trace of the path:
|
RP/0/0/CPU0:XR1#traceroute mpls traffic-eng tunnel-te 2 |
We see that traffic path SR3. Now we break link SR3-XR1 and wait a little bit for convergence (usually the convergence is really high, just a couple of hundreds of milliseconds, but as my lab is fully virtual, I have to disable interfaces both at SR3 and XR3 in order avoid waiting 30 seconds of RSVP PATH/RESV refresh).
In logs at Cisco IOS XR we see that the primary path for tunnel 2 is down, new is calculated and signaled:
|
RP/0/0/CPU0:Sep 29 12:48:56.523 : ifmgr[227]: %PKT_INFRA-LINK-5-CHANGED : Interface GigabitEthernet0/0/0/0.23, changed state to Administratively Down |
You can see that BW is reserved at another interface now. On the other hand Nokia (Alcatel-Lucent) SR OS shows only the message about down primary path and that’s it. The secondary route is hot standby and is already presignalled:
|
17 2016/10/12 15:35:32.29 UTC WARNING: MPLS #2012 Base VR 1: |
Let’s check lsp again:
|
RP/0/0/CPU0:XR1#traceroute mpls traffic-eng tunnel-te 2 |
Our traffic passes the backup LSP now without any manual intervention in the configuration, so our protection works. When we enable the link and wait for reconvergence, the traffic will be switched back to primary path. By default Nokia (Alcatel-Lucent) SR OS retries to reestablish the primary tunnel every 30 seconds (it’s called retry timer). In Cisco you have to do it manually by using command “mpls traffic-eng reoptimize” or you can configure optimize timer to any number of seconds/minutes. On the other hand reoptimization in Cisco IOS XR means not only attempt to reestablish primary path back, but it’s rather recalculation of CSPF for the path. This implies that if there is a new better path in the network, it will be used rather than previously used before the failure. In Nokia (Alcatel-Lucent) such process is called resignalling and it’s disabled by default. You can configure timer with the minimum value of 30 minutes.
So to make things clear: reoptimization of MPLS-TE tunnel in Cisco equals resignal of MPLS-TE tunnel in Nokia. Cisco seems not to have the analog for retry function of Nokia. Probably if the path protection was enabled, there will be another result, but I can’t test it now.
Fast reroute
Another method of MPLS LSP protection is called fast reroute. The theory is quite tough, but the configuration is really easy. Both in Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR you should just add one option to MPLS LSP configuration and that’s it. But in case of Cisco you also have to configure the MPLS LSPs that will be used for fast reroute. You can do it either manually (that is really time consuming) or you can use auto-tunnels. We’ll use the last option in our lab. There are also two type of fast-reroute protection: one-to-one (detour tunnel)) and facility (bypass tunnel). Cisco IOS XR supports only facility fast-reroute, so we’ll configure it both for tunnels 1 and 3:
| Nokia (Alcatel-Lucent) VSR (SR 7750) | Cisco IOS XR (ASR 9000) |
| SR1 | XR1 |
|
A:SR1>config>router# info |
RP/0/0/CPU0:XR1(config)#show conf |
Pay attention to “affinity ignore at Cisco. Default affinity at all Cisco RSVP-TE LSPs is 0x0/0xFFFF, meaning that tunnel is built only over link without any affinity bit set. This behavior must be changed.
At Nokia (Alcatel-Lucent) SR OS router SR1 both LSPs, which we want to protect, passes link with prefix 10.0.0.0/31. The fast-reroute analyze the topology and finds another suitable path (by default it tries to build node-protection, if it isn’t possible then it builds link-protection). Here is the result:
|
A:SR1# show router mpls bypass-tunnel protected-lsp |
Cisco side:
|
RP/0/0/CPU0:XR1#show mpls traffic-eng tunnels brief |
Under normal conditions tunnel 3 can’t use another link and tunnel 1 can use “red” link. Nevertheless if we break link 10.0.0.0/31, both of our tunnels continue working:
|
23 2016/10/12 16:38:35.03 UTC WARNING: MPLS #2013 Base VR 1: |
RSVP performance optimization
There are two differences, how Nokia (Alcatel-Lucent) SR OS and Cisco IOS XR implements RSVP:
- Nokia (Alcatel-Lucent) SR OS implements RSVP Hello in order to speed the convergence in case of link failure. Cisco IOS XR doesn’t use RSVP Hello (Cisco IOS / IOS XE uses);
- Cisco IOS XR by default reduces the number of RSVP PATH/RESV messages by aggregating them in SREFRESH (summary refresh) message. In Nokia (Alcatel-Lucent) SR OS you have to configure this feature explicitly;
So, let’s configure refresh-reduction at both our SR routers:
| Nokia (Alcatel-Lucent) VSR (SR 7750) – SR1 and SR3 |
|
A:SR1>config>router# info |
After applying the config you can check, that this technology work looking at RSVP interfaces:
|
A:SR1# show router rsvp interface detail | match expression “Interface|Ref” |
In order to see the real difference, I advise you to make packet capture at the interface level in the lab.
Lessons learned
There are some problems due to the fact that Cisco IOS XR is virtual device. So unfortunately (at least now), I was not able to test all features that I want. Take care of these things and search in Internet information about possible limitations.
Conclusion
MPLS/RSVP-TE was the best choice for communication service providers (CSP) networks and continues to hold its positions all over the world. The new rising technology that can replace it is called Segment Routing (SR) and we’ll get to know it in the next article. Nevertheless RSVP-TE is really strong and I’m absolutely sure that it will be used further in CSP networks for a long years. I hope you enjoy our journey today. Take care and good bye!
Support us
BR,
Anton Karneliuk
Hi Anton,
Appreciated for your every great article. Some minor questions
1.Does NOKIA support RSVP-TE by default, instead of LDP?
2.Does MPLS TE= RSVP-TE now? may be SR-TE future?
Thanks a lot.
Hi Zhoiu,
1) Nokia supports RSVP-TE, but the there is no meaning of default. You just need to configure that LSP is RSVP-TE based
2) Check another articles, Nokia supports SR-TE already. Here is ordinary SR: https://goo.gl/z3K1Ue. And here is SR-TE: https://goo.gl/Km17FQ
BR,
Anton
Hi, thank you for the great write-up.
lots of things to learn and try out.
I just had 1 questions – do we still have to enable “ldp” when using rsvp? (sdp with lsp bind).
Hi John,
LDP is also used for signaling of L2 services (like VPWS or VPLS) in form of T-LDP. So you need to have it if you don’t use BGP for SDP signaling.
BR,
Anton
hi Anton.
How do you route through the LSP nokia tunnels, can you point a static route to that tunnel or like autoroute in cisco
Hello Gabriel,
it depends on which traffic do you want to route. If that is any service traffic, then within the configuration of the service you have something called “auto-bind”, which dictates which LSP to use. You could choose the SR-TE or RSVP.
If you are willing to map to a specific LSP, then you need to use the concept of the admin-tags and tag policies, which would map the routes towards the LSPs.
Hope that helps.
Cheers,
Anton