Hello my friend,
In this HS blog series we have covered so far the automated build of the network topology for hyper scale data centre using Microsoft Azure SONiC. Today Nokia has announced a new product for data centre, which is called SRLinux. In the next couple of articles we’ll review it from the architectural and automation standpoint.
2
3
4
5
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
Thanks
We want to thank Nokia team for providing us the details and assisting in creating these materials. It won’t be possible without your help, dear partners.
Network automation training – now as a self-paced course as well
Following your asks we open a new format for the network automation training – self-paced format:
- It doesn’t matter what your timezone is.
- It doesn’t matter how much hours weekly do you have to study.
- It doesn’t matter how solid is your current background in automation, scripting and software development.
Because you decide on your own when, how often and how quickly you can learn.
At this training we teach you all the necessary concepts such as YANG data modelling, working with JSON/YAML/XML data formats, Linux administration basics, programming in Bash/Ansible/Python for multiple network operation systems including Cisco IOS XR, Nokia SR OS, Arista EOS and Cumulus Linux. All the most useful things such as NETCONF/RESTCONF, REST API, OpenConfig and many others are there. Don’t miss the opportunity to improve your career.
Big and Small Data Centres
The world of modern data centres in web-scale companies and networks is a disaggregated world. It means that the hardware and software of the network devices come from the different manufacturers:
- The biggest companies produce their network operating systems. For example, Facebook has its own FBOSS or Microsoft created Azure SONiC. Then they make their network operating systems open-source, so that others can use them as well. In terms of the hardware they typically use either merchant suppliers, such as Mellanox, Broadcom, EdgeCore, Arista and others; however, they may create their own HW as well.
- The web-scale companies of a bit smaller size often use either the open-source network operating systems created by the hyper-scalers (what is not always an easy task) or the commercial SW, such as Cumulus Linux and Arista with disaggregated HW.
- The enterprises traditionally stays without disaggregation with a traditional data centre network equipment such as Cisco Nexus, Arista EOS and so on.
Nokia in Data Centres
Originally Nokia had data centre networking gear coming from Nuage Networks, such as WBX 210. In the past couple of years, Nokia introduced the new range of the data centre switches/routers called 7250 IXR. However, none of these devices were disaggregated and run only native software (like. traditional Nokia SR OS on 7250 IXR). And now, finally, with the introduction of the SRLinux Nokia comes into the club of the hyper scale networks with the possibility to run their software on the white box switches.
Anatomy of Nokia SRLinux
From the name, SRLinux, it is getting obvious that Nokia decided to convert its SR OS into a set of the application based on the Linux; however, it is not just containerised version of the SR OS. It is full-featured network operating systems, which has its own internal architecture and management interfaces.
From the presentation Nokia delivered on the 9th July 2020, it was announced that the Nokia SRLinux can work on the following platforms:
- Nokia 7250 IXR Data centre routers
- Nokia 7220 Interconnect routers
As such, the functionality of the SRLinux will be to a certain degree shaped by the capabilities of the chipset.
First launch
The SRLinux has a demo image, which is provided as a Docker container. As such, it is possible to create a lab with several Nokia SRLinus containerised network functions inside the Linux host to do some tests. From the deployment perspective, it is done in a way similar to the Microsoft Azure SONiC:
- The containers are deployed without networking part for the date plane interfaces.
- The containers are interconnected to each other using
some black magicveth interfaces between the namespaces inside Linux hosting the containers with Nokia SRLinux and other operating systems. - The containers have local storage with the configuration files, so that we have the configuration persistent across the reboot of the containers.
There is one huge difference to the Microsoft Azure SONiC, though: the SRLinux requires the license. So you need to get in touch with Nokia representatives to obtain it.
In the next article we will explain in a greater details how to build a lab with the Nokia SRLinux.
Managing SRLinux with model-driven CLI
Once the SRLinux is booted in whatever mode (like a Docker container or on a real device), you are getting to the CLI. If you have worked previously with the model-drive CLI in SR OS, you will have similar experience.
The commands are different, however. The reason for that is the different YANG modules used in SRLinux and SR OS.
To put things in the context, let’s configure the following simple leaf/spine topology, which very popular in the data centres:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
| |
| lo: 10.0.255.1/32 |
| fc00:10:0:255::1/128 |
| +-+-+ |
| | |
| 10.0.0.0/31 +-----+-----+ |
| fc00:10::/127 .0/:0 | | .2/:2 10.0.0.2/31 |
| +-----------+ spine11 +---------+ fc00:10::2/127 |
| | e1-1 | AS:65001 | e1-2 | |
| | +-----------+ | |
| .1/:1 | e1-1 .3/:3 | e1-1 |
| +----+-----+ +----+-----+ |
| | | | | |
| | leaf11 | | leaf12 | |
| | AS:65011 | | AS:65012 | |
| +----+-----+ +----+-----+ |
| | | |
| +-+-+ +-+-+ |
| lo: 10.0.255.11/32 lo: 10.0.255.12/32 |
| fc00:10:0:255::11/128 fc00:10:0:255::12/32 |
| |
| |
| (c) 2020, karneliuk.com // Sample Nokia SRLinux lab |
| |
+-------------------------------------------------------------------------+
As the CLI is model-based, we need to find the proper context. The SRLinux data model (YANG, actually) is very similar to the OpenConfig. If you are familiar with OpenConfig syntax, you will find many things in the names and structures of the elements similar. This similarity allows us to perform the configuration of the interfaces:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
A:leaf11# info
interface ethernet-1/1 {
description to-spine11
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.0.1/31 {
}
}
ipv6 {
address fc00:10::1/127 {
}
}
}
}
interface lo0 {
description system
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.255.11/32 {
}
}
ipv6 {
address fc00:10:0:255::11/128 {
}
}
}
}
network-instance default {
admin-state enable
ip-forwarding {
receive-ipv4-check true
receive-ipv6-check true
}
interface ethernet-1/1.0 {
}
interface lo0.0 {
}
}
--{ * candidate exclusive }--[ ]--
A:leaf11# commit now save
Split for the interfaces and network-instance is something very OpenConfig-ish.
Same configuration we do on the leaf12:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
A:leaf12# info
interface ethernet-1/1 {
description to-spine11
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.0.3/31 {
}
}
ipv6 {
address fc00:10::3/127 {
}
}
}
}
interface lo0 {
description system
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.255.12/32 {
}
}
ipv6 {
address fc00:10:0:255::12/128 {
}
}
}
}
network-instance default {
admin-state enable
ip-forwarding {
receive-ipv4-check true
receive-ipv6-check true
}
interface ethernet-1/1.0 {
}
interface lo0.0 {
}
}
--{ * candidate exclusive }--[ ]--
A:leaf12# commit now
All changes have been committed. Leaving candidate mode.
--{ * running }--[ ]--
And spine11:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
A:spine11# info
interface ethernet-1/1 {
description to-leaf11
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.0.0/31 {
}
}
ipv6 {
address fc00:10::/127 {
}
}
}
}
interface ethernet-1/2 {
description to-leaf12
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.0.2/31 {
}
}
ipv6 {
address fc00:10::2/127 {
}
}
}
}
interface lo0 {
description system
admin-state enable
subinterface 0 {
ipv4 {
address 10.0.255.1/32 {
}
}
ipv6 {
address fc00:10:0:255::1/128 {
}
}
}
}
network-instance default {
admin-state enable
ip-forwarding {
receive-ipv4-check true
receive-ipv6-check true
}
interface ethernet-1/1.0 {
}
interface lo0.0 {
}
}
--{ * candidate exclusive }--[ ]--
However, don’t be fooled by the similarity: Nokia SRLinux modules are different.
Once the configuration of the interfaces is complete, we can verify the connectivity using the ping between the network functions:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
Using network instance default
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=4.57 ms
--- 10.0.0.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 4.570/4.570/4.570/0.000 ms
--{ running }--[ network-instance default ]--
A:spine11# ping 10.0.0.3 network-instance default -c 1
Using network instance default
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
64 bytes from 10.0.0.3: icmp_seq=1 ttl=64 time=16.4 ms
--- 10.0.0.3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 16.400/16.400/16.400/0.000 ms
--{ running }--[ network-instance default ]--
A:spine11# ping6 fc00:10::1 network-instance default -c 1
Using network instance default
PING fc00:10::1(fc00:10::1) 56 data bytes
64 bytes from fc00:10::1: icmp_seq=1 ttl=64 time=2.17 ms
--- fc00:10::1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.171/2.171/2.171/0.000 ms
--{ running }--[ network-instance default ]--
A:spine11# ping6 fc00:10::3 network-instance default -c 1
Using network instance default
PING fc00:10::3(fc00:10::3) 56 data bytes
64 bytes from fc00:10::3: icmp_seq=1 ttl=64 time=5.91 ms
--- fc00:10::3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 5.911/5.911/5.911/0.000 ms
--{ running }--[ network-instance default ]--
The connectivity between our network elements works properly, what confirms the fact that the connectivity between the containers is configured properly.
The next step is external BGP to announce the loopbacks and we will do that relatively quickly without too much description (example of leaf11 is shown), which is also very much OpenConfig-ish:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
A:leaf11# info
network-instance default {
protocols {
bgp {
autonomous-system 65011
router-id 10.0.255.11
group PG_IPV4 {
admin-state enable
}
group PG_IPV6 {
admin-state enable
}
ipv4-unicast {
admin-state enable
}
ipv6-unicast {
admin-state enable
}
neighbor 10.0.0.0 {
admin-state enable
export-policy RP_IPV4_LO
import-policy RP_IPV4_LO
peer-as 65001
peer-group PG_IPV4
ipv4-unicast {
admin-state enable
}
send-community {
standard true
large true
}
}
neighbor fc00:10:: {
admin-state enable
export-policy RP_IPV6_LO
import-policy RP_IPV6_LO
peer-as 65001
peer-group PG_IPV6
ipv6-unicast {
admin-state enable
}
send-community {
standard true
large true
}
}
route-advertisement {
rapid-withdrawal true
}
}
}
}
routing-policy {
prefix-set PS_LO_IPV4 {
prefix 10.0.255.0/24 mask-length-range 32..32 {
}
}
prefix-set PS_LO_IPv6 {
prefix fc00:10:0:255::/112 mask-length-range 128..128 {
}
}
policy RP_IPV4_LO {
default-action {
reject {
}
}
statement 10 {
match {
prefix-set PS_LO_IPV4
}
action {
accept {
}
}
}
}
policy RP_IPV6_LO {
default-action {
reject {
}
}
statement 10 {
match {
prefix-set PS_LO_IPv6
}
action {
accept {
}
}
}
}
}
Read the Data Centre series to learn about BGP DC fabrics.
Once the configuration is committed, you can verify, what are the status of the protocol:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BGP is enabled and up in network-instance "default"
Global AS number : 65001
BGP identifier : 10.0.255.1
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total paths : 11
Received routes : 4
Received and active routes: 4
Total UP peers : 4
Configured peers : 4, 0 are disabled
Dynamic peers : None
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Default preferences
BGP Local Preference attribute: 100
EBGP route-table preference : 170
IBGP route-table preference : 170
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Wait for FIB install to advertise: True
Send rapid withdrawals : True
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Ipv4-unicast AFI/SAFI
Received routes : 2
Received and active routes : 2
Max number of multipaths : 1, 1
Multipath can transit multi AS: True
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Ipv6-unicast AFI/SAFI
Received routes : 2
Received and active routes : 2
Max number of multipaths : 1,1
Multipath can transit multi AS: True
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
And status of the BGP neighbours:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
BGP neighbor summary for network-instance "default"
Flags: S static, D dynamic, L discovered by LLDP, B BFD enabled, - disabled, * slow
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+--------------------+-----------------------------+--------------------+-------+-----------+----------------+----------------+--------------+-----------------------------+
| Net-Inst | Peer | Group | Flags | Peer-AS | State | Uptime | AFI/SAFI | [Rx/Active/Tx] |
+====================+=============================+====================+=======+===========+================+================+==============+=============================+
| default | 10.0.0.1 | PG_IPV4 | S | 65011 | established | 0d:0h:2m:56s | ipv4-unicast | [1/1/2] |
| | | | | | | | ipv6-unicast | [0/0/0] |
| default | 10.0.0.3 | PG_IPV4 | S | 65012 | established | 0d:0h:2m:56s | ipv4-unicast | [1/1/2] |
| | | | | | | | ipv6-unicast | [0/0/0] |
| default | fc00:10::1 | PG_IPV6 | S | 65011 | established | 0d:0h:2m:57s | ipv4-unicast | [0/0/0] |
| | | | | | | | ipv6-unicast | [1/1/2] |
| default | fc00:10::3 | PG_IPV6 | S | 65012 | established | 0d:0h:2m:57s | ipv4-unicast | [0/0/0] |
| | | | | | | | ipv6-unicast | [1/1/2] |
+--------------------+-----------------------------+--------------------+-------+-----------+----------------+----------------+--------------+-----------------------------+
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Summary:
4 configured neighbors, 4 configured sessions are established,0 disabled peers
0 dynamic peers
BGP builds the routing table for, so we can take a look there as well:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IPv4 Unicast route table of network instance default
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-------------------------------------+-------+------------+-----------------+---------+-------+----------------------------------------------------+-----------------------+
| Prefix | ID | Active | Owner | Metric | Pref | Next-hop (Type) | Next-hop Interface |
+=====================================+=======+============+=================+=========+=======+====================================================+=======================+
| 10.0.0.0/31 | 0 | true | local | 0 | 0 | 10.0.0.1 (direct) | ethernet-1/1.0 |
| 10.0.0.1/32 | 0 | true | host | 0 | 0 | None (extract) | None |
| 10.0.255.1/32 | 0 | true | bgp | 0 | 170 | 10.0.0.0 (indirect) | None |
| 10.0.255.11/32 | 0 | true | host | 0 | 0 | None (extract) | None |
| 10.0.255.12/32 | 0 | true | bgp | 0 | 170 | 10.0.0.0 (indirect) | None |
+-------------------------------------+-------+------------+-----------------+---------+-------+----------------------------------------------------+-----------------------+
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
5 IPv4 routes total
5 IPv4 prefixes with active routes
0 IPv4 prefixes with active ECMP routes
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
IPv6 Unicast route table of network instance default
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
+-------------------------------------+-------+------------+-----------------+---------+-------+----------------------------------------------------+-----------------------+
| Prefix | ID | Active | Owner | Metric | Pref | Next-hop (Type) | Next-hop Interface |
+=====================================+=======+============+=================+=========+=======+====================================================+=======================+
| fc00:10::/127 | 0 | true | local | 0 | 0 | fc00:10::1 (direct) | ethernet-1/1.0 |
| fc00:10::1/128 | 0 | true | host | 0 | 0 | None (extract) | None |
| fc00:10:0:255::1/128 | 0 | true | bgp | 0 | 170 | fc00:10:: (indirect) | None |
| fc00:10:0:255::11/128 | 0 | true | host | 0 | 0 | None (extract) | None |
| fc00:10:0:255::12/128 | 0 | true | bgp | 0 | 170 | fc00:10:: (indirect) | None |
+-------------------------------------+-------+------------+-----------------+---------+-------+----------------------------------------------------+-----------------------+
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
5 IPv6 routes total
5 IPv6 prefixes with active routes
0 IPv6 prefixes with active ECMP routes
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Now we can verify the reachability between the loopbacks of the leaf11 and leaf12 via spine11:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Using network instance default
PING 10.0.255.12 (10.0.255.12) from 10.0.255.11 : 56(84) bytes of data.
64 bytes from 10.0.255.12: icmp_seq=1 ttl=63 time=4.22 ms
--- 10.0.255.12 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 4.229/4.229/4.229/0.000 ms
A:leaf11# ping6 fc00:10:0:255::12 network-instance default -I fc00:10:0:255::11 -c 1
Using network instance default
PING fc00:10:0:255::12(fc00:10:0:255::12) from fc00:10:0:255::11 : 56 data bytes
64 bytes from fc00:10:0:255::12: icmp_seq=1 ttl=63 time=4.51 ms
--- fc00:10:0:255::12 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 4.515/4.515/4.515/0.000 ms
On a side note, the interesting thing about the CLI is Nokia allows to build a customised CLI based on the original Nokia SRLinux YANG module. During the announcement it was shown the possibility to create a custom aliases, which can selectively show only relevant interface. It is generally possible to create your own YANG module and bring them on the platform, but that requires way more efforts, as you need to have an access to the platform SDK to understand how to program the hardware.
Opportunities for automation
Out of the box SRLinux supports the gRPC/gNMI interface with Nokia SRLinux YANG modules. This allows to integrate it into the programmable network management framework. gRPC transport with gNMI specification is actively promoted by the Google and the community to manage the networks. As primary focus of the networking in Google is on the data centres, it is not an occasion that the gRPC/gNMI is implemented in the Nokia SRLinux.
More about gRPC/gNMI automation for Nokia and other vendors you can learn at our network automation training.
Lessons learned
Working with a container networking is always the fun, especially if we need to get something more complicated than a standard NAT’ed interface. Despite we have such issues earlier as well, we have to recall the networking in Linux namespaces. As said, if something broken, it is all the time network 🙂
Conclusion
The Nokia created a new interesting product to attack the high scale data centres and steps into a yet unknown environment, where others vendors find themselves for quite a while already. On the other hand, the capabilities (YANG-based CLI + gRPC/gNMI + possibility to adapt other YANG modules) created in the product makes its quite an interesting. In the upcoming articles we will cover the interoperability of this product with other data centre products from our portfolio as well as its automation with HS graph-based automation. Take care and good bye.
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk