Hello my friend,
We are continuing studying the Python library pySROS, which Nokia recently published. With all our passion for the Model-Driven Automation, we know that still a lot of people use the CLI daily. As such, today we’ll take a look on how we can automate execution and processing of the output of CLI commands in Nokia SR OS devices.
2
3
4
5
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
I See 10 Years Old Kids Creating Tools in Python… Is That Late to Start?
It is not. In fact, it shows that the there are no barriers to start learning and using Python. People come to a programming and software development with different backgrounds and for different purposes. However, all of them are united by a single goal: how to do something more efficient. This something is in fact can be anything: starting from a simple data analysis to a complex web application to games and, of course, to a network automation. At our trainings we use Python a lot; however, we know that network engineers may have no background in software development, and, therefore, we teach you basics of Python syntax, semantic and architecture, so that you can use it for network automaton (and other purposes) with confidence in your company’s environment.
We offer the following training programs:
- Zero-to-Hero Network Automation Training
- Automation with Nornir (2nd step after advanced network automation)
- Automation Orchestration wth AWX
During these trainings you will learn the following topics:
- Success and failure strategies to build the automation tools.
- Principles of software developments and tools for that.
- Data encoding (free-text, XML, JSON, YAML, Protobuf)
- Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
- Full configuration templating with Jinja2 based on the source of truth (NetBox).
- Best programming languages (Python, Bash) for developing automation, configuration management tools (Ansible) and automation frameworks (Nornir).
- Network automation infrastructure (Linux, Linux networking, KVM, Docker).
- Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.
Training from Karneliuk.com is very good
Maxim Parfenchyk @ Head of Data Network Planning, A1 BLR
Moreover, we put all mentions technologies in the context of the real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data centre networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depts and have discussions about your own projects. And on top of that, each technology is provided with online demos and you are doing the lab afterwards to master your skills. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.
Brief Description
Why do we need to run MD-CLI commands? Why shall we use pySROS for that at all? There are other, well established tools for SSH/CLI network automation, such as paramiko with a wrapper netmiko, scrapli, and some others. Those are the questions, you may have in your head right now. All of them are absolutely valid. Let’s try to answer them.
The main purpose of any automation is to improve operational efficiency via:
- doing things quick, much quicker than engineers do manually
- doing the things right, without a mistakes and/or types, which engineers make
As such, whether the automation is Model-Driven or Text-Driven (i.e., CLI-based), it doesn’t really matter as long as the requested use case is solved. In a longer run, for sure, Model-Driven Automation is a single correct approach. However, as network engineers use CLI daily to conduct operational activities, it is okay to start automation journey with an automation of this CLI.
The question of a library to use is often a personal choice. Some people prefer to use netmiko due to its wide range of supported vendors. Others prefer to use paramiko natively to control the messaging avoiding netmiko wrapping. One of the biggest advantage of mentioned libraries is that they are generally vendor agnostic, and, therefore, allow you to automate not only Nokia SR OS based network devices.
At the same time, none of them can be executed directly on the network device (unless you pure Linux based operating system such Nokia SR Linux or Cumulus Linux and to a degree Arista EOS and Cisco Nexus, where you can install Python external libraries). For traditional network operating system, which you can find in Service Provider networks, such Nokia SR OS, Cisco IOS XR, and others, you can use only what is available on the device by default, as you cannot install other components. This is where pySROS for Nokia SR OS comes to a stage.
The pySROS library is installed on Nokia SR OS network device, and if your intention is to create a script to manage such a device and have a possibility to execute script both remotely (from your Laptop) and locally on the network device, pySROS is a viable option for you.
On top of that, the execution of the commands itself, especially operational commands) often doesn’t lead to some result unless a network engineer visually analyse and interpret the information. Therefore, some post processing of the information is needed in order to get some useful result.
Let’s take a look how all these things could be implemented together.
Lab Setup
We continue using the lab setup we built in our first blog in this series:
- Servers:
- Linux Debian 11
- Python 3.9
- pySROS 21.10 (pay attention that version of pySROS is changing following the Nokia SR OS upgrade itself)
- Network Device:
- Nokia SR OS 21.10.R1
In previous blogposts (part 1, part 2, and part 3) you can learn more the setup and how to build that.
Scenario Description
Following the conversation started in the Brief Description part, we focus today on a few topics:
- How to execute the MD-CLI command in Nokia SR OS locally on Nokia network device or remotely?
- How to process and parse the output using regular expressions locally on the device and remotely?
- What is the difference in regular expressions in Python (remote execution) and MicroPython (local execution)
Join our Network Automation Training to learn Python to automate your network with NETCONF, RESTCONF, GNMI and, even CLI!
Besides that, we’ll review a few useful scenarios, how to deal with exceptions, which may arise when you use pysros.
Python Script Development
As said above, the feature of the CLI execution via pysros was introduced in the latest to the date Nokia SR OS version 21.10.R1. The versioning of pysros library follows the versioning of SR OS; therefore, we need to make sure we have the corresponding version, before we start creating a Python script.
Step #0. Upgrade pySROS for Remote Execution
It is worth to mention that upgrade is needed only for remote host (i.e., when you run Python script from your laptop). The pySROS is already upgraded towards the corresponding version in new Nokia SR OS 21.10.R1 and there is no upgrade needed.
The upgrade of pysros is following the same pattern as any Python library:
2
3
4
5
6
7
!
! OUTPUT IS TRUNCATED FOR BREVITY
!
Uninstalling pysros-21.7.2:
Successfully uninstalled pysros-21.7.2
Successfully installed pysros-21.10.2
Once this step is done, you can proceed with the development of the script itself.
Step #1. Create a Python Script for Remote Execution and Parsing of MD-CLI Commands
In the version of pysros, Nokia has introduced a new method cli(), which take a single argument, that is a CLI command you are willing to execute on the device. As pySROS generally requires the network device to operate in the model-driven mode, it, therefore, executes MD-CLI commands.
It is interesting that in the documentation to pySROS Nokia specifically mentions that it is possible to provide output redirections (e.g., “no-more”) as well.
Let’s take as a basis one of the previous scripts and amend it to use this new method:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Modules
import datetime
import os
from dotenv import load_dotenv
from pysros.management import connect
import re
# Body
if __name__ == "__main__":
## Get connectivity details
load_dotenv()
host={
"ip_address": os.getenv("NOKIA_IP"),
"username": os.getenv("NOKIA_USER"),
"password": os.getenv("NOKIA_PASS")
}
## Timestamp: started
t1 = datetime.datetime.now()
## Interecting with the device
connect_obj = connect(host=host["ip_address"], username=host["username"],
password=host["password"])
results = connect_obj.cli("show router route-table | no-more")
connect_obj.disconnect()
## Timestamp: completed
t2 = datetime.datetime.now()
## Printing results
tl = os.get_terminal_size()
print(f"{'=' * tl.columns}\nCompleted in {t2 -t1}\n{'=' * tl.columns}\n")
## Printing collected data
print(results)
print(f"\n{'=' * tl.columns}\n")
Refer to the original blog for the step-by-step explanation.
Compared to the original script, we have replaced the NETCONF get-config operation connect_obj.running.get() with an MD-CLI connect_obj.cli(), the rest of the script stays untouched. As you see, in the argument we specify command “show router route-table | no-more” (as we don’t know, how long the output could be, we add the no-more output redirect).
Let’s execute the script and see the results:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
==========================================================================================================
Completed in 0:00:05.048169
==========================================================================================================
===============================================================================
Route Table (Router: Base)
===============================================================================
Dest Prefix[Flags] Type Proto Age Pref
Next Hop[Interface Name] Metric
-------------------------------------------------------------------------------
0.0.0.0/0 Remote Static 01h11m36s 5
10.0.0.1 1
10.0.0.0/31 Local Local 14h14m16s 0
oc_1/1/c2/1_0 0
10.0.0.2/31 Local Local 14h14m16s 0
oc_1/1/c1/1_0 0
10.0.255.11/32 Local Local 14h14m16s 0
system 0
-------------------------------------------------------------------------------
No. of Routes: 4
Flags: n = Number of times nexthop is repeated
B = BGP backup route available
L = LFA nexthop available
S = Sticky ECMP requested
===============================================================================
==========================================================================================================
Pay attention to the completion time. It is over 5 seconds, which is very long. You will see it though much different, when you execute the script directly on the network device.
So, we have duped the routing table… So what? You can think that you can run the corresponding command in CLI SSH and get the result. You are not wrong; however, let us give you a bit broader context.
Whenever you are collecting output of certain CLI commands, you are typically looking or something specific (e.g., presence of a specific route in a routing table, etc) with a purpose of answering the question: is the state of network device inline with expectations or not. As such, we can split the task in two actions:
- Collect a raw data from a network device.
- Process the collected data.
What does “process the collected data” mean? In a simplest form it means that you shall find the key data and extract it (parsing) and then make a statement based on that data (analysis). In a standard Python distribution we have a library named re, which stands for regular expression. This is the easiest, yet very powerful, tool to find a subtract the data, you may be interested in.
Let’s say, we don’t want to see the output of the routing table, but rather we are willing to know if:
- The default route is present or not.
- How many routes do we have in total.
Let’s do the corresponding amendments to our script:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
#!/usr/bin/env python
# Modules
import datetime
import os
from dotenv import load_dotenv
from pysros.management import connect
import re
# Body
if __name__ == "__main__":
## Get connectivity details
load_dotenv()
host={
"ip_address": os.getenv("NOKIA_IP"),
"username": os.getenv("NOKIA_USER"),
"password": os.getenv("NOKIA_PASS")
}
## Timestamp: started
t1 = datetime.datetime.now()
## Interecting with the device
connect_obj = connect(host=host["ip_address"], username=host["username"],
password=host["password"])
results = connect_obj.cli("show router route-table | no-more")
connect_obj.disconnect()
## Timestamp: completed
t2 = datetime.datetime.now()
## Printing results
tl = os.get_terminal_size()
print(f"{'=' * tl.columns}\nCompleted in {t2 -t1}\n{'=' * tl.columns}\n")
## Processing Data
is_default_route_bool = True if re.search(r'\b0\.0\.0\.0/0\b', results) else False
num_routes_int = 0
for line_str in results.splitlines():
if re.match(r'^No\. of Routes:', line_str):
num_routes_int = int(re.sub(r'^No\. of Routes: (\d)$', r'\1', line_str))
print(f"Routes in routing table: {num_routes_int}\nDoes default route exist: {is_default_route_bool}")
print(f"\n{'=' * tl.columns}\n")
What have we done so far?
- Added a new module re.
- Removed print(results) function.
- Created a new variable is_default_route_bool of a Boolean type, which obtains True value, if the the default route is found using the search() function from the re module. Otherwise, it is False.
- Using splitlines() function the multiline string with the collected output of MD-CLI command is converted into a list of strings, which then validated for a corresponding pattern by the match() function from the re module. If there is a match, then the sub() function from the same module extracts the value with the number of the routes.
- Finally, we print() the analysed information to given answers to mentioned questions.
A bit jumping the gun, in an “ordinary” Python the 4th step could be replaced by findall() function. However, it is not implemented in MicroPython. Therefore, we have to take a longer road to achieve portability.
Let’s see how this script would work:
2
3
4
5
6
7
8
9
==========================================================================================================
Completed in 0:00:05.270960
==========================================================================================================
Routes in routing table: 4
Does default route exist: True
==========================================================================================================
As you see, the data is successfully parsed, extracted and put in the context. Now you may start thinking, which perspectives are opening in front of you: you can collect some data, parse and extract the key values, which use an input for the next request, etc. It is powerful, isn’t it?
Step #2. Create a Python Script for Execution and Parsing of MD-CLI Commands Locally in Nokia SR OS
It is even more powerful, as you can run it directly on the Nokia SR OS router. However, there is a slight modification needed for the regular expressions.
As discussed in the previous blogpost, Nokia SR OS has a MicroPython distribution, which generally has truncated functionality to be able to run directly on various controllers. The goos news is that the re library is part of MicroPython. The bad news is that besides absence of a number of functions, not all expressions are available either. For example, the expression “\b“, which specifies the beginning or end of the word is not implemented. Therefore, we need to amend the template.
Per the Release Notes for Nokia SR OS 21.10.R1, datetime library is not implemented either. However, we already the last time used the time module, so it is an easy replacement for us.
So, the script to run on Nokia SR OS device would look like as follows:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
A:admin@sr1# file
[/file "cf3:"]
A:admin@sr1# show python_scripts/pysros_mdcli_get_routes_sros.py
File: pysros_mdcli_get_routes_sros.py
-------------------------------------------------------------------------------
# Modules
import time
from pysros.management import connect
import re
# Body
if __name__ == "__main__":
## Get the timesamp in the beginning of the command
t1 = time.ticks_ms()
## Collect the information
connect_obj = connect()
results = connect_obj.cli("show router route-table | no-more")
connect_obj.disconnect()
## Processing results
is_default_route_bool = True if re.search(r'0\.0\.0\.0/0', results) else False
num_routes_int = 0
for line_str in results.splitlines():
if re.match(r'^No\. of Routes:', line_str):
num_routes_int = int(re.sub(r'^No\. of Routes: (\d)$', r'\1', line_str))
print("Routes in routing table: {}\nDoes default route exist: {}".format(num_routes_int, is_default_route_bool))
## Get the timesamp at the end
t2 = time.ticks_ms()
## Print time
print("\nCompleted in {} ms".format(time.ticks_diff(t2, t1)))
===============================================================================
Read the previous blogpost to get more details about the reasons of the syntax change.
The major changes are:
- The module datetime is replaced by time.
- No connectivity details are needed.
- Regular expression “r’\b0\.0\.0\.0/0\b’” is replaced by “r’0\.0\.0\.0/0′“.
Let’s test the created script directly from Nokia SR OS based device:
2
3
4
5
Routes in routing table: 4
Does default route exist: True
Completed in 3 ms
As you see, now the duration of the script is just 3 milliseconds compared to the 5 seconds, when you run the script remotely.
The output of the script is the same as the one above, which proves that the choice of libraries and overall logic was right for portability.
Examples in GitHub
You can find this and other examples in our GitHub repository.
Lessons Learned
Working on this blogpost forced us thinking, that portability of Python applications between different platforms is not something, we should take for granted. In vast majority of the cases we create applications for Linux or MAC, where we can install any extra library at our choice, ignoring the libraries available by default. Using MicroPython is a good refresher to see what does Python has built-in by default and make use of it. Creativity always comes via constraints.
Build your own and your network’s stable future with our Network Automation Training.
Conclusion
In this blogpost we have covered the development of the Multiplatform Python script, which even can be executed by various Python interpreters. In a less abstract terms, in pySROS 21.10.R1 the new feature was introduced, which allows you to run arbitrary CLI commands from Python script. This functionality is a good building block for your custom workflows with Python, which may help during troubleshooting, analysis and investigations. Take care and good bye.
Support us
P.S.
If you have further questions or you need help with your networks, we are happy to assist you, just send us a message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk
