Hello my friend,
As mentioned in the previous blogpost, we started talking about practical usage of Python and Go (Golang) for network and IT infrastructure automation. Today we’ll take a look how we can interact with any SSH-speaking device, whether it is a network device, server, or anything else.
You Put So Much Content For Free Online, Why To Join Trainings Then?
Our ultimate goal is to make you successful with software developing for IT infrastructure management. Out blogs are the first step so that you can get up to speed if you already well equipped with fundamentals as protocols, data formats, etc. We believe that sharing is caring, hence we share back our knowledge with you, so that your path could be a little bit easier and quicker, so that you have more time to focus on what matters. If that’s enough for you to move forward, that’s great.
At the same time, if you feel you need more, you want to have finely-curated labs, slack support and deep dive not just in coding, but really in fundamentals, our training programs are here for you:
We offer the following training programs in network automation for you:
- Zero-to-Hero Network Automation Training
- High-scale automation with Nornir
- Ansible Automation Orchestration with Ansble Tower / AWX
- Expert-level training: Closed-loop Automation and Next-generation Monitoring
During these trainings you will learn the following topics:
- Success and failure strategies to build the automation tools.
- Principles of software developments and the most useful and convenient tools.
- Data encoding (free-text, XML, JSON, YAML, Protobuf).
- Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
- Full configuration templating with Jinja2 based on the source of truth (NetBox).
- Best programming languages (Python, Bash) for developing automation
- The most rock-solid and functional tools for configuration management (Ansible) and Python-based automation frameworks (Nornir).
- Network automation infrastructure (Linux, Linux networking, KVM, Docker).
- Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.
- Collection network data via SNMP and streaming telemetry with Prometheus
- Building API gateways with Python leveraging Fast API
- Integration of alerting with Slack and your own APIs
- … and many more
Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data center networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depth and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.
Start your automation training today.
What Are We Going To Talk Today?
Nowadays, SSH is the most popular protocol to interact with network devices as well as compute nodes, such as servers, virtual machines (VMs) and to a degree even containers. System and network engineers use SSH daily to get data from devices for analysis, for health checking or to perform configuration change. Having such a wide spread, it is no wonder that we put it first to the queue of network management.
Today we are going to discuss:
- How to connect to network devices using SSH with Python and Go (Golang)?
- How to retrieve operational status?
Explanation
Interaction with network and IT infrastructure devices via SSH meaning opening the interactive terminal session remotely, which gives look and feel as if you are directly connected to the device console. This interactive session lasts for as long as you interact with the devices and involves exchange of data between you / your application and destination node, which raises two important questions to address:
- How to ensure that you is you when session is established? In other words, there shall be some authentication and authorization mechanisms in place.
- How to ensure that data exchange between your host and node, even if eavesdropped, aren’t altered and/or its content visible to the attacker? This requires some encryption techniques.
Both these issues were not addressed in telnet, which made it very unfavorable for network and IT infrastructure management, although this is typically the first protocol engineers learn to use. And both these issues are addressed in SSH.
Once the SSH session is established between the destination node and your application, free form text is sent back and force. Typically, you send some command and receive some response to them. Albeit seems extremely simple, the important part of this process is to have mechanism, which will detect, when you have received the output in full so that you can send new instruction. Some SSH client have this functionality built-in, whilst others would require you do it yourself.
Join Zero-to-Hero Network Automation Training to learn the details of SSH, which are important for building scalable network and IT automation solutions.
Examples
In this blog we’ll show the basics of SSH interaction. Basics mean that we will actually execute one off commands and collect their output, whilst in the next blog post we will show advanced interaction with network and IT infrastructure devices using SSH.
The today lab’s scenario:
- Connect to network device using SSH.
- Send command and receive output in full.
- Display the collected information in CLI (stdout).
Python
First comes Python as our series is called from Python to Go. Before we can create an appliation in Python, which will be connecting to remote devices using SSH, we need to install corresponding package. We’ll use today paramiko, which is by far the most popular and widely used SSH library. Some other great libraries, such as netmiko is based on paramiko. We’ll also install pyyaml to parse input YAML file:
Once libraries are available, we can develop our software with Python:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
# Modules
import argparse
import datetime
import os
import sys
import time
import re
from dataclasses import dataclass
from typing import List
import yaml
import paramiko
# Classes
@dataclass
class Credentials:
"""Class to store credentials."""
username: str
password: str
@dataclass
class Result:
"""Class to store command execution data."""
command: str
output: str
timestamp: datetime.datetime
class Device:
"""Class to interact with netowrk device."""
def __init__(self, hostname: str, ip_address: str, credentials: Credentials):
self.hostname = hostname
self.ip_address = ip_address
self.credentials = credentials
self.results: List[Result] = []
def execute_command(self, command: str) -> None:
"""Method to execute a command."""
# Create a new SSH client
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
# Connect to the device
client.connect(
self.ip_address,
username=self.credentials.username,
password=self.credentials.password,
look_for_keys=False,
allow_agent=False,
)
# Invoke the session
session = client.invoke_shell()
session.recv(65535)
# Execute the command
session.send(command + "\n")
output = ""
print(f"{regex=}")
while not regex.search(output):
time.sleep(.1)
output += session.recv(65535).decode("utf-8")
# Store the result
self.results.append(Result(command, output, datetime.datetime.now()))
# Close the connection
session.close()
# Functions
def read_args() -> argparse.Namespace:
"""Helper function to read CLI arguments."""
parser = argparse.ArgumentParser(description="User input.")
parser.add_argument("-i", "--inventory", type=str, help="Path to inventory file.")
return parser.parse_args()
def load_inventory(filename: str, credentials: Credentials) -> List[Device]:
"""Function to load inventory data."""
# Open file
try:
with open(filename, "r", encoding="utf-8") as f:
data = yaml.safe_load(f)
except FileNotFoundError as e:
print(e)
sys.exit(1)
# Populate list of devices
result = []
for device in data:
result.append(Device(credentials=credentials, **device))
return result
def get_credentials() -> Credentials:
"""Function to get credentials."""
username = os.getenv("AUTOMATION_USER")
password = os.getenv("AUTOMATION_PASS")
return Credentials(username, password)
# Main code
if __name__ == "__main__":
# Read CLI arguments
args = read_args()
# Get credentials
credentials = get_credentials()
# Load inventory
devices = load_inventory(args.inventory, credentials=credentials)
# Execute command
for device in devices:
device.execute_command("show version")
# Print results
for device in devices:
print(f"Device: {device.hostname}")
for result in device.results:
print(f"Command: {result.command}", f"Output: {result.output}", f"Timestamp: {result.timestamp}", sep="\n")
This code is long and it is assumes that you have read and practiced all our previous blog posts in “From Python to Go” series or have relevant knowledge already.
We’ll focus our explanation on “Device” class as all other things were already explained:
- Object-oriented programming and classes
- Reading environment variables
- Reading files
- Parsing YAML
- Processing CLI arguments
- Handling exceptions
Break down for this blog post:
- All the user inputs (credentials via environment variables and inventory via YAML file) is passed for class Device, which contains method “execute_command” for interacting with network devices.
- This method takes a single external argument, which is the command, which shall be executed on the remote device and returns the result of execution.
- Within this method, the new SSH client is created via instantiation of object of “SSHClient” class from paramiko library.
- Once the client is created, the policy policy is set to automatically add SSH keys for device, which allows to connect to devices for the first time, before its SSH key is known to your host “client.set_missing_host_key_policy(paramiko.AutoAddPolicy())“.
- Then the SSH session is established to the remote node (network device, server) using “connect()” method of “SSHClient” class. This method takes as arguments connectivity details, such as target node IP address or FQDN, port, credentials, etc.
- After the SSH session is established, you need to open interactive shell using “invoke_shell()” method, which allows you to send and receive information from the endpoint. The interactive shell is a new object, which you need to store.
- Finally, you can send and receive messages. Be mindful though that this is a raw SSH session and you need to handle sending and receiving on buffer level, using “send()” and “recv()” methods. For the latter you need to specify how many bytes you want to read from the buffer.
- The tricky part is to detect when the output of your command is received in full. In network devices and in servers this is typically signaled with prompt with hostname followed by some special character. So the following approach is taken:
- Regular expression is created, which is looking for hostname followed by “#” or “>” symbol.
- This regular expression is capable to read multiline strings.
- the output is read from buffer and is added to the received data. It is then checked against regular expression and if there is no match, the data read again. It is repeated unlimited amount of time until the pattern is matched.
- Then result is sent back to the caller.
Let’s test this application. First of all, here is the inventory:
2
3
4
---
- hostname: dev-pygnmi-eos-001
ip_address: 192.168.51.79
Set the credentials to environment:
2
$ export AUTOMATION_PASS="password"
And execute the Python application:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Device: dev-pygnmi-eos-001
Command: show version
Output: show version
show version
dev-pygnmi-eos-001>show version
vEOS
Hardware version:
Serial number:
Hardware MAC address: b239.c742.24ec
System MAC address: b239.c742.24ec
Software image version: 4.26.0.1F
Architecture: i686
Internal build version: 4.26.0.1F-21994874.42601F
Internal build ID: e41b7ab2-f5ed-45cb-ba9c-f320cb81332f
Uptime: 0 weeks, 6 days, 7 hours and 58 minutes
Total memory: 2006640 kB
Free memory: 1188532 kB
dev-pygnmi-eos-001>
Timestamp: 2025-02-22 17:31:21.631414
It works!
Go (Golang)
Same as with Python, we need to install extra package to be able to establish SSH connectivity, which is called “crypto/ssh” and it is also one of the foundational ones. Same as with Python, we install package yaml for parsing YAML files:
2
$ go get gopkg.in/yaml.v3
And here is code for our application written in Go (Golang) to communicate with network devices (switches, routers, firewalls, load balancers) as well as other IT infrastructure devices (servers, virtual machines, containers, etc) using SSH:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
package main
import (
"bytes"
"flag"
"fmt"
"log"
"os"
"time"
"golang.org/x/crypto/ssh"
"gopkg.in/yaml.v3"
)
// Imports
// Types and Receivers
type Arguments struct {
/* Class to starte CLI arguments */
Inventory string
}
type Crendetials struct {
/* Struct to store credentials. */
Username string
Password string
}
type Result struct {
/* Struct to store command execution result. */
Command string
Output string
Timestamp time.Time
}
type Device struct {
/* Struct to interact with netowrk device. */
Hostname string `yaml:"hostname"`
IpAddress string `yaml:"ip_address"`
Crendetials Crendetials
Result []Result
}
func (d *Device) executeCommand(c string) {
/* Method to execute command */
interactiveAuth := ssh.KeyboardInteractive(
func(user, instruction string, questions []string, echos []bool) ([]string, error) {
answers := make([]string, len(questions))
for i := range answers {
answers[i] = (*d).Crendetials.Password
}
return answers, nil
},
)
// Create a new SSH client
sshClientConfig := &ssh.ClientConfig{
User: (*d).Crendetials.Username,
Auth: []ssh.AuthMethod{interactiveAuth},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
}
sshClient, err := ssh.Dial("tcp", fmt.Sprintf("%v:22", (*d).IpAddress), sshClientConfig)
if err != nil {
log.Fatalln("Failed to dial: ", err)
}
defer sshClient.Close()
// Create session
session, err := sshClient.NewSession()
if err != nil {
log.Fatalln("Failed to open the session: ", err)
}
defer session.Close()
// Execute the command
buffer := bytes.Buffer{}
session.Stdout = &buffer
if err := session.Run(c); err != nil {
log.Fatalln("Failed to execute command: ", err)
}
// Update the result
(*d).Result = append((*d).Result, Result{
Command: c,
Output: buffer.String(),
Timestamp: time.Now(),
})
}
// Functions
func readArgs() Arguments {
/* Helper function to read CLI arguments */
result := Arguments{}
flag.StringVar(&result.Inventory, "i", "", "Path to the inventory file")
flag.Parse()
return result
}
func loadInventory(p string) *[]Device {
/* Function to load inventory data. */
// Open file
bs, err := os.ReadFile(p)
if err != nil {
fmt.Println("Get error ", err)
os.Exit(1)
}
// Load inventory
result := &[]Device{}
err = yaml.Unmarshal(bs, result)
if err != nil {
fmt.Println("Get error ", err)
os.Exit(1)
}
// Return result
return result
}
func getCredentials() Crendetials {
/* Function to get credentials. */
return Crendetials{
Username: os.Getenv("AUTOMATION_USER"),
Password: os.Getenv("AUTOMATION_PASS"),
}
}
// Main
func main() {
/* Core logic */
// Read CLI arguments
cliArgs := readArgs()
// Get credentials
sshCreds := getCredentials()
// Load inventory
inventory := loadInventory(cliArgs.Inventory)
// Execute commands
for i := 0; i < len(*inventory); i++ {
(*inventory)[i].Crendetials = sshCreds
(*inventory)[i].executeCommand("show version")
}
// Print results
for i := 0; i < len(*inventory); i++ {
for j := 0; j < len((*inventory)[i].Result); j++ {
fmt.Printf(
"Command: %v\nOutput: %v\nTimestamp: %v\n",
(*inventory)[i].Result[j].Command,
(*inventory)[i].Result[j].Output,
(*inventory)[i].Result[j].Timestamp,
)
}
}
}
As mentioned above, this code is long and it is assumes that you have read and practiced all our previous blog posts in “From Python to Go” series or have relevant knowledge already.
And also repeating here for your convenience, we’ll focus our explanation on “Device” struct as all other things were already explained and repeating them will take a lot of time:
- Object-oriented programming and structs
- Dealing with environment variables
- Reading files from disk
- Parsing YAML/JSON and XML
- Processing CLI arguments
- Handling errors
This is how we interact with network device using Go (Golang):
- Struct “Device” contains hostname, ip address and credentials to connect to the network device as well as field to store the result of collection. As we initially read it from YAML, some extra instructions such as “yaml:”xxx”” are needed.
- The receiver function “executeCommand()” takes one argument with command content. It is applied to the pointer towards struct device, as it requires changing the content of the original struct.
- First thing within this receiver function is to create the SSH authentication method. Go (Golang) is a low-level programming language, therefore it is a little bit more involved that Python, where you just specify credentials and that’s it. There are a lot of different authentication mechanisms available such as password, SSH keys, etc. From our experiment, to connect to network devices you need to use function “KeyboardInteractive()” from “crypto/ssh” package.
- After than you create struct with the configuration of SSH “ClientConfig“, which contains username, authentication object and HostKeyCallback. The latter is similar to SSH key policy in Python as it controls if you can or cannot connect the devices, which you don’t have SSH key on your system yet.
- Then you open SSH channel to the device using “Dial()” function from “crypto/ssh” package, to which arguments you pass IP address, port and your ssh ClientConfig. Don’t forget to auto-close session before exit using “defer” instruction.
- Similar to Python, your create interactive session using “NewSession()” receiver function of created SSH channel.
- To read data from session, you need first to create buffer. As you receive bytes from wire, you create buffer using “Buffer” struct from Go (Golang) built-in package “bytes“.
- You points the Stdout of your session to the memory address (pointer) of the created duffer, so that you can save it for processing.
- Finally, using “Run()” receiver function of created session, you sends the command to the device. the output is stored in the buffer defined above. Once the output is received, interactive SSH session is terminated, which is a big difference with Python.
In the previous part (Python) we’ve shown the inventory and set environment variables; as such, here we show only execution:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Command: show version
Output: vEOS
Hardware version:
Serial number:
Hardware MAC address: b239.c742.24ec
System MAC address: b239.c742.24ec
Software image version: 4.26.0.1F
Architecture: i686
Internal build version: 4.26.0.1F-21994874.42601F
Internal build ID: e41b7ab2-f5ed-45cb-ba9c-f320cb81332f
Uptime: 0 weeks, 6 days, 8 hours and 31 minutes
Total memory: 2006640 kB
Free memory: 1189920 kB
Timestamp: 2025-02-22 18:04:34.242902767 +0000 GMT m=+1.248006951
And it works too.
Lessons in GitHub
You can find the final working versions of the files from this blog at out GitHub page.
Conclusion
As you can see, it is possible to interact with network devices and IT infrastructure using both Python and Go (Golang) and it is not very complicated. However, there are nuances both in Python and in Go (Golang). The fact that “crypto/ssh” closes SSH session after sending command required us to look for a better SSH library and we found one. The good thing, the same library exists both for Python and Go (Golang) with syntax being identical as much as it could be. And in the next blog we will cover it. Take care and good bye.
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk