Hello my friend,
Today’s topic is critical to complete full picture of software development for network automation. Today’s topic is what allows you to conduct your tasks within meaningful time frame, especially when you have a lot of network devices, servers, virtual machines to manage. Today’s topic is concurrency of code execution in Python and Golang.
What Other Programming Languages Makes Sense To Study?
There are more than 100 programming languages out there. Some of them are quite universal and allow development of almost any kind of application. Others are more specific. Python is probably the most universal programming language, from what I’ve worked with or heard of. It can be used in infrastructure management at scale (e.g., OpenStack is written in Python), web applications, data science and many more. Golang is much more low-level compared to Python and, therefore, way more performant. Various benchmarks available online suggests that the same business tasks could be 3-30 times quicker in Golang compared to Python; therefore Golang is suitable for system programing (e.g, Kubernetes and Docker are created in Go). That’s what we cover in our blogs. Apart from them there are a lot of other: C/C++ if you need even lower level than Golang. Rust is also quite popular for its speed as it is also low-level. Java is popular for enterprise applications. JavaScript/TypeScript is suitable for front-end development. So if you are willing to grow outside of infrastructure and network management, select the area and we advise on languages then.
The truth is, each new language is easier to learn than the previews one. Therefore, enroll today to our network automation programs to get started:
We offer the following training programs in network automation for you:
- Zero-to-Hero Network Automation Training
- High-scale automation with Nornir
- Ansible Automation Orchestration with Ansble Tower / AWX
- Expert-level training: Closed-loop Automation and Next-generation Monitoring
During these trainings you will learn the following topics:
- Success and failure strategies to build the automation tools.
- Principles of software developments and the most useful and convenient tools.
- Data encoding (free-text, XML, JSON, YAML, Protobuf).
- Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
- Full configuration templating with Jinja2 based on the source of truth (NetBox).
- Best programming languages (Python, Bash) for developing automation
- The most rock-solid and functional tools for configuration management (Ansible) and Python-based automation frameworks (Nornir).
- Network automation infrastructure (Linux, Linux networking, KVM, Docker).
- Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.
- Collection network data via SNMP and streaming telemetry with Prometheus
- Building API gateways with Python leveraging Fast API
- Integration of alerting with Slack and your own APIs
- … and many more
Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data center networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depth and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.
Start your automation training today.
What Are We Going To Talk Today?
In modern business world, speed is critical to success. Implementation of desired changes to IT and network infrastructure must be quick, correct, and controlled. We talked about how to make interaction controlled and correct. Today we talk about speed of application.
Speed can be achieved on different levels, using different techniques. Today we focus specifically on concurrency and parallelism:
- What are the different approaches exist to execute multiple parts of code simultaneously?
- What is available in Python? What is available in Golang?
- How to develop software leveraging concurrency?
Explanation
Why Concurrency Is Needed?
One of the distinct aspect of network and infrastructure management is scale. It is not uncommon to manage simultaneously hundreds and thousands of devices. At such scale, the software architecture and selected patterns start playing a critical role. Think about a simple scenario: duration of interaction with network device to perform the required pre-/post- check as well as relevant configuration is 30 seconds. And you need to configure 1000 network devices. In the most straightforward case, you will interact with devices one by one, which results in 30000 seconds, or about 8 hours 20 minutes. Way too long.
The idea of concurrency is to execute some parts of code simultaneously or near simultaneously, which is heavily dependent on the nature of your tasks.
Join our Zero-to-Hero Network Automation Training to dive into details of concurrency.
Concurrency 101
To understand how concurrency is implemented in Python and in Golang (Go), you first need to understand how the computer executes your code:
- Your computer has some amount of CPUs and cores per CPU. Could be anything from 1x core CPU (probably not anymore, but it was the case 25 years ago) to 4x/12x core modern laptops to 2x/4x CPU servers with 24x cores each.
- When you start your application, written in any programming language, including those written in Golang/Python, which we created in previous blogs, it is started on one core of one CPU.
- Application can be actively doing things (e.g., computing anything with local data) or being idle (e.g., waiting on response from remote servers).
Based on the explanation above, we come to two types of concurrency existing in software development:
- Concurrency for IO-bound tasks, that is a collection of techniques to utilize efficiently application idle time, whilst waiting the response
- Concurrency for CPU-bound tasks, that is a collection of techniques to leverage other available CPU cores (if any) for spreading computational tasks, when application isn’t idle.
From the network and IT infrastructure management we predominantly, but not always, deal with IO-bound tasks, whilst in data science including AI/ML, we typically deal with CPU-bound tasks
Concurrency in Python
For IO-bound concurrency, Python has two techniques:
- Multi-threading, which is an approach to spin up light threads on the same CPU core, where the Python process is started. It is relatively simple to implement and doesn’t require any specific support from libraries.
- Asyncio, which is the said to be the most scalable approach to IO-bound tasks as it asynchronously execute tasks on the same CPU core, where the Python process is started. More complicated to implement, but typically is more performant than threading. Also it requires Python libraries to implement async functionality, which is not the case for many Python packages.
Both of these approaches are used in IO-bound tasks as they use only one single core of one CPU.
For CPU-bound concurrency, Python implements:
- Multi-processing, which is an approach to spin up multiple Python processes on different CPU cores. It is as complicated as multi-threading, but has extra overhead on memory utilization as memory is to be copied from a parent to all child process, effectively multiplying its usage.
In theory it shall be possible to combine multi-threading and multi-processing, but I haven’t used it as it makes code quite complicated and, therefore, difficult to maintain.
Typically, you select between asyncio and multi-threading for infrastructure management, and the result will depend on many parameters. As such, testing is King.
Concurrency in Golang (Go)
Golang was developed later than Python (almost 18 year later) in the era of multi-core CPUs; therefore, it was created by speed. The approach to concurrency in Golang is called goroutines, which is similar to threads in Python with a major difference: Golang distributes goroutines across all the available cores, potentially placing multiple goroutines on a single core, which allows to achieve much better performance as all available CPU resources are utilized. In my experience, such approach on the one hand drives your overall CPU utilization across all cores up to 100%, which could lead to impact of other applications. On the other hand, you get 3x-30x, or even more, gain in performance, which allows you to complete your tasks much faster.
Example
In this scenario we will extend the code from our previous blog, which included GNMI/YANG and REST API towards NetBox.
In that blog, our example application, which is created both in Python and Go (Golang), fetches inventory from NetBox via REST API, and then connect to devices one by one using for-loop.
What we are going to do today is to modify the logic of that loop so that it can concurrently interact with multiple network devices, what will significantly improve application performance with any amount of network devices more than one.
Python
In Python we will use multi-threading approach. Library, which is needed for this is called “concurrent” and it is a part of standard package distribution in Python, what means you don’t need to install. As such, the only dependencies you need are the same as in the previous blog post:
Here is the code of the network automation application written in Python:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
# Modules
import datetime
import os
import sys
from dataclasses import dataclass
from typing import List, Tuple
import difflib
from concurrent.futures import ThreadPoolExecutor
import pygnmi.client
import httpx
# Classes
@dataclass
class Credentials:
"""Class to store credentials."""
username: str
password: str
@dataclass
class InventoryCredentials:
"""Class to store credentials."""
url: str
token: str
@dataclass
class Instruction:
"""Class to store instructions."""
command: List[str]
config: List[tuple]
@dataclass
class Result:
"""Class to store command execution data."""
instruction: Instruction
diff: str
timestamp: datetime.datetime
class Device:
"""Class to interact with netowrk device."""
def __init__(self, hostname: str, ip_address: str, port: int, platform: str, credentials: Credentials):
self.hostname = hostname
self.ip_address = ip_address
self.port = port
self.platform = platform
self.credentials = credentials
self.results: List[Result] = []
def execute_change(self, instruction: Instruction) -> None:
"""Method to execute change."""
# Connect to device
with pygnmi.client.gNMIclient(
target=(self.ip_address, self.port),
username=self.credentials.username,
password=self.credentials.password,
skip_verify=True,
timeout=5,
) as gconn:
# Get state before change
before = gconn.get(path=instruction.command, datatype="config")
before_stringified = self.dict_to_xpath(before)
# Apply change
config_result = gconn.set(update=instruction.config, encoding="json_ietf")
print(f"{config_result=}")
# Get state after change
after = gconn.get(path=instruction.command, datatype="config")
after_stringified = self.dict_to_xpath(after)
# Diff
diff = "\n".join(
difflib.context_diff(
before_stringified,
after_stringified,
lineterm="",
)
)
self.results.append(
Result(
instruction=instruction,
diff=diff,
timestamp=datetime.datetime.now(),
)
)
def dict_to_xpath(self, data: dict) -> list:
"""Method to convert dict to xpath."""
result = []
if isinstance(data, str):
return data
for key, value in data.items():
if isinstance(value, list):
for ind, item in enumerate(value):
tr = self.dict_to_xpath(item)
result.extend([f"{key}/{ind}/{_}" for _ in tr])
elif isinstance(value, dict):
tr = self.dict_to_xpath(value)
result.extend([f"{key}/{_}" for _ in tr])
else:
result.append(f"{key} = {value}")
return result
# Functions
def load_inventory(inventory: InventoryCredentials, credentials: Credentials) -> List[Device]:
"""Function to load inventory data."""
# Create HTTP client and set headers
hclient = httpx.Client(
base_url=inventory.url.rstrip("/"),
headers={"Authorization": f"Token {inventory.token}"},
)
# Retrieve data from REST API
try:
response = hclient.get(
"/api/dcim/devices/",
params={
"site": "ka-blog",
}
)
response.raise_for_status()
data = response.json()
except Exception as e:
print(e)
sys.exit(1)
# Populate list of devices
result = []
for device in data["results"]:
result.append(
Device(
hostname=device["name"],
ip_address=device["primary_ip"]["address"].split("/")[0],
port=device["custom_fields"].get("gnmi_port", 50051),
platform=device["platform"]["slug"],
credentials=credentials,
)
)
return result
def get_credentials() -> Tuple[Credentials, InventoryCredentials]:
"""Function to get credentials."""
return (
Credentials(
os.getenv("AUTOMATION_USER"),
os.getenv("AUTOMATION_PASS"),
),
InventoryCredentials(
os.getenv("AUTOMATION_INVENTORY_URL"),
os.getenv("AUTOMATION_INVENTORY_TOKEN"),
),
)
# Main code
if __name__ == "__main__":
# Get credentials
credentials, inventory_credentials = get_credentials()
# Load inventory
devices = load_inventory(inventory_credentials, credentials=credentials)
# Config
instruction = Instruction(
command=["/interfaces"],
config=[
(
"/interfaces",
{
"interface": [
{
"name": "Loopback 23",
"config": {
"name": "Loopback 23",
"description": "Test-gnmi-python-24",
}
},
],
},
),
],
)
# Execute command
with ThreadPoolExecutor(max_workers=10) as executor:
execution_results = executor.map(
lambda device: device.execute_change(instruction),
devices,
)
# Print results
for device in devices:
print(f"Device: {device.hostname}")
for result in device.results:
print(f"Config: {result.instruction.config}", f"Impact: {result.diff}", f"Timestamp: {result.timestamp}", sep="\n")
We suggest to read previous blogs to get better understanding of this code, as we omit many details in this post.
Here is what is changed:
- Class ThreadedPoolExecutor is imported from concurrent.future library. This class is used instantiate an object executor, which has capabilities spin up threads for tasks.
- Using method map() of the object executor, which has two arguments (can have more than two):
- Name of function (so called, callable), which is to be executed in a threaded way
- Arguments to this callable, which shall be an iterator (list, dictionary, etc). There could be more than one argument that you need; hence, it is possible to provide multiple iterators.
- Ultimately, we call in the threaded fashion method execute_change() of the device object, with list devices being the iterator for creating threads.
- The results of execution are stored in the object execution_results, which is iterator itself. We don’t use them however in this scenario, as we store results within the object device itself.
- We loop through results in the same way, as in the previous blog.
As you can see, adding multi-threading to Python is very straightforward. The syntax is different to for-loops, but it follows it in spirit, as you have what to do (callable) and iterator against which elements the callable it is to be executed.
Let’s see the execution of this Python application:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
ssl_target_name_override is applied, should be used for testing only!
ssl_target_name_override is applied, should be used for testing only!
config_result={'timestamp': 1745696412856245684, 'prefix': None, 'response': [{'path': 'interfaces', 'op': 'UPDATE'}]}
config_result={'timestamp': 1745696415292591043, 'prefix': None, 'response': [{'path': 'interfaces', 'op': 'UPDATE'}]}
Device: ka-blog-001
Config: [('/interfaces', {'interface': [{'name': 'Loopback 23', 'config': {'name': 'Loopback 23', 'description': 'Test-gnmi-python-23'}}]})]
Impact: ***
---
***************
*** 1,4 ****
! notification/0/timestamp = 1745696415279571842
notification/0/prefix = None
notification/0/alias = None
notification/0/atomic = False
--- 1,4 ----
! notification/0/timestamp = 1745696415677702589
notification/0/prefix = None
notification/0/alias = None
notification/0/atomic = False
***************
*** 48,58 ****
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/addresses/address/0/config/prefix-length = 31
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/addresses/address/0/ip = 10.0.0.1
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/config/enabled = True
! notification/0/update/0/val/openconfig-interfaces:interface/3/config/description = Test-gnmi-python-2
notification/0/update/0/val/openconfig-interfaces:interface/3/config/loopback-mode = FACILITY
notification/0/update/0/val/openconfig-interfaces:interface/3/config/name = Loopback23
notification/0/update/0/val/openconfig-interfaces:interface/3/config/type = iana-if-type:softwareLoopback
notification/0/update/0/val/openconfig-interfaces:interface/3/name = Loopback23
! notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/config/description = Test-gnmi-python-2
notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/index = 0
notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/config/enabled = True
--- 48,58 ----
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/addresses/address/0/config/prefix-length = 31
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/addresses/address/0/ip = 10.0.0.1
notification/0/update/0/val/openconfig-interfaces:interface/2/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/config/enabled = True
! notification/0/update/0/val/openconfig-interfaces:interface/3/config/description = Test-gnmi-python-23
notification/0/update/0/val/openconfig-interfaces:interface/3/config/loopback-mode = FACILITY
notification/0/update/0/val/openconfig-interfaces:interface/3/config/name = Loopback23
notification/0/update/0/val/openconfig-interfaces:interface/3/config/type = iana-if-type:softwareLoopback
notification/0/update/0/val/openconfig-interfaces:interface/3/name = Loopback23
! notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/config/description = Test-gnmi-python-23
notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/index = 0
notification/0/update/0/val/openconfig-interfaces:interface/3/subinterfaces/subinterface/0/openconfig-if-ip:ipv4/config/enabled = True
Timestamp: 2025-04-26 20:40:17.240665
Device: ka-blog-002
Config: [('/interfaces', {'interface': [{'name': 'Loopback 23', 'config': {'name': 'Loopback 23', 'description': 'Test-gnmi-python-23'}}]})]
Impact: ***
---
***************
*** 818,824 ****
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/openconfig-if-ip:ipv6/state/mtu = 1500
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/state/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/state/index = 0
! notification/0/update/0/val/openconfig-interfaces:interface/5/config/description = Test-gnmi-python-2
notification/0/update/0/val/openconfig-interfaces:interface/5/config/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/config/arista-intf-augments:load-interval = 300
notification/0/update/0/val/openconfig-interfaces:interface/5/config/loopback-mode = True
--- 818,824 ----
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/openconfig-if-ip:ipv6/state/mtu = 1500
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/state/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/4/subinterfaces/subinterface/0/state/index = 0
! notification/0/update/0/val/openconfig-interfaces:interface/5/config/description = Test-gnmi-python-23
notification/0/update/0/val/openconfig-interfaces:interface/5/config/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/config/arista-intf-augments:load-interval = 300
notification/0/update/0/val/openconfig-interfaces:interface/5/config/loopback-mode = True
***************
*** 833,839 ****
notification/0/update/0/val/openconfig-interfaces:interface/5/state/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/state/name = Loopback23
notification/0/update/0/val/openconfig-interfaces:interface/5/state/openconfig-vlan:tpid = openconfig-vlan-types:TPID_0X8100
! notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/description = Test-gnmi-python-2
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/index = 0
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/index = 0
--- 833,839 ----
notification/0/update/0/val/openconfig-interfaces:interface/5/state/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/state/name = Loopback23
notification/0/update/0/val/openconfig-interfaces:interface/5/state/openconfig-vlan:tpid = openconfig-vlan-types:TPID_0X8100
! notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/description = Test-gnmi-python-23
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/enabled = True
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/config/index = 0
notification/0/update/0/val/openconfig-interfaces:interface/5/subinterfaces/subinterface/0/index = 0
Timestamp: 2025-04-26 20:40:17.138377
Go (Golang)
In Golang there is no need to use any extra packages for concurrency, it is implemented via built-in functions. As such, like in Python, all we need is to install the same dependencies for gnmi and structs comparison as in the previous blog:
2
3
$ go get github.com/openconfig/gnmic/api
$ go get google.golang.org/protobuf/encoding/prototext
The following software written in Go (Golang) adds concurrency to interacting with network devices via GNMI/YANG (suitable for any protocol):
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
package main
// Imports
import (
"context"
"encoding/json"
"fmt"
"io"
"log"
"net/http"
"os"
"strings"
"time"
"github.com/google/go-cmp/cmp"
"github.com/openconfig/gnmic/pkg/api"
"google.golang.org/protobuf/encoding/prototext"
)
// Types and Receivers
type Arguments struct {
/* Class to starte CLI arguments */
Inventory string
}
type Crendetials struct {
/* Struct to store credentials. */
Username string
Password string
}
type InventoryCredentials struct {
/* Struct to store inventory credentails */
Url string
Token string
}
type Instruction struct {
Command string
Config struct {
Path string
Value any
}
}
type Result struct {
/* Struct to store command execution result. */
Instruction Instruction
Diff string
Timestamp time.Time
}
type Device struct {
/* Struct to interact with netowrk device. */
Hostname string `yaml:"hostname"`
IpAddress string `yaml:"ip_address"`
Port uint `yaml:"port"`
Platform string `yaml:"platform"`
Crendetials Crendetials
Result []Result
}
func (d *Device) executeChange(i Instruction) {
/* Method to execute command */
// Create GNMI Target
gnmiTarget, err := api.NewTarget(
api.Name(d.Hostname),
api.Address(fmt.Sprintf("%s:%d", d.IpAddress, d.Port)),
api.Username(d.Crendetials.Username),
api.Password(d.Crendetials.Password),
api.SkipVerify(true),
)
if err != nil {
log.Fatal("Cannot create GNMI Target: ", err)
}
// Create context
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
// Create GNMI client
err = gnmiTarget.CreateGNMIClient(ctx)
if err != nil {
log.Fatal("Cannot create GNMI Client: ", err)
}
defer gnmiTarget.Close()
// Get state before change
getReq, err := api.NewGetRequest(
api.Path(i.Command),
api.DataType("config"),
api.Encoding("json_ietf"),
)
if err != nil {
log.Fatal("Cannot create Get request: ", err)
}
beforeGetResponse, err := gnmiTarget.Get(ctx, getReq)
if err != nil {
log.Fatal("Cannot make a Get request: ", err)
}
beforeStruct := OpenConfigInterfaces{}
err = json.Unmarshal(beforeGetResponse.Notification[0].Update[0].Val.GetJsonIetfVal(), &beforeStruct)
if err != nil {
log.Fatal("Cannot unmarshall JSON: ", err)
}
// Make change
setReq, err := api.NewSetRequest(
api.Update(
api.Path(i.Config.Path),
api.Value(i.Config.Value, "json_ietf"),
),
)
if err != nil {
log.Fatal("Cannot create Set request: ", err)
}
setResp, err := gnmiTarget.Set(ctx, setReq)
if err != nil {
log.Fatal("Cannot make a Set request: ", err)
}
log.Println(prototext.Format(setResp))
// Get state after change
afterGetResponse, err := gnmiTarget.Get(ctx, getReq)
if err != nil {
log.Fatal("Cannot make a Get request: ", err)
}
afterStruct := OpenConfigInterfaces{}
err = json.Unmarshal(afterGetResponse.Notification[0].Update[0].Val.GetJsonIetfVal(), &afterStruct)
if err != nil {
log.Fatal("Cannot unmarshall JSON: ", err)
}
// Diff
diff := cmp.Diff(beforeStruct, afterStruct)
// Update the result
(*d).Result = append((*d).Result, Result{
Instruction: i,
Diff: diff,
Timestamp: time.Now(),
})
}
// Functions
func loadInventory(iC InventoryCredentials) *[]Device {
/* Function to load inventory data. */
// Create HTTP client
hclient := &http.Client{}
// Prepare request
NetboxRequest, err := http.NewRequest("GET", iC.Url+"/api/dcim/devices/", nil)
if err != nil {
fmt.Println("Error during preparing HTTP Request ", err)
os.Exit(1)
}
// Set headers
NetboxRequest.Header.Add("Authorization", fmt.Sprintf("Token %s", iC.Token))
// Set URL params
q := NetboxRequest.URL.Query()
q.Add("site", "ka-blog")
NetboxRequest.URL.RawQuery = q.Encode()
// Get data
resp, err := hclient.Do(NetboxRequest)
if err != nil {
fmt.Println("Erorr during executing HTTP query ", err)
os.Exit(1)
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
if err != nil {
fmt.Println("Error during reading body of HTTP response ", err)
os.Exit(1)
}
td := NetboxDcimDevices{}
err = json.Unmarshal(body, &td)
if err != nil {
fmt.Println("Error during parsing JSON ", err)
os.Exit(1)
}
// Load inventory
result := &[]Device{}
for _, v := range td.Results {
*result = append(*result, Device{
Hostname: v.Name,
Platform: v.Platform.Slug,
IpAddress: strings.Split(v.PrimaryIp4.Address, "/")[0],
Port: uint(v.CustomFields.GnmiPort),
})
}
// Return result
return result
}
func getCredentials() (Crendetials, InventoryCredentials) {
/* Function to get credentials. */
return Crendetials{
Username: os.Getenv("AUTOMATION_USER"),
Password: os.Getenv("AUTOMATION_PASS"),
},
InventoryCredentials{
Url: os.Getenv("AUTOMATION_INVENTORY_URL"),
Token: os.Getenv("AUTOMATION_INVENTORY_TOKEN"),
}
}
// Main
func main() {
/* Core logic */
// Get credentials
sshCreds, invCreds := getCredentials()
// Load inventory
inventory := loadInventory(invCreds)
// Config
instruction := Instruction{
Command: "/interfaces",
Config: struct {
Path string
Value any
}{
Path: "/interfaces",
Value: map[string]any{
"interface": []map[string]any{
{
"name": "Loopback 23",
"config": map[string]any{
"name": "Loopback 23",
"description": "Test-gnmi-golang-23",
},
},
},
},
},
}
// Create communication channel
c := make(chan Device)
// Execute commands
for i := 0; i < len(*inventory); i++ {
// Set credentals
(*inventory)[i].Crendetials = sshCreds
// Launch goroutines
go func(d Device, ins Instruction, c chan<- Device) {
// Execute task
d.executeChange(ins)
// Send device back
c <- d
}((*inventory)[i], instruction, c)
}
// Collect results
iventory_with_results := make([]Device, 0)
for i := 0; i < len(*inventory); i++ {
iventory_with_results = append(iventory_with_results, <-c)
}
// Print results
for i := 0; i < len(iventory_with_results); i++ {
for j := 0; j < len((iventory_with_results)[i].Result); j++ {
fmt.Printf(
"Config: %v\nImpact: %v\nTimestamp: %v\n",
(iventory_with_results)[i].Result[j].Instruction.Config,
(iventory_with_results)[i].Result[j].Diff,
(iventory_with_results)[i].Result[j].Timestamp,
)
}
}
}
We suggest to read previous blogs to get better understanding of this code, as we omit many details in this post.
Some Details On Goroutine
Implementation of concurrency in Golang is more complicated than in Python, which requires two components:
- Goroutine, which is an instruction to start task as a separate goroutine (read thread). The syntax is as follows:
2
3
a++
}(10)
- The keyword “go” in front of function name, or in this case anonymous function, same as lambda-funciton in Python instructs for function start as goroutine.
- Second component is called channel, and this is pretty much a single mechanism to pass data from one goroutine to another one. Channel is a variable, which you create using funciton make(), same as you do for slices and maps. the content of the channel is the data type you want to transfer from one goroutine to another prepended by the instruction “chan“.
- Here is how full example with goroutine and channel looking like:
2
3
4
5
6
7
8
go func (a int, c chan int){
a++
c <- a
}(10)
newData := <- c
Two important notes: even if your gouroutine is a method (receiver function) of pointer, you HAVE to communicate it back; original pointer won’t change (1). If you are using gorotine to modify shared variable, you need mutex (2)
Back to explanation
Based on the quick explanation above, let’s see what’s changed in the code:
- The channel c is created with Device data type:
2
c := make(chan Device)
- Within the for loop, the goroutine is started to interact with every device. The inputs to goroutine are:
- Device, which the interaction is done against
- Instruction, which interaction shall it be
- channel Device, to communicte back the changed state with results
2
3
4
5
6
7
8
9
10
11
12
13
14
for i := 0; i < len(*inventory); i++ {
// Set credentals
(*inventory)[i].Crendetials = sshCreds
// Launch goroutines
go func(d Device, ins Instruction, c chan<- Device) {
// Execute task
d.executeChange(ins)
// Send device back
c <- d
}((*inventory)[i], instruction, c)
}
- New slice inventory_with_results are created by created as an empty slice, which is populated with date from c channel:
2
3
4
5
iventory_with_results := make([]Device, 0)
for i := 0; i < len(*inventory); i++ {
iventory_with_results = append(iventory_with_results, <-c)
}
- The rest of the code is identical
Such a code allows you to start interactions with all the devices in your inventory simultaneousely and then to process results as they are ready. For example, if you have a few devices, which are slow to reponse, and a lot of others, which are quick, the duration of this code execution will be defined by the slowest device alone.
Let’s execute this Go (Golang) application:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
2025/04/26 20:56:36 response: {
path: {
elem: {
name: "interfaces"
}
}
op: UPDATE
}
timestamp: 1745697392374689242
2025/04/26 20:56:36 response: {
path: {
elem: {
name: "interfaces"
}
}
op: UPDATE
}
timestamp: 1745697394881050173
Config: {/interfaces map[interface:[map[config:map[description:Test-gnmi-golang-23 name:Loopback 23] name:Loopback 23]]]}
Impact: main.OpenConfigInterfaces{
Interface: []main.OpenConfigInterface{
... // 3 identical elements
{Name: "Loopback51", Config: {Name: "Loopback51", Description: "pytest-update-test-33"}},
{Name: "Loopback0", Config: {Name: "Loopback0", Enabled: true}},
{
Name: "Loopback23",
Config: struct{ Name string "xml:"name,omitempty" json:"name,omitempty""; Description string "xml:"description,omitempty" json:"description,omitempty""; Enabled bool "xml:"enabled,omitempty" json:"enabled,omitempty"" }{
Name: "Loopback23",
- Description: "Test-gnmi-golang-3",
+ Description: "Test-gnmi-golang-23",
Enabled: true,
},
},
},
}
Timestamp: 2025-04-26 20:56:36.703237967 +0100 BST m=+1.125671022
Config: {/interfaces map[interface:[map[config:map[description:Test-gnmi-golang-23 name:Loopback 23] name:Loopback 23]]]}
Impact: main.OpenConfigInterfaces{
Interface: []main.OpenConfigInterface{
{Name: "Management1", Config: {Name: "Management1"}},
{Name: "Ethernet2", Config: {Name: "Ethernet2"}},
{Name: "Ethernet1", Config: {Name: "Ethernet1"}},
{
Name: "Loopback23",
Config: struct{ Name string "xml:"name,omitempty" json:"name,omitempty""; Description string "xml:"description,omitempty" json:"description,omitempty""; Enabled bool "xml:"enabled,omitempty" json:"enabled,omitempty"" }{
Name: "Loopback23",
- Description: "Test-gnmi-golang-3",
+ Description: "Test-gnmi-golang-23",
Enabled: false,
},
},
},
}
Timestamp: 2025-04-26 20:56:37.002338539 +0100 BST m=+1.424771596
Lessons in GitHub
You can find the final working versions of the files from this blog at out GitHub page.
Conclusion
This blog post concludes our introduction to Golang for network and IT infrastructure management with Python serving as a scaffolding. Within the course of 21 blog posts, we’ve covered everything you need to start developing and using software for IT and network infrastructure management with both Golang and Python. The range of topics spans basics, such as data types, working with variables and code flow control to object-oriented programming and parsing/serializing XML/JSON/YAML to handling exceptions and user input to existing protocols for interacting with servers and network devices (SSH, NETCONF/YANG, GNMI/YANG) to templating and concurrency.
We may create second series in future, where we will cover more advanced protocols and external components, such as various databases, message brokers and advanced APIs. If you want to make this happen, please, star our GitHub repo and/or repost our posts so that we see it is needed for you and for the community.
Thank you so much for reading us. Take care and good bye.
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk
