Hello my friend,
Whenever we develop any network and IT infrastructure automation applications, we need to have some options to provide user input. In previous blog posts in these series we already covered how to provide user input via environment variables and files. Whilst these two approaches can cover majority of your use cases, especially if you develop containerized applications running in autonomy, there are still two options we would like to talk today about.
Why To Bother Learning Automation?
For many years I was doing network design and operation without automating it (or at least without structured approach to automate it). And there are still loads of such job positions out there. And I see it based on the audience of my blog: majority of people here for networking knowledge, much less are for automation topics. From pure pragmatic standpoint of writing popular blogs, I should stick to network technologies, especially something fancy as SD-WAN and others. However, from the direction of the technologies development, I see that value (including jobs) comes from intersection of domains: networking, compute, storage, software development, data bases, Kubernetes, observability, etc. I’m of a strong opinion that engineers these days must be aware of all the domains and be reasonable well in them. Otherwise they lock themselves to a small niche and miss all the wonders of IT career.
And that’s what we encourage you not to do. Instead, study network and IT infrastructure automation and be ready for wonders.
We offer the following training programs in network automation for you:
- Zero-to-Hero Network Automation Training
- High-scale automation with Nornir
- Ansible Automation Orchestration with Ansble Tower / AWX
- Expert-level training: Closed-loop Automation and Next-generation Monitoring
During these trainings you will learn the following topics:
- Success and failure strategies to build the automation tools.
- Principles of software developments and the most useful and convenient tools.
- Data encoding (free-text, XML, JSON, YAML, Protobuf).
- Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
- Full configuration templating with Jinja2 based on the source of truth (NetBox).
- Best programming languages (Python, Bash) for developing automation
- The most rock-solid and functional tools for configuration management (Ansible) and Python-based automation frameworks (Nornir).
- Network automation infrastructure (Linux, Linux networking, KVM, Docker).
- Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.
- Collection network data via SNMP and streaming telemetry with Prometheus
- Building API gateways with Python leveraging Fast API
- Integration of alerting with Slack and your own APIs
- … and many more
Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data center networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depth and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.
Start your automation training today.
What Are We Going To Talk Today?
As aforementioned, there is a number of essential ways for user to provide input to application to avoid hard-coding things and ensure applications are fit for purpose:
| Approach | Use Case | Covered |
|---|---|---|
| Environment variables | Passing parameters to application. Often used in cloud-native deployments as perfect way to pass parameters in non-interactive way. Suitable for providing sensitive data, such as credentials, as not seen in launch logs, etc. | Yes |
| Files | Used when providing huge amount of non-sensitive information (e.g., inventory files) or application configuration. Needs to be available at application launch. Also can be used in cloud-native scenarios as Kubernetes Config Map. | Yes |
| CLI arguments | Often used to control execution of code (e.g., enabling debug mode) or providing certain parameters (e.g., username, input file path, etc). Applicable in cloud-native deployment as command section in deployment manifests. | Will be covered today |
| Standard Input | This is the only method requiring user after application is launched. Therefore, it is not used in cloud deployments of automated jobs. The primary area is CLI tools (e.g terraform, etc), where user is required to review some output and dynamically confirm or reject further actions. Also can be used as a last-resort option if neither environmental nor CLI arguments are provided. | Will be covered today |
Although we emphasized cloud-native deployments, all these approaches are perfectly working outside of Kubernetes as well.
With this in mind, we are going to review today:
- What are CLI arguments?
- How and when to use them?
- What is standard input?
- How and when to use it?
Explanation
Why do we at all need user input into applications? Despite the question may seem silly, we always need to get foundations right before building anything. From our perspective, the following list, although not conclusive, is a good start point:
- Providing sensitive variable data. For example, user credentials. You wouldn’t like to store credentials in your code due to security reasons, neither you would like to change code of application every time credentials change.
- Controlling execution of application (e.g., enabling/disabling some parts of code, changing logging severity, etc). Rewriting and rebuilding code each time you want to enable debugging output is not practical.
- Providing setup specific details. For example, paths to configuration files, etc.
- User confirmation for application execution to avoid changes in states if application was launched by mistake.
- Application configuration data (e.g., inventory files), etc.
This list can go on and on. The key point here us that application requires user input. As we already touched files and environmental variables before, we now will focus on user input via CLI arguments and standard input.
CLI arguments
In a nutshell, CLI arguments is what you provide in the terminal after the name of application. For example:
In this snippet “application” is the name of application you start, which is in this case is followed by three arguments “arg1“, “arg2” and “arg3“.
This approach allows you to provide data to your application, which can be used to control and/or enrich execution of the application functionality. However, what if you don’t need to provide value of the arg2 as you can rely on some default value, whilst you need arg1 and arg3? It is not easy to achieve it in the provided example as these arguments are positional, meaning that the value is associated with certain key depending on the value position after the argument.
To overcome this limitation, they key-worded arguments are used. The implementation of key-worded arguments with respect to CLI is called flags. Here is example of flags:
With such an approach you specify yourself, which parameters are needed and which are not. You can also specify them in any order you want.
Various network and IT infrastructure automation tools uses flags heavily: “ansible“, “kubectl” to mention at least.
Standard Input
Standard input is literally what you types in terminal. With respect to passing user data into applications, standard output looks like answering questions when applications asks you. Take a look at the following snippet:
2
3
Provide username: abc
Provide password:
Here the username and password are provided via standard input. You don’t see the password though. The reason for that is that not all the information you provide via standard output you want to be visible for security and privacy reasons. For example, all the passwords you don’t want to be seen on terminal as anyone standing behind your shoulder (or audience at your webcast) will be able to see the passwords.
Typically you would use standard input in CLI applications, where user really interacts with application, not just passing data between. For example, terraform utilizes this approach a lot, where user is asked to provide value for variables, if theses variables aren’t provided via different channels (e.g., environmental variables for configuration files), or user is asked to provide confirmation if he/she is really up for deploying/destroying configuration.
Examples
To show how both concepts work in Python and Golang, we’ll implement the following scenario:
- The path of the file, which shall be read by application, shall be provided as key-worded CLI argument.
- The user credentials shall be provided via standard input with the following constraints:
- The username can be visible in standard input when you type it.
- The password cannot be visible in standard input when you type it.
Python
As usual, we start with example in Python:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
# Modules
import argparse
from getpass import getpass
from dataclasses import dataclass
import sys
# Classes
@dataclass
class Credentials:
"""Class to store credentials."""
username: str
password: str
# Functions
def read_args() -> argparse.Namespace:
"""Helper function to read CLI arguments."""
parser = argparse.ArgumentParser(description="User input.")
parser.add_argument("-p", "--path", type=str, help="Path to the input file.")
return parser.parse_args()
def load_file(path) -> str:
"""Function to load a file."""
with open(path, "r", encoding="utf-8") as file:
return file.read()
# Main
if __name__ == "__main__":
# Get arguments
args = read_args()
# Load file
if args.path:
print(load_file(args.path))
# Exit if no path provided
else:
sys.exit("No path provided.")
# Get user input
creds = Credentials(username=input("Username: "), password=getpass("Password: "))
print(f"{creds=}")
Read the previous blogs in the series to get better overview of what happens in this code beyond what’s explained below.
The high-level explanation:
- We are importing a few libraries we haven’t used before
- “argparse” is used to deal with CLI arguments.
- “getpass” deals with user input, when you need to avoid showing what you type in terminal.
- We are defining the helper function “read_args()“, which doesn’t take any arguments and returns an object with arguments provided via CLI:
- First of all, object of the class “argparse.ArgumentParser” is created.
- Then arguments are added. The two consequtive positional arguments “-p” and “–path” are short and long flag name how you pass the argument via CLI. The latter is also a name of attribute within resulting object of “argparse.Namespace” class.
- To process user arguments, the method “.parse_args()” is called on the parser object.
- Helper function “load_file(path: str)” reads content of the file from the provided path.
- Inside the main body, the new concept is how the data class “Credentials” is populated:
- Using built-in function “input()” the username is requested via stdin (standard input) with prompt “Username: “.
- Using function “getpass()” from “getpass” module the password is requested with prompt “Password: “. What you type in terminal isn’t displayed there for this function, which matches the security and privacy requirements.
Let’s see the execution of this code:
2
3
4
5
6
7
8
9
10
11
12
13
14
[credentials]
username: karneliuk
password: lab
[inventory]
hostname: leaf-1
ip_address: 192.168.1.1
port: 830
nos: arista-eos
Username: abc
Password:
creds=Credentials(username='abc', password='def')
As you can see, the file is read from path provided in CLI arguments as well as credentials are picked up from stdin. It is worth mentioning that for CLI arguments the automatic help is built, which you can call by passing argument “-h” or “–help”:
2
3
4
5
6
7
8
usage: main.py [-h] [-p PATH]
User input.
options:
-h, --help show this help message and exit
-p PATH, --path PATH Path to the input file.
Go (Golang)
Now it is Go (Golang) term. In contrast to Python, which has a built-in function for reading passwords, Go (Golang) doesn’t have one. So we need to install the relevant external package “term”:
Once the package is installed, we are ready to write the code of our application in Go (Golang):
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
package main
// Import
import (
"flag"
"fmt"
"os"
"golang.org/x/term"
)
// types
type CliFlags struct {
Path string
}
type Credentials struct {
Username string
Password string
}
// Functions
func readArgs() CliFlags {
/* Helper function to read CLI arguments. */
// Prepare result
result := CliFlags{}
flag.StringVar(&result.Path, "p", "", "Path to the input file.")
// Parse arguments
flag.Parse()
// Result
return result
}
func loadFile(p string) string {
/* Function to load a file. */
bs, err := os.ReadFile(p)
if err != nil {
os.Exit(2)
}
// Result
return string(bs)
}
func getCreds() Credentials {
/* Helper function to get credentials */
// Initialise result
result := Credentials{}
// Read Username
fmt.Print("Username: ")
_, err := fmt.Scanln(&result.Username)
if err != nil {
os.Exit(1)
}
// Read password
fmt.Print("Password: ")
bytepw, err := term.ReadPassword(int(os.Stdin.Fd()))
if err != nil {
os.Exit(1)
}
result.Password = string(bytepw)
// Return result
return result
}
// Main
func main() {
/* Main business logic */
// Get arguments
arg := readArgs()
// load file
if arg.Path != "" {
fmt.Println(loadFile(arg.Path))
// Exit if no path provided
} else {
os.Exit(3)
}
creds := getCreds()
fmt.Printf("\n%+v\n", creds)
}
Read the previous blogs in the series to get better overview of what happens in this code beyond what’s explained below.
Here is what happens:
- A number of packages are imported:
- Built-in “flag” to deal with CLI flags.
- Built-in “fmt” to process user stdin input in addition to what we used it before: printing to stdout.
- Built-in “os” package to read files, raise errors, etc.
- External package “golang.org/x/term” to process user input, where privacy is required.
- Structs are defined to store:
- CLI arguments (called in our code “CliFlags“). In contrast to Python, which automatically builds data class based on flags, in Go (Golang) it is required to create a struct, which will be populated with value of arguments.
- User credentials (called in our code “Credentials“).
- Much as Python, we create helper function “readArgs()“, which doesn’t take any input but returns a struct. Its logic is broadly similar, but not identical:
- We create instance of “CliFlags” struct.
- Using “StringVar” function from “flag” package, we write content of the flag “p” to the memory address specified in pointer “&result.Path“.
- Function “parse()” from “flag” package is utilized to read CLI arguments and essentially populate the “result” variable.
- Helper function “loadFile(p string)” is used to read content of the file from the specified path.
- We’ve defined new function “getCreds()” which reads user data from stdin:
- To read data without protecting the output, the function “Scanln()” can be used from “fmt” package, which stores the result of what is read into pointer, so you need to provide pointer as an execution argument. You also need to print the prompt separately, which is used by “Print()” function.
- To read data with privacy, we use “ReadPassword()” function from installed “term” package. Input of the function is an ID of the file descriptor, associated with the input terminal, which is fetched via “os.Stdin.Fn()” function. The output of the function is a byte slice with read data without newline character and an error. The byte slice is cast with “string” data type before storing in the struct key.
- Finally, in the main block:
- arguments are read from CLI via the helper function “readArgs()“.
- File is read using the helper function “loadFile()“.
- Credentials are read from stdin using the helper function “getCreds()“.
Let’s execute this application written in Go (Golang:
2
3
4
5
6
7
8
9
10
11
12
13
14
[credentials]
username: karneliuk
password: lab
[inventory]
hostname: leaf-1
ip_address: 192.168.1.1
port: 830
nos: arista-eos
Username: abc
Password:
{Username:abc Password:def}
As much as Python, Go (Golang) automatically generates help for CLI arguments:
2
3
4
Usage of /tmp/go-build1247901543/b001/exe/012:
-p string
Path to the input file.
Important difference between Python and Go (Golang) is that Golang flags are always prepended by a single dash “–“, whilst in Python it can be single for short flags and double for long flags.
Go (Golang) has other non-built-in packages for parsing CLI input, where you build complex CLIs. One of the popular and useful examples is Kong, which is helpful when you build heavy CLI application requiring a lot of custom user input.
Lessons in GitHub
You can find the final working versions of the files from this blog at out GitHub page.
Conclusion
By now we covered all possible (if not all possible, then definitely all you need for network and IT network automation applications) user inputs. As such, you are equipped with knowledge how to process user data in different cases and can use it for development of your application. In the next blog post we’ll cover one more important foundational topic, which is essential for automation tools that is handling exceptions in your applications. Take care and good bye.
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk
