Site icon Karneliuk

From Python to Go 007. (Data)Classes, Structs, and Custom Data Types.

Anton Karneliuk

Hello my friend,

Today we are going to talk about the last data type, which in generally exists in Python and Go (Golang), and which we need dearly for all meaningful applications including network and IT infrastructure automation. We are talking about structured, typed data, which is represented in Python in form of objects and classes and in Go (Golang) in form of structs. These structures are truly powerful and once you figure out how to use them, I’m quite confident you will be using it everywhere, where you can.

You Talk So Much About Go (Golang), But You Offer Python In Trainings. Why?

This question I’ve been asked rather frequently recently. Indeed, why do we in each blog post talk about Network Automation Trainings, which gravitate around Python/Ansible duet. The answer is straightforward: whilst Go (Golang) is very powerful as we showing it in these blogs, its usage in many cases is justified only in very high-scale environment. For majority of networks and IT systems, Python is great. It is suffice to say that entire OpenStack is built in Python. And ourselves we use it extensively in many customers’ systems. Go (Golang) is useful as extension of automation skills, not as replacement of Python in our opinion. Therefore…

We offer the following training programs in network automation for you:

During these trainings you will learn the following topics:

Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data centre networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depth and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.

Start your automation training today.

What Are We Going To Talk Today About?

The real story from my perspective is that I’ve avoided using structured data in Python for literally years. Why to bother to think about classes (that is collection attributes and methods) and predefined attributes (that is the name of the variable within the class) if for the purpose of storing data I can use dictionaries. Putting aside the object-oriented programming (which is the topic for the next blog post), if I solely wanted to store data, I didn’t use classes. Things changed, when I started building API and web-apps using fastAPI and Django.

By the way, you can learn FastAPI and build blazing fast APIs for your network and infrastructure automation in our closed-loop automation training.

Things changed, because those frameworks forces you to use structured strictly-typed data. And they force you to use structured strictly-typed data on API layer, where you define what application shall receive from external interaction or what it shall it send back in a response. It is critical to have API create clear, as quite often developers of client- and server-side applications are very different people, often they even don’t know each other and all they have to ensure that applications can talk to each other is API documentation. After I’ve appreciated this concept truly,we’ve started using it actively internally, wherever we need to communicate data between functions, etc. Then, when I was on a C-journey, I’ve used structs a lot, as there are no objects in C. Likewise, there are no objects in Go (Golang), but there are structs like in C as well.

With that lengthy introduction, let’s get to the core of our discussion. We are going to overview:

  1. What are (data)classes in Python and structs in Go (Golang)?
  2. Which problems do they solve?
  3. How and where to use them?

Explanation

Before jumping to the core of today’s topic, let’s first understand the problem it solves. In previous blog post we’ve described dictionaries/maps in Python and Go (Golnag), and as a part of the discussion we showed how to check if a certain key exists in dictionary/map. You need to do that, because dictionary/map per-se doesn’t have any schema, meaning a definition of:

If you try to call non-existing key in dictionary/map, you will get an error during the execution of your application.

All these questions are answered, when we use structured strictly-typed data types. In Python this is achieved using classes and objects.

Classes and objects are far more than just structured data, they are at heart of object-oriented programming (OOP) in Python. That’s why we will touch base on OOP in the next blog post.

It works as such:

Python is not strictly typed, so to achieve the true type-strictness, external libraries needs to be used.

Go (Golang) isn’t an object-oriented, programming language, so it doesn’t have concept of objects and classes. Instead, much like C, it is a procedural programming language. So it doesn’t have classes. Instead, it has structs, which is serving the exact purpose of having structured strict typed data.

At a pseudo code level, it would look like as follows:



1
2
3
4
5
6
7
8
9
10
datacontainer SomeData:

   key1 : data_type_1

   key2 : data_type_2

   key3 : data_type_3

   key4 : data_type_4



item = SomeData(key1="abc", key2=123, ...)



print(item.key1)

some_other_func(item)

Where:

There is no such term as data container neither in Python or Go (Golang), we’ve completely made up this term for the purpose of this discussion

Let’s see how it works on the code examples.

Examples

We are going to implement the following scenario:

  1. You are going create three structured data:
    • one to store credentials, which shall have 2 keys: username and password.
    • one to store information about devices, that is hostname, ip address, port to connect, and network operating system
    • one to store multiple devices to act as an inventory
  2. These data structures shall be populated from environment variables.
  3. Print the content of the variables

Python

We propose the following code in Python to complete this scenario:



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
"""From Python to Go: Python: 007 - Classes and Structs"""



# Import os

import os

from  typing import List





# Data models

class User:

    """Class to store user credentials"""

    def __init__(self, username, password):

        self.username: str = username

        self.password: str = password





class Device:

    """Class to store device information"""

    def __init__(self, name: str, port: int, nos: str = None, ip: str = None):

        self.name: str = name

        self.port: int = port

        self.nos = nos

        self.ip = ip





class Inventory:

    """Class to store inventory information"""

    def __init__(self):

        self.devices: List[Device] = []





# Functions

def get_credentials() -> User:

    """Function to retrieve credentials from the environment"""

    return User(*os.getenv("AUTOMATION_CREDS").split(","))





def get_inventory() -> Inventory:

    """Function to retrieve inventory from the environment"""

    # Create an empty list to store devices

    result = Inventory()



    # Loop through the environment variables

    for key, value in os.environ.items():

        # Check if the key starts with AUTOMATION_DEVICE_

        if key.startswith('AUTOMATION_DEVICE_'):

            # Split the value by comma and create a new device object

            split_value = value.split(',')

            result.devices.append(

                Device(

                    name=split_value[0],

                    port=int(split_value[1]),

                    nos=split_value[3],

                    ip=split_value[2],

                )

            )



    # Return the result

    return result





# Execution

if __name__ == "__main__":

    # Get the credentials

    user = get_credentials()



    # Print the credentials

    print(f"Username: {user.username}")

    print(f"Password: {user.password}")



    # Get the inventory

    inventory = get_inventory()



    # Print inventory memory address

    print(f"Memory address of inventory: {id(inventory):#x}")



    # Print the inventory

    print("Inventory:")

    for device in inventory.devices:

        print(f"Device: {device.name}")

        print(f"Port:   {device.port}")

        print(f"IP:     {device.ip}")

        print(f"NOS:    {device.nos}")

        print("\n")

Many concepts we have covered in previous blog posts in this series about Python and Go (Golang). We recommend to look in previous to get more information about for-loops, module imports, and read from Linux/MAC environment.

Let’s review the key new components:

  1. We have classes instances, which start with the keyword “class” followed by the name of the class.
  2. Each class has a mandatory constructor method “__init__()“. Method is a function associated with class (much like attribute is a variable associated with class). This function may take external arguments.
  3. These external arguments we convert to attributes (e.g., self.username, self.password are attributes).
  4. Self is a representation of the object from the its own inside. All methods must have self as the first argument. This though we will discuss in the next blog post, for now we use just a single built-in method “__init__()“
  5. You can provide a default value for specific variable. E.g., “nos : str = None” means that argument nos shall be of type string with default value set to None, which is not a string. That highlights that Python is not strict-typed by default.
  6. Function “get_credentails()” returns instance of “User” class. Within this function we create a new object, which requires two arguments: username and password. We can pass them in a key-worded format, meaning specifying pairs of “key=value” or using positional format, where arguments must be in the same order as defined in “__init__()” function. The second approach is taken, which resulted in line “*os.getenv(“AUTOMATION_CREDS”).split(“,”)”. Let’s decode it:
    • Function “os.getenv(“AUTOMATION_CREDS”)” looks in environment variable “AUTOMATION_CREDS” and return its value.
    • This value is then split over comma using “.split(“,”)” method into a list
    • and then using “*” in the beginning of this statement we turn list into a sequence individual positional arguments for the function.
  7. Function “get_inventory()” return instance of “Inventory” class. This function iterates over dictionary keys, and if the key starts with sub-string “AUTOMATION_DEVICE_“, then its value split into a list with “,” as separator. Then the instance of “Device()” class is created and is added to the inventory. The instance is populated this time using key-worded arguments, with port argument being also passed via type conversion.
  8. The main code is executed. Cryptic command “f”{id(inventory):#x}”” returns in hexadecimal format the memory address of the variable, which contains this variable.

Here is the example of the code execution:



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
$ python main1.py 

Username: karneliuk

Password: lab

Memory address of inventory: 0x7f93376eb880

Inventory:

Device: leaf-1

Port:   830

IP:     192.168.1.1

NOS:    arista-eos





Device: leaf-2

Port:   22

IP:     192.168.1.1

NOS:    cisco-nxos

As mentioned a few times, Python by default isn’t strict typed language. For it to be strictly typed, you need to add quite a bit of validation for your input data, as nothing prevents you to put boolean instead of string and this code will accept it just fine. Rather than you write all the validators yourself, you can rely on 3rd party packages. The one we are using a lot ourselves is called “pydantic“, which is a major driving power behind FastAPI. In the code examples for this post in GitHub you can find example written with Pydantic.

Go (Golang)

The very same flow we’ve just created in Python using classes and objects we now re-create in Go (Golang):



1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
/* From Python to Go: Go (Golang): 007 - Classes and Structs" */



package main



// Imports

import (

    "fmt"

    "os"

    "strconv"

    "strings"

)



// Data types

type User struct {

    // Class to store user credentials

    username string

    password string

}



type Device struct {

    // Class to store device information

    hostname string

    port     uint64

    ip       string

    nos      string

}



// Class to store inventory information

type Inventory []Device



// Aux functions

func getCredentials() User {

    // Function to retrieve credentials from the environment

    blocks := strings.Split(os.Getenv("AUTOMATION_CREDS"), ",")

    return User{

        blocks[0],

        blocks[1],

    }

}



func getInventory() *Inventory {

    // Create an empty list to store devices

    result := &Inventory{}



    // Loop through the environment variables

    for _, kv := range os.Environ() {

        // Check if the key starts with AUTOMATION_DEVICE_

        if strings.Contains(kv, "AUTOMATION_DEVICE_") {

            // Split the value by comma and create a new device object

            blocks := strings.Split(strings.Split(kv, "=")[1], ",")



            devicePort, err := strconv.ParseUint(blocks[1], 10, 64)

            if err != nil {

                fmt.Printf("Got error when converting string to uint: %v\n", err)

                os.Exit(1)

            }



            *result = append(*result, Device{

                blocks[0],

                devicePort,

                blocks[3],

                blocks[2],

            })

        }

    }



    // Result

    return result

}



// Main function

func main() {

    /* Main business logic */



    // Get the credentials

    user := getCredentials()



    // Print credentails

    fmt.Printf("%+v\n", user)



    // Get inventory

    inventory := getInventory()



    // Print inventory memory address

    fmt.Printf("Memory address of inventory: %v\n", &inventory)



    // Print inventory content

    fmt.Printf("%+v\n", *inventory)

}

As we mentioned in Python, many concepts we’ve covered before; please, refer to previous blog posts if you have questions.

Here is what relevant new here:

  1. The structs are defined using the instruction “type“, which stands for type definition. You need to provide name and the type, which is “struct” in two cases and slice of defined struct “[]Device” in one case. Which each type you specify keys and associated data types. In contrast to Python, Go is by default strict-typed, so you have to ensure your data will match.
  2. We create a helper function “getCredentials()” which returns struct “User“. The struct is populated in Go using positional arguments, so we read from environment the variable of “AUTOMATION_CREDS” using “os.Getenv()” function and then split it into a list, which is fed to “User” struct.
  3. The next helper function “getInventory()” returns pointer to a struct “Inventory“, what allows to avoid copy data between functions, which is a significant performance improvement if you have a few hundreds of devices in your inventory. The pointer is created using “result := &Inventory{}” statement, which equals to “new()” function covered earlier in this series. Within this function we iterate through all environment variables and when we find a match for our pattern, this variable is split using “,” as separator and then populate struct Device(). There is a bit of complexity in comparison to Python, as we need to convert string to unsigned integer. This is achieved in Golang using function “ParseUint()” from “strconv” library, which returns two values: converted value and error. The latter shall be checked to ensure that conversion was successful.
  4. In the main block we call the functions and print their results. When we print content of structs, it is useful to use “%+v” pattern instead of “%v“. The former will print both key and value, whilst the latter prints only value.

Here is the result of execution:



1
2
3
4
$ go run .

{username:karneliuk password:lab}

Memory address of inventory: 0xc00005a048

[{hostname:leaf-1 port:830 ip:arista-eos nos:192.168.1.1} {hostname:leaf-2 port:22 ip:cisco-nxos nos:192.168.1.1}]

It is worth mentioning that in Go (Golang) you can create custom data types not only for structs or list of structs, but for any other data type, including standard ones. In the next blog post you will learn why you may want to do it.

Lessons in GitHub

You can find the final working versions of the files from this blog at out GitHub page.

Conclusion

This blog post concludes our overview of all data types existing in Python and Go (Golang). If you completed all the labs, which is a prerequisite for proper understanding of this topic, you shall have by now a good grasp of how to store data in these programming languages and how to use it for computations. In the next blog we’ll touch, as mentioned a few times, on the topic of object-oriented programming. Take care and good bye.

Support us






P.S.

If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.

BR,

Anton Karneliuk 

Exit mobile version