Dear friend,
It has been a long, very long break in posting. In fact, it was almost for half a year, which are not proud of in any capacity. However, this was needed to allow my to accomplish a few things in my own study and get back even more loaded, so that I can share more useful information and insights about networking, computing, software development and IT in general.
So, Have You Stopped Doing Automation?
We have absolutely not. In fact, developing software for distributes systems, which requires network connectivity between their components, I see how critical is to have the entire IT infrastructure (including networking) to have done right. By “done right” I mean the IT/network infrastructure, which is stable, predictable and deterministic. And in the fast pace environment, where you scale your infrastructure quick up and down (we are not talking about Kubernetes here), the network automation is the only way to achieve this level of precision and sustainability. So, my dudes, network automation is THE ONLY way how the network can be operated in now and in future. Your software development/business teams will be happy if you do things in your network automated way, I can judge by my own experience.
We offer the following training programs in network automation for you:
- Zero-to-Hero Network Automation Training
- High-scale automation with Nornir
- Ansible Automation Orchestration with Ansble Tower / AWX
- Expert-level training: Closed-loop Automation and Next-generation Monitoring
During these trainings you will learn the following topics:
- Success and failure strategies to build the automation tools.
- Principles of software developments and the most useful and convenient tools.
- Data encoding (free-text, XML, JSON, YAML, Protobuf).
- Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
- Full configuration templating with Jinja2 based on the source of truth (NetBox).
- Best programming languages (Python, Bash) for developing automation
- The most rock-solid and functional tools for configuration management (Ansible) and Python-based automation frameworks (Nornir).
- Network automation infrastructure (Linux, Linux networking, KVM, Docker).
- Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.
- Collection network data via SNMP and streaming telemetry with Prometheus
- Building API gateways with Python leveraging Fast API
- Integration of alerting with Slack and your own APIs
- … and many more
Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data centre networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depth and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.
Disclaimer
This is not a typical blogpost, where we are drilling in the details of technologies, their application and configuration. We will do that, for sure, but a little bit later. Today we will talk about my personal experience of developing myself into being a software development, what motivates me, what challenges and benefits it brings.
First of all, Why Software Development?
Since 2007 I started career as a network/system administrator in a small retail business in Belarus thanks to my dad, who helped me to arrange a part-time job when I was a student. I started leaning network technologies from scratches and even managed to work with IPX, before it fully disappeared in favor of TCP/IP. So I was learning network technologies by doing it: configure some routers/switches, Windows server and more. I was not always aware of what exactly I’m doing and why, was just driven by examples I was about to found Internet. I didn’t like it. I mean, I liked the job, but I didn’t like the fact I was not able to understand on all the levels what’s going on in networking and how hosts (server, switches, routers) do communicate between each others. So I started looking around, are there any foundational learning about modern networking. Despite I was learning that time for my bachelor in telecommunication, I was learning too foundational things such as Fourier analysis and Optical Windows. Although, all these things are very important and genuinely made a lot of fun, they didn’t tell me much about what IP address and how IP packets are encapsulated in Ethernet packets and how all these communication between devices in networks works. Apparently, I studied that later in my university, but as I needed it for my job before, I somehow landed at pursuing CCNA from Cisco. That was a real breakthrough for me that time, as I got enormous amount of practical knowledge to get my career of the ground. Since the second attempt I’ve landed a job in the biggest and I believe the most successful Service Provider in my home country. Funny enough, in both cases my manager was the same person, although the first time he shamelessly literally put my CV in shredder…
Never the less, this curiosity pushed me further to learn more and practice more in the networking field going the entire Cisco ladder up to 2x CCIE in Routing and Switching and Service Provider. I changed a few companies and even countries looking for bigger challenges and somewhat found myself in a sadness that networking doesn’t make any fun anymore. It became lame, everything is the same, something is bigger, something is smaller; but, ultimately, OSI stack is OSI stack everywhere. I elaborated understanding that all these 6x CCIE from Cisco or any other expert level exam from any other vendor is nothing more than money making on our, network engineers, ambitions and desires. When you get CCIE, that is cool, that feels like, probably, climbing Everest. But to achieve next CCIE you put about 30% energy compared to the first one and get 10% of satisfaction. I don’t discourage folks to pursuing CCIEs, its your own choice, although these days I think this is really waste of time. That’s why I let my CCIE expiring without going that manic way of endlessly refreshing them.
So, if networking became lame, what could be not? That’s how I started looking further, where shall I go now? Where can I benefit from networking background without being bored? That’s how I appeared in network automation world: here we have whole loads of problems to solve, where the problem is, actually, the networking itself. However, the way we solve them is different to traditional CLI jockeys. And yes, my dudes, netmiko is not network automation. I may hurt your feeling, but simple scripting is just a learning path towards network automation, but not the destination. Using netmiko is like replacing an operator in the admin division with a fast typing robot, which really presses keys on the keyboard. So even back in 2016 I started looking in NETCONF/YANG, trying to understand the whole new world of data structuring with XML and how it works. I started using Ansible for network managing back in 2016 before it became a mainstream continuously learning and applying my network background and learning new things: data types (XML, YAML, JSON), protocols (NETCONF, REST API) and tools (Ansible, Python, Ansible Tower). Within a few years I solidified understanding that it is, pretty much like in networking, everything is more or less the same: you have a problem, you got a toolkit, you solve it. It is not about Ansible vs Python, like you can find in endless Reddit or LinkedIn threads; it is about right tool for a specific job and using it.
However, this time the outcome was different: it didn’t became lame. In the networking industry we are limited by functionality supported in specific network operating systems: whether it is Cisco IOS, IOS XR, JUNOS, etc; it doesn’t really matter, as all of them have certain RFC-based functionality and some proprietary stuff. You don’t have freedom at the end of the day. With network automation, which is a subset of software development, this is really different. You are limited at the end of the day only by our own imagination: you have endless possibilities to do things differently, which is inspiring and daunting simultaneously. it is inspiring because you can do things in your own way everywhere still smashing goals of delivering sustainable IT/network infrastructure. It is daunting, though, as soon you realize there are certain things you need to be aware of, such as time complexity O(t), space complexity, and many others; if you don’t care about them you don’t write develop efficient network automation solutions. This is OK for small infrastructure with a dozen (even few dozen of nodes), but it is NOT OK for networks with thousands of devices.
This is, pretty much like years ago, I started looking in foundation how do to develop and delivery the efficient software solution to solve IT/network infrastructure needs, but this time keeping the broad view on the users and their business needs. That’s how I started studying proper Computer Science and bringing those knowledge into network automation. On the other hand, this opened for me the entire new world of proper software development including possibility of writing my own distributed applications, which requires network connectivity to operate. It started as hobby project, as usual, and so far continues in such a way. But finally, as I started writing application entirely myself including their network connectivity stack, I finally understand why I needed that network background. Although I’m still lacking many traditional computer science skills and methodologies, which I will learn shortly anyway, I knows in-depths what’s going in networks and how I should build my TCP/IP (or QUIC) stack in my applications to ensure it works properly, and not like the this post from my friend Pavel Odintsov.
Why is C now?
With having discussed how I transitioned from pure network engineering to proper software development, let’s talk about choice of languages. As we are talking from (computer) science perspective, we shall be driven by facts. Here is an interesting statistics about popularity of programming languages (how it was collected, that’s another question but typically Statista is a credible source):
If we put aside things such as JavaScript, HTML, SQL and so on, our top three languages are:
- Python
- Java
- C#, C++, C
We put mentioned things asides because you don’t really use SQL alone, but other you use SQL syntax from other languages, such as Python, Java and C; whilst JavaScript/HTML/CSS, etc are used solely for the development of web pages / web development.
Each of the mentioned languages have its own strengths and weaknesses, but more than that, each language has the main area, where it shines the best. Python, for sure, shines in a huge amount of tasks from simple scripts till full blown websites with dynamic content generation to managing tens and hundreds of devices in network automation. However, where Python stops working well is where you need high performance (e.g., processing millions of packets per seconds, very quick lookup in multiple gigabytes of data, processing of thousands of requests per second, etc, etc).
When you need that kind performance, the only programming language, which can give it you is C (and C++). And at this stage in my career I’m very interested in high performance, preferably, the highest speed and performance I can get, so I’ve put my eyes on C/C++. I’m not touching C++ yet, and focusing solely on C for a few reason:
- C is a procedural language, so you need to write everything in functions.
- C++ supports both procedural-oriented development and object-oriented development (which is pretty much in Python, so I’m aiming to leverage my Python skills).
Now, besides the speed and performance, which I’m getting in C, there Linux, the base platform for all sane application these days, is written in C and, therefore, it is naturally to implement all system calls to kernel using C. And this is what I need now, as I’m doing right now development of a distributed systems, which heavily relies on network communications in different forms:
- TCP/IP unicast communication in a standard client-server fashion with multiple independent servers serving different applications.
- UDP/IP multicast communication with multiple different channels simultaneously streaming different information
All these things are implemented via network sockets, which are available in C in via system calls to kernel.
By any mean I say that Java is bad or that C/C++ is better, but after some analysis I came to conclusion that C/C++ match my needs more.
Why not Rust?
Rust is an interesting beast. There is a lot of talks about it these days and there are some projects appearing, which are written in Rust in various areas, including infrastructure management (e.g., network automation, Kubernetes, etc). I decided to put it aside for a bit simply because the market for C/C++ is much bigger these days than for Rust; however, at some point in future, I well may take a look into it as well.
Why not Go?
Go is used in the same domain as Python and has similar performance limitations. As such, I decided not to waste my time on it these days, as all things it is possible to do in go, it is possible to do in Python, but much easier due to huge amount of libraries and community available.
How difficult is to make transition from Python to C?
First of all, I would say, I haven’t finalized my transition from Python to C and, moreover, it is not a transition, it is rather learning new paradigm. C is very different to Python in majority of aspects (to be specific, I’m talking about C, not C++ here, which I’m not yet very familiar with). I won’t bother you with a long list of feature by feature comparison, you surely can find a lot of them in Internet already. However, I want to highlight a few aspects which are important, if you start learning C after Python like I did.
| Category | Python | C |
|---|---|---|
| Procedural/object-oriented | Procedural and object-oriented | Procedural |
| Memory management | Dynamic (automatic) | Static (manual) |
| Types | Dynamic | Static |
| Interpreted/compiled | Interpreted | Compiled |
| 3rd-party libraries | A lot available | A lot available |
| Community/availability of documentation/examples | Huge | Huge |
When I was writing my second lab in C (first after “hello world”), I felt huge discomfort, “Where are my lists? Where are my dictionaries?” All these beautiful things I’m heavily using a lot, they are simply not existing in C. You have to figure out yourself, how to replace them, and this is where you have to be very creative. Even strings, they don’t exist in C; you have an array of chars, and you need to know that it shall be one character longer than your string as you must terminate it with the specific character ‘\0’. Oh yes, by the way, you cannot have any arbitrary long string, as you need to specify in advance how long your array shall be and moreover, you also need to ensure that you clean everything what C has in memory addresses associated with this array of chars using “memset()“, because if you fail to do so you will have bloody tears from hours of troubleshooting you app, like I had… still having regularly. So what I’m having in Python done in 1-2 strings, can be easily 10-20 strings in C.
Another my favorite, check out this snippet:
2
3
4
5
6
7
8
9
! Some important stuff
!
int64_t sockfd;
if ((sockfd = socket(AF_INET, SOCK_STRAM, 0)) < 0)
{
perror("Cannot create socket: ")
return 1;
}
First of all, I declare a variable called sockfd, which will be of signed 64-bit integer. I need to do it outside the if-statement as all the variables in C are strictly scoped, and I need to declare it outside the if-statment I want it to be available for me after if statement. But what a beauty comes next: in one if-statement I call function socket() with some parameters, assign this value to sockfd and then compare it with 0 to determine if I have an error indicated by value -1.
And there are many further similar things. However, once you spent some time (say 1-2 months of regular practicing without Copilot) with C, I’m start getting it. It is important to keep Copilot shut, otherwise you won’t learn things, sadly. I use a lot it for my daily job and it does boosts productivity, but for learning and understanding this is truly bad.
You may have ready by this point and started thinking, “Gosh, do I really need all this complicated things? Screw that, I would go for Java or Go or just stick further to Python”. And this is a reasonable answer. Maybe you don’t need it all, if you don’t need the topmost speed and performance. But if you do need, then C is right here. Once you start understanding it more, you starts picking up more advanced things, which as structs (hello Python data classes) or trie (hello Python, you don’t have such cool thing, really) and you start focusing more on benefits and performance gains rather than syntax complexity. And there is something else…
Libraries and Communities
One of major killers why I continue using Python over Go is because Python has tremendous amount of useful libraries and community support, whereas Go is just fraction of it. Whilst I mentioned above that you have to do many things in C yourself, it doesn’t mean that there are no useful 3rd party libraries, which you can use. In fact, there are a lot and you definitely can benefit from them. For instance, I’ve used Redis library for C, called hiredis in order to connect my network apps to Redis and got really great user experience as this library is both simple and robust. And such as with many other libraries. You are not on your own with C.
… So, is that really a transition from Python to C?
As said, it is not really a migration, rather than expanding my horizons, what I can do. Although C is very much different from Python, as I’m working with Python on a quite a deep level, I’m seeing, so to say, ears of C here and there. Moreover, many things which were also for me originally weird in Python, such as coping vs referencing lists ad dictionaries now makes perfect thanks, as well as understanding why set has O(1) lookup complexity, whilst list has O(n), but why in set you cannot have duplicated items, whilst you can perfectly fine have them in lists.
A few more words
Whilst I was and am digging into C and fundamental software development, I even more understand how important is that to get the basics of software development right in order you can progress in this field. I’ve done recently revision of our network automation trainings, especially Zero-to-Hero Network Automation Training, and was quite pleased that we’ve built it right from the very beginning teaching you with all the foundations before jumping to writing the code. Get your foundations in network automation and network software development right with us:
Summary
Ultimately this is about having the right tool for a job. I continue using Python in many projects, finding new libraries and using more advanced solutions, which speed up performance and capability of my tools. At the same time, I consider C is an another important and useful tool to. As I called this blogpost part 00, that suggests I have ideas about posting some things about C in future. Not that I’m planning to write in-depth stuff like in Python, but more than a notes for myself and you, my friend, like we started back in days with notes about Cisco and Nokia devices. Take care and good bye!
Need Help? Contract Us
If you need a trusted and experienced partner to automate your network and IT infrastructure, get in touch with us.
P.S.
If you have further questions or you need help with your networks, we are happy to assist you, just send us a message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk