Automation 12. Automated EVPN Customer Deployment with Ansible, NETCONF, and NetBox. 6WIND version.

Anton Karneliuk

2 years ago

Hello my friend,

We’ve been preparing this blogpost for quite a while, but for various reasons it was put on the back burner. Now we finally are bringing this back to light. We’ll go over a practical use case of automation of 6WIND configuration with Ansible and NetBox relying NETCONF.


1
2
3
4
5
No part of this blogpost could be reproduced, stored in a 

retrieval system, or transmitted in any form or by any 

means, electronic, mechanical or photocopying, recording, 

or otherwise, for commercial purposes without the 

prior permission of the author.

Do I Need to Automate Everything?

The answer is, as usual: it depends. With our passion to automation, we would say: yes, definitely you should automate everything. But this is possible, only if you have unlimited resources (time, money, people). In reality, all the resources are limited and, moreover, may be even scarce. In such a case you would need to choose, where would you obtain the biggest leverage from automation. For example, some tasks are more frequent or time consuming than others. Clearly they are to be automated.

How to find them? Join our automation training and you will find that out!

We offer the following training programs:

During these trainings you will learn the following topics:

Success and failure strategies to build the automation tools.
Principles of software developments and tools for that.
Data encoding (free-text, XML, JSON, YAML, Protobuf)
Model-driven network automation with YANG, NETCONF, RESTCONF, GNMI.
Full configuration templating with Jinja2 based on the source of truth (NetBox).
Best programming languages (Python, Bash) for developing automation, configuration management tools (Ansible) and automation frameworks (Nornir).
Network automation infrastructure (Linux, Linux networking, KVM, Docker).
Orchestration of automation workflows with AWX and its integration with NetBox, GitHub, as well as custom execution environments for better scalability.

Moreover, we put all mentions technologies in the context of real use cases, which our team has solved and are solving in various projects in the service providers, enterprise and data centre networks and systems across the Europe and USA. That gives you opportunity to ask questions to understand the solutions in-depts and have discussions about your own projects. And on top of that, each technology is provided with online demos and labs to master your skills thoroughly. Such a mixture creates a unique learning environment, which all students value so much. Join us and unleash your potential.

Start your automation training today.

Brief Description

Following approach mentioned above, what may be a task or tasks, which are used the most for authentication in any Service Provider network?

By service provider we mean a company, which provides any sort of connectivity and/or IT infrastructure services to other companies (e.g., communication/Internet service providers (CSP/ISP), hosting providers, cloud providers, etc).

If a Service Provider is successful, the most widely used operation is an onboarding of a new users or a modification of the configuration for the existing one. Time to time, it would also need to remove the configuration, when customers are leaving. These three operations would be by far most widely used than any other operations, which change configuration at the network devices. As such, we would focus on them. To be precise, we will focus on the customer on-boarding.

What does customer onboarding mean? It means that the service for a new customer shall be provisioned in a network. However, before we can provision the service, we need to know how it shall look like. Therefore, the customer’s on-boarding will consist from a few steps:

Documentation of the customer configuration in some database.
Validation of the documented information.
Preparing configuration for a deployment
Rollout of the configuration in a live network.
Post-rollout validation, if the desired state is achieved.

At each of this step there are various tools and techniques existing.

Enrol to our zero-to-hero network automation training to get practical knowledge how to efficiently perform each of these tasks.

For the purpose of this blogpost, we’ll focus on the 1st, 4th, and bit of the 5th steps.

However, the story is incomplete until we put it into the proper context. Our context is a service provider, which operates 6WIND-based network, both inside data centres and for DCI purpose. The service provider implements multi-tenant architecture with EVPN/VXLAN providing data and control planes and with tenant’s separation implemented via placement customers in different VRFs and terminating all their servers in those VRFs. That’s fairly standard data centre deployment these days both in the could providers and in enterprise worlds. Let’s see how that can be automated.

Lab Setup

The original scenario is quite a big one and is worth of having a standalone blogpost. For the purpose of the lab, we will use the one we created in one of the previous 6WIND blogposts:

Here is what we run:

6WIND version 3.*

Other network devices are out of scope for this blogpost.

But in scope for our network automation training, which you can start today.

We also have a management host running Debian 11.1 with:

Python 3.9
Ansible 5.*
Latest version of pyang

On top of that, as a source of truth for the network we are using:

NetBox 3.0

Let’s get started.

Scenario Description

Based on the high-level description above, here is the list of questions we are to cover in the blogpost

What are the parameters we need to have to document the customer in multi-tenant EVPN setup?
How can them be documented in the NetBox?
How to provision new customer in 6WIND with Ansible via NETCONF?

Enrol to our Zero-to-Hero Network Automation Training to master practical skills in NETCONF/YANG.

Step #1. Approach for a Traffic Separation in a Multi-Tenant Service Provider

First thing first, how is the traffic of one traffic is separated from another customer’s traffic? As you may know, one the most popular and widely used methods of serration is a VLAN tagging. It works as this:

For each customer its own VLAN is allocated
All the packets belonging to a customer are tagged with that VLAN.
Traffic can cross the VLAN boundary only via routing, as each is typically associated with a separate subnet.

Strictly speaking, it is possible to stitch two separate VLANs together via VLAN-tag rewrite, but we will put this outside of our discussion.

Such separation works well, if all you need is to interconnect multiple customer’s endpoint and provide them L2 connectivity in a small domain. What if the customer requires multiple different subnets to be interconnected?

Here the routing comes to the stage. But it may be, that two different customers have the same IP address schema and, therefore, overlapping IP address. The solution to that is VRF, where we create a virtual router for each customer. There are two 2 key parameters associated with each VRF:

Route Distinguisher, which is a unique 64-bit identifier, which prepends the original customer IP address and, therefore, makes a customer route unique.
Route Target, which is also a unique 64-bit identifier, which is used upon signalling customer network via the service provider network to identify belonging the routes to a particular VRF and, therefore, a particular tenant.

The next piece is to span multiple distant customer’s locations into a single instance, so that a customer can have L2 or L3 connectivity service between sites, depending on their needs. There are multiple ways, how this can be achieved these days:

BGP IP/MPLS VPN, which is known already for a few decades and still playing a crucial role in building ISPs and CSPs across the globe.
VPLS or VPWS, which are also MPLS-based VPNs, which provide L2 connectivity over CSP/ISP networks.
EVPN with VXLAN or MPLS data places, which provides you combined L2/L3 VPN services.

There are some other types of data planes possible, such as GENEVE, but we will put it aside, at least for the time being.

EVPN/VXLAN is a clear leader in a data centre segment for a simple reason: it is much easier (hence, cheaper) to build router or switch, which can produce a high-performant VXLAN encapsulation/decapsulation than a device doing similar operation in regards of MPLS. That’s why, started in data centres, EVPN/VXLAN spread its influence into Campus networking (same EVPN/VXLAN fabric as in data centres), SD-WAN (EVPN/VXLAN over IPSec) and even service providers, where capacity is more important than MPLS Traffic Engineering capabilities.

As such, the last piece for the information we need to document and then to configure for the customer’s onboarding is the VNI ID, which is a unique identifier used to separate customers traffic in the routed domain.

Bringing all the pieces together, you could imagine the following picture:

The described flow was very much simplified, and there are a lot of details about EVPN/VXLAN design and operation. Reach us out to learn more.

Step #2. Documentation of a Customer Service in NetBox

Summing the previous step, we need to document the following parameters for each customer:

Parameter	Description
tenant id	Customer’s unique ID in the Service Provider network
interface id	Customer’s facing interface per each provider’s network device
vlan id	If customer’s connectivity if done in SVI/BVI, the associated VLAN ID
vrf name	the name of the VRF associated with the customer
rt	Route Targets associated with customer routes
rd	Route Distinguisher associated with customer routes
vni id	the ID of the customer’s VXLAN

Now, let’s start documenting these details.

Step #2.1. Create a Tenant in NetBox

In NetBox, navigate choose “Tenant” tab from “Organisations” category. Press “+” to add customer. In the opened window, provide a customer name.

Step #2.2. Create a Route Target in NetBox

In NetBox, naviaged to “Route Targets” tab in “IPAM” category. Press “+” to add a new Route Target. In the opened window, provide its value and map the created tenant.

Step #2.3. Create a VRF in NetBox

In NetBox, naviaged to “VRF” tab in “IPAM” category. Press “+” to add a new VRF. In the opened window, provide its name, create a Route Distinguisher (RD), associate the created Route Targets and map it to the tenant.

Step #2.4. Create a Tenant’s VLAN in NetBox

In NetBox, naviaged to “VLAN” tab in “IPAM” category. Press “+” to add a new VLAN. In the opened window, provide its ID, name, create a Route Distinguisher (RD), and map it to the tenant.

Step #2.5. Create a Tenant’s Prefix in NetBox

In NetBox, naviaged to “Prefix” tab in “IPAM” category. Press “+” to add a new Prefix. In the opened window, provide the IP prefix and prefix length, map it to VRF, associate with the created IP Prefix and the tenant.

Step #2.6 Add to the PE Device Tenant’s Interfaces

The last in the device’s modelling is to use create tenant’s interfaces. In the vendor we use in this blogpost, which is 6WIND, the following interfaces are needed (some key elements are highlighted as well):


1
2
3
4
5
6
7
8
+--bridgeXXX

   +--ip_address

   +--vrrp

   +--children:

     +--vxlanYYY

     |  +--vxlan_id

     +--physicalZZZ

        +--vlan_id

First, let’s create a bridge interface. To do that, naviaged to “Interface” tab of the chosen device in the “Devices” category. Press “+ Add Components” to add a new Interface.

In the opened window, provide the interface’s name and type, as well as assign a corresponding VLAN to it.

In the similar way, create also physical interface and VXLAN interface. The key difference for them is to add a parent interface, which shall be the created bridge:

Finally, add the IP address (in fact, 2 IP addresses: virtual IP and interface IP) to a bridge interface:

Adding IP Address to a New Interface in NetBox

NetBox is a truly powerful tool for modelling. At our Network Automation Training you will master it in-depth.

Step #3. Create Customer’s Configuration at 6WIND with Ansible and NETCONF/YANG

In one of the previous blogposts about 6WIND, we have shown some details about its NETCONF implementation and shared a basic playbook to collect YANG modules. Now we will extend it to create a configuration.

The components for our automation:

NETCONF is one of the most popular and, therefore, widely used protocols for model-driven network automation these days. It allows to configure device in a format of multiple key-value pairs leveraging YANG and, therefore, avoids the hassle of generating the flat configuration files as well as problems with parsing semi-formatted text.
NetBox is an open source tool for data modelling. You have seen above its UI side, whereas the key benefit of NetBox is that it allows to retrieve all the date we have provided via REST API in a programmable way.
Ansible is an open source configuration management system, which is the most popular tool these days for such type of actions. It has a huge variety of built-in plugins, including plugins for NetBox and NETCONF, which we use in this automation

Putting that together, we create the following structure for our Ansible automation (it is very simplified):


1
2
3
4
5
6
7
8
9
10
11
12
13
+--configurator.yaml

+--roles

   +--300_prepare_vars

   |  +--tasks

   |  | +--main.yaml

   |  +--templates

   |     +--interface_vrf_mapping.j2

   +--301_push_config

      +--tasks

      |  +--change.yaml

      |  +--main.yaml

      +--templates

         +--{... multiple config templates}

Enrol to our Network Automation Training to master Ansible skills to be able create carrier-grade automation.

Step #4. Verify the Automation’s Operation

We have created the VRF with scalability in mind. Therefore, we have introduced two variables, which needs to be filled in:

target_vrf is used to provide the name of the customer’s VRF, which we are to create.
tags customer or infrastructure, which define if you want to onboard the new customer or to provision a core functionality.

Let’s run the provisioning for the network device, where we have added the tenant’s interfaces:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
# ansible-playbook configurator.yaml --limit vr01-a --tags customer -e 'target_vrf=imba-vrf'

&#91;DEPRECATION WARNING]: &#91;defaults]callback_whitelist option, normalizing names to new standard, use callbacks_enabled instead. This feature will be removed 

from ansible-core in version 2.15. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

&#91;WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details

&#91;WARNING]: An error occurred while calling ansible.utils.display.initialize_locale (unsupported locale setting). This may result in incorrectly calculated

text widths that can cause Display to print incorrect line lengths



PLAY &#91;CONFIGURATION OF 6WIND ROUTER] **************************************************************************************************************************



TASK &#91;Gathering Facts] ****************************************************************************************************************************************

Sunday 06 March 2022  20:57:59 +0000 (0:00:00.023)       0:00:00.023 ********** 

ok: &#91;vr01-a]



TASK &#91;100_collect_netbox : 10. POLL SITE DATA] ****************************************************************************************************************

Sunday 06 March 2022  20:58:01 +0000 (0:00:02.460)       0:00:02.483 ********** 

ok: &#91;vr01-a]



TASK &#91;100_collect_netbox : 20. POLL VRFS DATA] ****************************************************************************************************************

Sunday 06 March 2022  20:58:03 +0000 (0:00:01.548)       0:00:04.031 ********** 

ok: &#91;vr01-a]



TASK &#91;300_prepare_vars : 300.10. MAP INTERFACES TO VRFS] ******************************************************************************************************

Sunday 06 March 2022  20:58:04 +0000 (0:00:01.405)       0:00:05.437 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.10. CREATE TEMPORARY DIRECTORY] ***************************************************************************************************

Sunday 06 March 2022  20:58:05 +0000 (0:00:00.967)       0:00:06.405 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20. RUN CONFIGURATION LOOP] *******************************************************************************************************

Sunday 06 March 2022  20:58:06 +0000 (0:00:00.615)       0:00:07.021 ********** 

included: /root/6wind-automation/roles/301_push_config/tasks/change.yaml for vr01-a =&gt; (item=vrf)

included: /root/6wind-automation/roles/301_push_config/tasks/change.yaml for vr01-a =&gt; (item=vrrp)

included: /root/6wind-automation/roles/301_push_config/tasks/change.yaml for vr01-a =&gt; (item=interface)

included: /root/6wind-automation/roles/301_push_config/tasks/change.yaml for vr01-a =&gt; (item=bgp)



TASK &#91;301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main vrf] ******************************************************************************

Sunday 06 March 2022  20:58:06 +0000 (0:00:00.118)       0:00:07.140 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.20. PRE-CHANGE // SAVING CONFIG] ***********************************************************************************************

Sunday 06 March 2022  20:58:09 +0000 (0:00:02.924)       0:00:10.065 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY] ************************************************************************************

Sunday 06 March 2022  20:58:10 +0000 (0:00:01.071)       0:00:11.137 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main vrf] ****************************************************************************

Sunday 06 March 2022  20:58:11 +0000 (0:00:00.620)       0:00:11.757 ********** 

changed: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main vrf] *****************************************************************************

Sunday 06 March 2022  20:58:13 +0000 (0:00:02.730)       0:00:14.487 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.60. POST-CHANGE // SAVING CONFIG] **********************************************************************************************

Sunday 06 March 2022  20:58:16 +0000 (0:00:02.655)       0:00:17.143 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.70. COMPARING PRE AND POST CHANGE] *********************************************************************************************

Sunday 06 March 2022  20:58:17 +0000 (0:00:00.685)       0:00:17.828 ********** 

--- before

+++ after

@@ -1,4 +1,6 @@

 {

+    "data.config.vrf.name": "imba-vrf",

+    "data.config&#91;'@xmlns']": "urn:6wind:vrouter",

     "data&#91;'@xmlns']": "urn:ietf:params:xml:ns:netconf:base:1.0",

     "data&#91;'@xmlns:nc']": "urn:ietf:params:xml:ns:netconf:base:1.0"

 }



changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.80. SAVING DIFF] ***************************************************************************************************************

Sunday 06 March 2022  20:58:17 +0000 (0:00:00.115)       0:00:17.944 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main vrrp] *****************************************************************************

Sunday 06 March 2022  20:58:17 +0000 (0:00:00.634)       0:00:18.578 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.20. PRE-CHANGE // SAVING CONFIG] ***********************************************************************************************

Sunday 06 March 2022  20:58:20 +0000 (0:00:02.430)       0:00:21.009 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY] ************************************************************************************

Sunday 06 March 2022  20:58:21 +0000 (0:00:00.665)       0:00:21.674 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main vrrp] ***************************************************************************

Sunday 06 March 2022  20:58:21 +0000 (0:00:00.716)       0:00:22.391 ********** 

changed: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main vrrp] ****************************************************************************

Sunday 06 March 2022  20:58:23 +0000 (0:00:01.984)       0:00:24.376 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.60. POST-CHANGE // SAVING CONFIG] **********************************************************************************************

Sunday 06 March 2022  20:58:26 +0000 (0:00:02.608)       0:00:26.984 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.70. COMPARING PRE AND POST CHANGE] *********************************************************************************************

Sunday 06 March 2022  20:58:27 +0000 (0:00:00.641)       0:00:27.625 ********** 

--- before

+++ after

@@ -1,4 +1,28 @@

 {

+    "data.config.vrf.interface.vrrp.enabled": "true",

+    "data.config.vrf.interface.vrrp.mtu": "9000",

+    "data.config.vrf.interface.vrrp.name": "vrrp101",

+    "data.config.vrf.interface.vrrp.preempt": "true",

+    "data.config.vrf.interface.vrrp.priority": "150",

+    "data.config.vrf.interface.vrrp.version": "2",

+    "data.config.vrf.interface.vrrp.vrid": "2",

+    "data.config.vrf.interface.vrrp&#91;'@xmlns']": "urn:6wind:vrouter/vrrp",

+    "data.config.vrf.interface.vrrp&#91;'advertisement-interval']": "1000",

+    "data.config.vrf.interface.vrrp&#91;'garp-delay']": "5",

+    "data.config.vrf.interface.vrrp&#91;'init-state']": "backup",

+    "data.config.vrf.interface.vrrp&#91;'link-interface']": "bridge101",

+    "data.config.vrf.interface.vrrp&#91;'preempt-delay']": "120",

+    "data.config.vrf.interface.vrrp&#91;'track-fast-path']": "true",

+    "data.config.vrf.interface.vrrp&#91;'track-link-interface']": "true",

+    "data.config.vrf.interface.vrrp&#91;'use-vmac']": "false",

+    "data.config.vrf.interface.vrrp&#91;'virtual-address'].ip": "10.100.11.1/32",

+    "data.config.vrf.interface.vrrp&#91;'vmac-xmit-base']": "false",

+    "data.config.vrf.interface&#91;'@xmlns']": "urn:6wind:vrouter/interface",

+    "data.config.vrf.name": "imba-vrf",

+    "data.config.vrf.vrrp.enabled": "true",

+    "data.config.vrf.vrrp&#91;'@xmlns']": "urn:6wind:vrouter/vrrp",

+    "data.config.vrf.vrrp&#91;'router-id']": "10.100.1.0",

+    "data.config&#91;'@xmlns']": "urn:6wind:vrouter",

     "data&#91;'@xmlns']": "urn:ietf:params:xml:ns:netconf:base:1.0",

     "data&#91;'@xmlns:nc']": "urn:ietf:params:xml:ns:netconf:base:1.0"

 }



changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.80. SAVING DIFF] ***************************************************************************************************************

Sunday 06 March 2022  20:58:27 +0000 (0:00:00.117)       0:00:27.743 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main interface] ************************************************************************

Sunday 06 March 2022  20:58:27 +0000 (0:00:00.649)       0:00:28.392 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.20. PRE-CHANGE // SAVING CONFIG] ***********************************************************************************************

Sunday 06 March 2022  20:58:30 +0000 (0:00:02.569)       0:00:30.961 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY] ************************************************************************************

Sunday 06 March 2022  20:58:30 +0000 (0:00:00.607)       0:00:31.569 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main interface] **********************************************************************

Sunday 06 March 2022  20:58:31 +0000 (0:00:00.726)       0:00:32.295 ********** 

changed: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main interface] ***********************************************************************

Sunday 06 March 2022  20:58:34 +0000 (0:00:02.384)       0:00:34.680 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.60. POST-CHANGE // SAVING CONFIG] **********************************************************************************************

Sunday 06 March 2022  20:58:36 +0000 (0:00:02.541)       0:00:37.221 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.70. COMPARING PRE AND POST CHANGE] *********************************************************************************************

Sunday 06 March 2022  20:58:37 +0000 (0:00:00.640)       0:00:37.861 ********** 

--- before

+++ after

@@ -1,4 +1,22 @@

 {

+    "data.config.vrf.interface.bridge.enabled": "true",

+    "data.config.vrf.interface.bridge.ipv4.address.ip": "10.100.11.2/24",

+    "data.config.vrf.interface.bridge.ipv4.enabled": "true",

+    "data.config.vrf.interface.bridge.ipv6.enabled": "true",

+    "data.config.vrf.interface.bridge.mtu": "9000",

+    "data.config.vrf.interface.bridge.name": "bridge101",

+    "data.config.vrf.interface.bridge&#91;'@xmlns']": "urn:6wind:vrouter/bridge",

+    "data.config.vrf.interface.bridge&#91;'link-interface']&#91;0].learning": "true",

+    "data.config.vrf.interface.bridge&#91;'link-interface']&#91;0].slave": "lag0.101",

+    "data.config.vrf.interface.bridge&#91;'link-interface']&#91;1].learning": "false",

+    "data.config.vrf.interface.bridge&#91;'link-interface']&#91;1].slave": "vxlan101",

+    "data.config.vrf.interface.vlan.enabled": "true",

+    "data.config.vrf.interface.vlan.mtu": "9000",

+    "data.config.vrf.interface.vlan.name": "lag0.101",

+    "data.config.vrf.interface.vlan&#91;'@xmlns']": "urn:6wind:vrouter/vlan",

+    "data.config.vrf.interface.vlan&#91;'link-interface']": "lag0",

+    "data.config.vrf.interface.vlan&#91;'link-vrf']": "main",

+    "data.config.vrf.interface.vlan&#91;'vlan-id']": "101",

     "data.config.vrf.interface.vrrp.enabled": "true",

     "data.config.vrf.interface.vrrp.mtu": "9000",

     "data.config.vrf.interface.vrrp.name": "vrrp101",

@@ -17,6 +35,13 @@

     "data.config.vrf.interface.vrrp&#91;'use-vmac']": "false",

     "data.config.vrf.interface.vrrp&#91;'virtual-address'].ip": "10.100.11.1/32",

     "data.config.vrf.interface.vrrp&#91;'vmac-xmit-base']": "false",

+    "data.config.vrf.interface.vxlan.enabled": "true",

+    "data.config.vrf.interface.vxlan.mtu": "9000",

+    "data.config.vrf.interface.vxlan.name": "vxlan101",

+    "data.config.vrf.interface.vxlan.vni": "101",

+    "data.config.vrf.interface.vxlan&#91;'@xmlns']": "urn:6wind:vrouter/vxlan",

+    "data.config.vrf.interface.vxlan&#91;'link-interface']": "loopback0",

+    "data.config.vrf.interface.vxlan&#91;'link-vrf']": "main",

     "data.config.vrf.interface&#91;'@xmlns']": "urn:6wind:vrouter/interface",

     "data.config.vrf.name": "imba-vrf",

     "data.config&#91;'@xmlns']": "urn:6wind:vrouter",



changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.80. SAVING DIFF] ***************************************************************************************************************

Sunday 06 March 2022  20:58:37 +0000 (0:00:00.119)       0:00:37.981 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main bgp] ******************************************************************************

Sunday 06 March 2022  20:58:38 +0000 (0:00:00.655)       0:00:38.637 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.20. PRE-CHANGE // SAVING CONFIG] ***********************************************************************************************

Sunday 06 March 2022  20:58:40 +0000 (0:00:02.539)       0:00:41.176 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY] ************************************************************************************

Sunday 06 March 2022  20:58:41 +0000 (0:00:00.629)       0:00:41.805 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main bgp] ****************************************************************************

Sunday 06 March 2022  20:58:41 +0000 (0:00:00.654)       0:00:42.460 ********** 

changed: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main bgp] *****************************************************************************

Sunday 06 March 2022  20:58:44 +0000 (0:00:02.528)       0:00:44.988 ********** 

ok: &#91;vr01-a]



TASK &#91;301_push_config : 301.20.60. POST-CHANGE // SAVING CONFIG] **********************************************************************************************

Sunday 06 March 2022  20:58:46 +0000 (0:00:02.555)       0:00:47.543 ********** 

changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.70. COMPARING PRE AND POST CHANGE] *********************************************************************************************

Sunday 06 March 2022  20:58:47 +0000 (0:00:00.641)       0:00:48.185 ********** 

--- before

+++ after

@@ -1,4 +1,20 @@

 {

+    "data.config.vrf.name": "imba-vrf",

+    "data.config.vrf.routing.bgp.as": "65000",

+    "data.config.vrf.routing.bgp.enabled": "true",

+    "data.config.vrf.routing.bgp&#91;'@xmlns']": "urn:6wind:vrouter/bgp",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'ipv4-unicast'].redistribute.protocol": "connected",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].advertise": "ipv4-unicast",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].enabled": "true",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].export&#91;'route-distinguisher']": "65000:100",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].export&#91;'route-target']": "65000:100",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].flooding": "head-end-replication",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn'].import&#91;'route-target']": "65000:100",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn']&#91;'advertise-all-vni']": "false",

+    "data.config.vrf.routing.bgp&#91;'address-family']&#91;'l2vpn-evpn']&#91;'auto-route-target']": "disabled",

+    "data.config.vrf.routing.bgp&#91;'router-id']": "10.100.1.0",

+    "data.config.vrf.routing&#91;'@xmlns']": "urn:6wind:vrouter/routing",

+    "data.config&#91;'@xmlns']": "urn:6wind:vrouter",

     "data&#91;'@xmlns']": "urn:ietf:params:xml:ns:netconf:base:1.0",

     "data&#91;'@xmlns:nc']": "urn:ietf:params:xml:ns:netconf:base:1.0"

 }



changed: &#91;vr01-a -&gt; localhost]



TASK &#91;301_push_config : 301.20.80. SAVING DIFF] ***************************************************************************************************************

Sunday 06 March 2022  20:58:47 +0000 (0:00:00.120)       0:00:48.306 ********** 

changed: &#91;vr01-a -&gt; localhost]



PLAY RECAP ****************************************************************************************************************************************************

vr01-a                     : ok=41   changed=25   unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   



Sunday 06 March 2022  20:58:48 +0000 (0:00:00.811)       0:00:49.118 ********** 

=============================================================================== 

301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main vrf ------------------------------------------------------------------------------ 2.93s

301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main vrf ---------------------------------------------------------------------------- 2.73s

301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main vrf ----------------------------------------------------------------------------- 2.66s

301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main vrrp ---------------------------------------------------------------------------- 2.61s

301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main interface ------------------------------------------------------------------------ 2.57s

301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main bgp ----------------------------------------------------------------------------- 2.56s

301_push_config : 301.20.50. POST-CHANGE // COLLECT CONFIG OF non_main interface ----------------------------------------------------------------------- 2.54s

301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main bgp ------------------------------------------------------------------------------ 2.54s

301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main bgp ---------------------------------------------------------------------------- 2.53s

Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------- 2.46s

301_push_config : 301.20.10. PRE-CHANGE // COLLECT CONFIG OF non_main vrrp ----------------------------------------------------------------------------- 2.43s

301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main interface ---------------------------------------------------------------------- 2.38s

301_push_config : 301.20.40. CHANGE // APPLY CONFIGURATION FOR non_main vrrp --------------------------------------------------------------------------- 1.98s

100_collect_netbox : 10. POLL SITE DATA ---------------------------------------------------------------------------------------------------------------- 1.55s

100_collect_netbox : 20. POLL VRFS DATA ---------------------------------------------------------------------------------------------------------------- 1.41s

301_push_config : 301.20.20. PRE-CHANGE // SAVING CONFIG ----------------------------------------------------------------------------------------------- 1.07s

300_prepare_vars : 300.10. MAP INTERFACES TO VRFS ------------------------------------------------------------------------------------------------------ 0.97s

301_push_config : 301.20.80. SAVING DIFF --------------------------------------------------------------------------------------------------------------- 0.81s

301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY ------------------------------------------------------------------------------------ 0.73s

301_push_config : 301.20.30. CHANGE // PREPARE NETCONF MESSAGE BODY ------------------------------------------------------------------------------------ 0.72s

During the execution cycle, the new configuration is being applied to the 6WIND network device. This Ansible automation is created in a way that you can you see the changes, of which exactly new configuration (key-value pairs) is applied. Moreover, this information is stored in the the new directory with the timestamp of execution:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
+--.ops

   +--20220306T205801667717

      +--vr01-a_non_main_bgp_change.json

      +--vr01-a_non_main_interface_change.json

      +--vr01-a_non_main_vrf_change.json

      +--vr01-a_non_main_vrrp_change.json

      +--vr01-a_non_main_bgp_diff.json

      +--vr01-a_non_main_interface_diff.json

      +--vr01-a_non_main_vrf_diff.json

      +--vr01-a_non_main_vrrp_diff.json

      +--vr01-a_non_main_bgp_post_change.json

      +--vr01-a_non_main_interface_post_change.json

      +--vr01-a_non_main_vrf_post_change.json  

      +--vr01-a_non_main_vrrp_post_change.json

      +--vr01-a_non_main_bgp_pre_change.json

      +--vr01-a_non_main_interface_pre_change.json

      +--vr01-a_non_main_vrf_pre_change.json

      +--vr01-a_non_main_vrrp_pre_change.json

Each of these files contains one of four item types:

Configuration of the network device before the change.
Content of the change itself.
Configuration of the network device after the change.
Difference of the configuration between pre- and post- change files.

In our Network Automation Training you will learn how to create such automation yourself.

Final touch if to check how the configuration looks like from the 6WIND router itself:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
pod-013-6wind1&gt; show config / vrf imba-vrf

vrf imba-vrf

    routing

        bgp

            enabled true

            as 65000

            always-compare-med false

            bestpath

                as-path

                    confederation false

                    ignore false

                    ..

                compare-routerid false

                ..

            client-to-client

                reflection true

                ..

            deterministic-med false

            ebgp-connected-route-check true

            fast-external-failover true

            graceful-shutdown false

            listen

                limit 100

                ..

            log-neighbor-changes false

            network-import-check false

            route-reflector-allow-outbound-policy false

            reject-as-sets false

            packet-rw-quantum

                read 10

                write 64

                ..

            router-id 10.100.1.0

            ebgp-requires-policy false

            address-family

                ipv4-unicast

                    enabled true

                    redistribute connected

                    maximum-path

                        ebgp 16

                        ibgp 16

                        equal-cluster-length false

                        ..

                    bgp-distance

                        external-routes 20

                        internal-routes 200

                        local-routes 200

                        ..

                    ..

                l2vpn-evpn

                    enabled true

                    advertise ipv4-unicast

                    advertise-all-vni false

                    auto-route-target disabled

                    flooding head-end-replication

                    export

                        route-target 65000:100

                        route-distinguisher 65000:100

                        ..

                    import

                        route-target 65000:100

                        ..

                    ..

                ..

            ..

        ..

    interface

        vxlan vxlan101

            mtu 9000

            enabled true

            ipv4

                enabled true

                ..

            ipv6

                enabled true

                ..

            vni 101

            link-interface loopback0

            link-vrf main

            learning true

            gbp false

            dst 4789

            src-range

                49152

                65535

                ..

            ..

        bridge bridge101

            mtu 9000

            enabled true

            ipv4

                address 10.100.11.2/24

                enabled true

                ..

            ipv6

                enabled true

                ..

            link-interface lag0.101 learning true

            link-interface vxlan101 learning false

            ..

        vlan lag0.101

            mtu 9000

            enabled true

            ipv4

                enabled true

                ..

            ipv6

                enabled true

                ..

            vlan-id 101

            link-interface lag0

            protocol 802.1q

            link-vrf main

            ..

        vrrp vrrp101

            mtu 9000

            enabled true

            version 2

            link-interface bridge101

            garp-delay 5

            use-vmac false

            vmac-xmit-base false

            vrid 2

            priority 150

            init-state backup

            preempt true

            preempt-delay 120

            advertisement-interval 1000

            track-link-interface true

            track-fast-path true

            virtual-address 10.100.11.1/32

            ..

        ..

    vrrp

        enabled true

        router-id 10.100.1.0

        traps-enabled false

        vrrp-startup-delay 0

        ..

    ..

Examples in GitHub

You can find this and other examples in our GitHub repository.

Lessons Learned

It took us to prepare the proper Ansible automation for two main reasons:

Relations between interfaces (parent-child) was introduced in NetBox fairly recently. Therefore, we have to explore how it works. Besides, it is not possible to map a VRF to an interface directly. It is possible, but only via the IP prefix, what required to create a separate step to do the proper mapping of all interfaces to a VRF.
We were aiming to create a repeatable logic, where we addition of new configuration item (e.g., VRF, interface, etc) shall be straightforward and simple. That forced us to re-engineer the entire playbook twice, before we achieved a desired outcome.

Conclusion

Automation of reputable tasks, such as customers onboarding or decommissioning is a vital step in service providers development to free up network engineering resources and provide a consistent and deterministic quality for customers. Relying leading open source tools, such as Ansible and NetBox, as well as standardised management frameworks, such as NETCONF/YANG, it is possible to achieve it. Take care and good bye.

Need Help? Contract Us

If you need a trusted and experienced partner to automate your network and IT infrastructure, get in touch with us.

P.S.

If you have further questions or you need help with your networks, we are happy to assist you, just send us a message. Also don’t forget to share the article on your social media, if you like it.

BR,

Anton Karneliuk