Hello my friend,
The last time we’ve started the discussion about the documentation of your data centre infrastructure using NetBox and its integration with Ansible to provision network elements, where NetBox plays network modelling role. Today we continue that activity by extending integration of NetBox in order to deploy enabler infrastructure services (DHCP, DNS, FTP and HTTP) as Docker containers in automated fashion using Ansible.
2
3
4
5
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
Brief description
I’ve already used all the hype words several times in order to help searching engines to put this blogpost on top of your search results, if you are looking for automation of your data centre or your network in general. Based on the results of the last article, we found that the NetBox is suitable not only for documentation of your data centre infrastructure, such as locations, racks, devices, ports and IP addresses. But it’s also suitable to be a vendor-agnostic data model of your network, which you can dress in the syntax of any vendor (you have learned about Cumulus Linux, Arista EOS and Cisco IOS XR integration). The latter was achieved by using Ansible to extract proper data (interfaces, IP addresses, connectivity between devices) from NetBox over REST API, fill in proper templates (interfaces, BGP for IPv4/IPv6 unicast and EVPN) and push config to the network functions (leaf and spine switches). That is very good in terms saving your time and making the infrastructure predictable and free of human mistakes, but there was still one step undone.
I’m speaking about the step, when you unpack your switch from its box, plug the power, connect the OOB management interface to the network for first time and boot it. Quite often, you plug it not to the network, but rather to your laptop to connect to the device and perform some initial configuration activities. This not practical at all, because either it is required you to be on-site in the data centre to configure the switch or it is requires the switch to be send it initially to your office for initial configuration and then to site. Both is equally bad, when we think about high scale data centres with hundreds of racks and several hundreds of switches.
I’m absolutely confident that the only working solution in such scenario is zero touch provisioning, where devices are automatically provisioned from the time, when they are connected to the OOB network. And by “provisioned” I mean really fully provisioned with all infrastructure roles. There are some steps necessary to come there and today we’ll do the first step, which is the creation of the automatically creation of the infrastructure with enabler services such as DHCP, DNS, FTP and HTTP. There are numerous of possibilities to deploy such infrastructure, we’ll focus to deploy them as Docker containers (today as a stand-alone, in future in Docker Swarm). Why does it matter? How does it help to the zero touch provisioning? Take a look to the next section.
What are we going to test?
This is how the infrastructure suppose to work:
- The DHCP application based on DHCP ISC inside the 1st Docker container is populated with the information from the NetBox instance to create a pool for OOB IP and static DHCP leases in advance (given you document MAC/IP in NetBox).
- The DNS application based on ISC BIND 9 inside the 2nd Docker container is populated with hostname/IP bindings for forward and reverse zones (both IPv4 and IPv6). Files with zones are generated automatically based on the OOB subnet.
- The FTP application based on VSFTPD inside the 3rd Docker container just host some shared files (like license for Nokia VSR), which can be reachable by respective hosts.
- The HTTP application based on NGINX inside the 4th Docker container has ZTP scripts for Cumulus and Arista, which are generated automatically and filled with relevant information out of NetBox.
In order to create the infrastructure as described above we have rely on Ansible by extending the playbooks we’ve created recently. Ansible is the magic glue, which spans NetBox with Docker to feed containers with necessary inputs via Jinja2 templates.
What is important to outline separately is the base image for the Docker containers. All these applications are running on Linux, hence the base image is Linux. There are various Linux flavours, like RHEL/CentOS or Ubuntu/Debian available. Nevertheless, the base image for the Docker containers within the Data Centre Fabric is Alpine Linux. There is very simple, the Linux image is super small: just about 6 megabytes. In the world, where we used to measure everything in Gygabytes, such size of base Linux image could look unbelievable. But I’m absolutely sure, for micro services this the proper size, cause we can scale them REALLY MASSIVELY.
The ultimate success criteria for this article would be, if all network functions we have (Nokia VSR, Cisco IOS XRv, Arista vEOS or Cumulus VX):
- can boot and get automatically proper IP address based on the planning done in NetBox over DHCP;
- download necessary ZTP scripts or license files from HTTP or FTP server;
- be able to communicate to each other using FQDN with central DNS server;
- be reachable from the managed host to be provisioned using previous Ansible roles;
Are you excited to understand how to create really automated data centre? Fasten your seat belts, we are about to start.
Software version
The following software components are used in this lab.
Management host:
- CentOS 7.5.1804 with python 2.7.5
- Ansible 2.7.0
- Docker-CE 18.09
- Containerized NetBox
Enabler infrastructure:
- Base Linux image for container: Alpine Linux 3.9
- DHCP: ISC DHCP 4.4.1-r2
- DNS: ISC BIND 9.12.3_p4-r2
- FTP: VSFTPD 3.0.3-r6
- HTTP: NGINX 1.14.2-r1
The Data Centre Fabric:
- Nokia SR OS 16.0.R7 [guest VNF]
- Arista EOS 4.21.1.1F [guest VNF]
- Cisco IOS XR 6.5.1 [guest VNF]
- Cumulus Linux 3.7.5 [guest VNF]
More details about Data Centre Fabric you may find in the previous articles.
Topology
In the image below, you can find the management network topology, same as the last time, but this time it is extended with Docker details:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| /\/\/\/\/\/\/\ |
| +-----------------+ +-----------------+ / Docker cloud \ |
| | de-bln-spine-101| (c)karneliuk.com // Data Centre Fabric | de-bln-spine-201| / +------+ \ |
| | (Cisco IOS XRv) | | (Nokia VSR) | / +---+ DHCP | / |
| | Lo0: .1 | | system: .2 | \ | .2+------+ \ |
| | BGP AS: 65000 | | BGP AS: 65000 | / | / |
| +-------+---------+ IP^4: 192.168.1.0/24 +--------+--------+ / | +------+ \ |
| | IPv6: fc00:de:1:ffff::/64 | +------------+ +---+ DNS | / |
| | MgmtEth0/CPU0/0 | MgmtEth0/CPU0/0| Management +----+ .3+------+ \ |
| | .25/:25 | .26/:26 | host |.1 | / |
| | | +------+-----+ | +------+ \ |
| | | | \ +---+ FTP | / |
| | | | ens33\ | .4+------+ \ |
| +-------------------+--------------+---------------------------------+-------------+-------------------+---+ .1 \ | / |
| | | | | :1 / | +------+ \ |
| | | | | \ +---+ HTTP | / |
| | | | | \ .5+------+ / |
| | eth0 | eth0 | Management1 | Management1 \172.17.0.0/16\ |
| | .21/:21 | .22/:22 | .23/:23 | .24/:24 \/\/\/\/\/\/\/ |
| | | | | |
| +------------------+ +---------+--------+ +---------+--------+ +---------+--------+ |
| | de-bln-leaf-111 | | de-bln-leaf-112 | | de-bln-leaf-211 | | de-bln-leaf-212 | |
| | (Cumulus VX) | | (Cumulus VX) | | (Arista vEOS) | | (Arista vEOS) | |
| | lo: .101 | | lo: .102 | | Lo0: .104 | | Lo0: .105 | |
| | BGP AS: 65101 | | BGP AS: 65102 | | BGP AS: 65104 | | BGP AS: 65105 | |
| +------------------+ +------------------+ +------------------+ +------------------+ |
| |
| |
| |
+--------------------------------------------------------------------------------------------------------------------------------------------------+
You can use any hypervisor of your choice (KVM, VMWare Player/ESXI, etc) to run guest VNFs. For KVM you can use corresponding cheat sheet for VM creation.
Actually, the IP addresses of the Docker network may vary depending on your installation. In my case there are just the default values provided.
The topology of the data plane hasn’t changed since the previous lab, so I hope you are already familiar with it:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| |
| +-----------------+ +-----------------+ |
| | de-bln-spine-101| (c)karneliuk.com // Data Centre Fabric | de-bln-spine-201| |
| | (Cisco IOS XRv) | | (Nokia VSR) | |
| | Lo0: .1 | | system: .2 | |
| | BGP AS: 65000 | IPv4 links: 169.254.0.0/24 eq 31 | BGP AS: 65000 | |
| +--+---+---+---+--+ IPv6 loopb: 10.1.1.0/24 eq 32 +--+---+---+---+--+ |
| |.0 |.4 |.8 |.12 IPv6 links: fc00:de:1:0::/64 eq 127 |.2 |.6 |.10|.14 |
| +-----------------+:0 |:4 |:8 |:c IPv6 loopb: fc00:de:1:1::/64 eq 128 |:2 |:6 |:a |:e |
| | | | | | | | | |
| | +------------------------------------------------------------------------+ | | | |
| | | | | | | | | |
| | | +-----------+ +------------------------------------------+ | | |
| | | | | | | | | |
| | | +-----------------------------------------+ +------------+ +----------------+ |
| | | | | | | | | |
| | | +-----------------------------------------------------------------------+ | |
| | | | | | | | | |
| | swp3 | swp4 | swp3 | swp4 | Eth3 | Eth4 | Eth3 | Eth4 |
| | .1/:1| .3/:3 | .5/:5| .7/:7 | .9/:9| .11/:b |.13/:d| .15/:f |
| +-----+------+-----+ +-----+------+-----+ +-----+------+-----+ +-----+------+-----+ |
| | de-bln-leaf-111 +--------------+ de-bln-leaf-112 | | de-bln-leaf-211 +--------------+ de-bln-leaf-212 | |
| | (Cumulus VX) | swp1 swp1 | (Cumulus VX) | | (Arista vEOS) | Eth1 Eth1 | (Arista vEOS) | |
| | lo: .101 | | lo: .102 | | Lo0: .104 | | Lo0: .105 | |
| | BGP AS: 65101 +--------------+ BGP AS: 65102 | | BGP AS: 65104 +--------------+ BGP AS: 65105 | |
| +--+------------+--+ swp2 swp2 +--+------------+--+ +--+------------+--+ Eth2 Eth2 +--+------------+--+ |
| | | | | | | | | |
| + + Anycast IP: .100 + + + + Anycast IP: .103 + + |
| |
+--------------------------------------------------------------------------------------------------------------------------------+
If you need more details on the topology explanation, refer to the previous article.
Today we don’t focus on the configuration of the network elements in terms of interfaces, BGP, etc, because we’ve done that already earlier (link). The main focus today is on management network and Docker cloud, as the zero touch provision (ZTP) process take place there.
The topologies and initial configuration files you can find on my GitHub.
General considerations on the Docker cloud for network services
Some time ago we’ve already deployed Docker for the Service Provider Fabric, that’s why we reuse the whole construct in terms of Ansible role for Docker installation and configuration.
For more details, refer to the article about Telegraf, InfluxDB, Grafana.
The core idea of this article is to reuse the NetBox instance from the previous article, where we have already filled in all the network data, to automatically launch the necessary Docker containers with the corresponding data from NetBox. The instantiation of NetBox itself isn’t yet automated, cause it would the topic for separated discussion. That’s why, our assumption for this lab that the NetBox is up and running and Ansible has already saved token to communicate with it. But before talking about automation with Ansible, we need to prepare the building blocks, which are home-crafted Docker containers.
#1.1. Docker // general approach for the containers creation
The first time we’ve introduced Docker in the Service Provider Fabric monitoring with Telegraf, InfluxDB and Grafana, we have used the official images from the vendor. The second time we have used Docker, it was the last time with NetBox, we have used the official images provided by the community. Today we do a step ahead and we will create our own Docker images. There are a couple of reasons for that:
- Because it’s interesting to practice.
- Because we can control what exactly we have inside our Docker container.
- Because we want to improve efficiency and decrease disc utilization.
The latter is important, because the standard CentOS Docker image without installed services is quite heavy, about 200 Mbytes. Ubuntu is more user friendly, but still about 90 Mbytes. The base image with Alpine Linux is a bit more than 5 Mbytes, what in my eyes makes it game changer for micro services. That’s the reason, why I’m using it to build all the containers. Let’s take a closer look this process.
If you want to learn about the Docker, I strongly recommend you to use “the Docker Book”. I have read it myself and constantly going back to it improve my knowledge.
#1.2. Docker // Container with DHCP server
The first container we create is a DHCP server. Actually, there is no any specific sequence, how you need to create and launch Docker containers in this lab, as they aren’t dependent on each other. That’s why if you prefer to start with DNS or HTTP, feel free to do it. I just follow the alphabetic order here.
That’s how the Dockerfile for the Docker image of DHCP looks like:
2
3
4
5
6
7
8
9
10
11
12
13
# DHCP Container for Data Centre Fabric
FROM alpine
LABEL maintainer="anton@karneliuk.com"
ENV REFRESHED_AT 2019-04-19
RUN apk update; apk add dhcp
RUN touch /var/lib/dhcp/dhcpd.leases
EXPOSE 67/tcp 67/udp 546/udp 547/udp
ENTRYPOINT ["/usr/sbin/dhcpd"]
CMD ["-4", "-f", "-d", "--no-pid", "-cf", "/etc/dhcp/dhcpd.conf"]
As I’m not a Linux expert, I was using heavily GitHub to find examples, which can help me. For DHCP I’ve found an interesting one, which I’ve partially rebuild and extend. Basically, this Docker file do the following:
- Takes the latest release of Alpine linux
- Updates the packages and installs DHCP server
- 4 ports are exposed: for IPv4 and IPv6 DHCP
- DHCP application is launched in foreground using details from “/etc/dhcp/dhcpd.conf”.
One of the great advantages of the Docker is so that you can attach files from your hard disc to the container, what makes possible to store and modify them separately from container.
Let’s build the Docker container with DHCP service:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
Sending build context to Docker daemon 7.68kB
Step 1/8 : FROM alpine
---> cdf98d1859c1
Step 2/8 : LABEL maintainer="anton@karneliuk.com"
---> Running in bcce2a94d94f
Removing intermediate container bcce2a94d94f
---> fb5de72c9375
Step 3/8 : ENV REFRESHED_AT 2019-04-19
---> Running in 01e24761172e
Removing intermediate container 01e24761172e
---> 83882de3611b
Step 4/8 : RUN apk update; apk add dhcp
---> Running in 8b9611915c87
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.3-36-g065f287605 [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.3-35-g640443eed6 [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9755 distinct packages available
(1/2) Installing libgcc (8.3.0-r0)
(2/2) Installing dhcp (4.4.1-r1)
Executing dhcp-4.4.1-r1.pre-install
Executing busybox-1.29.3-r10.trigger
OK: 10 MiB in 16 packages
Removing intermediate container 8b9611915c87
---> 87bfcd8e35b4
Step 5/8 : RUN touch /var/lib/dhcp/dhcpd.leases
---> Running in 1e1b604828d0
Removing intermediate container 1e1b604828d0
---> fdf8588248c2
Step 6/8 : EXPOSE 67/tcp 67/udp 546/udp 547/udp
---> Running in 11800b5aed52
Removing intermediate container 11800b5aed52
---> e1f73018cea5
Step 7/8 : ENTRYPOINT ["/usr/sbin/dhcpd"]
---> Running in 44e14f5a0e19
Removing intermediate container 44e14f5a0e19
---> b4d92adda2d2
Step 8/8 : CMD ["-4", "-f", "-d", "--no-pid", "-cf", "/etc/dhcp/dhcpd.conf"]
---> Running in 96b02a580083
Removing intermediate container 96b02a580083
---> e856e54621f3
Successfully built e856e54621f3
Successfully tagged akarneliuk/dcf_dhcp:latest
As you can see, step by step the container is evolved from a base Alpine Linux image to a full-feature containerized DHCP application, which is ready for operation. To launch this application properly, we would modify the “/etc/dhcp/dhcpd.conf” on our management host in the following way:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# dhcpd.conf
# private DHCP options
option cumulus-provision-url code 239 = text;
default-lease-time 600;
max-lease-time 7200;
authoritative;
# IPv4 OOB management subnet for the data center `DE` `Berlin`
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.129 192.168.1.254;
option domain-name-servers 192.168.1.1;
option domain-name "de.karnet.com";
option routers 192.168.1.1;
option broadcast-address 192.168.1.255;
}
# IPv4 OOB // Fixed lease
host de-bln-leaf-111 {
hardware ethernet 52:54:00:06:02:00;
fixed-address 192.168.1.21;
option host-name "de-bln-leaf-111";
option cumulus-provision-url "http://192.168.1.1/cumulus-ztp.sh";
}
The content of the file is based on the device and prefixes entries in NetBox (link). As of now, there is only one static entry in addition to the overall OOB IPv4v subnet. According to our logic, this file should be automatically generated out of NetBox using Ansible, as you will see later.
To check, if everything is created and configured properly, we can manually launch the container:
2
3
-v $PWD/data/dhcpd.conf:/etc/dhcp/dhcpd.conf:ro \
--name dcf_dhcp akarneliuk/dcf_dhcp
If the container is launched and we can see the logs using “sudo docker container logs dcf_dhcp” command, then everything is correct. Again, later on the container will be launched automatically using Ansible.
#1.3. Docker // Container with DNS server
The second infrastructure enabler service is DNS. This service is essential both for the network functions such as routers/switches and servers/applications. As said earlier, we use one of the most popular DNS servers ever, which is called BIND. Following the approach for DHCP, we create our own Docker container with BIND9 using Alpine Linux. Below you can find the Dockerfile for it:
2
3
4
5
6
7
8
9
10
11
12
# DNS Container for Data Centre Fabric
FROM alpine:latest
LABEL maintainer="anton@karneliuk.com"
LABEL GitHub="https://github.com/akarneliuk/data_center_fabric"
ENV REFRESHED_AT 2019-04-16
RUN apk update; apk add bind
EXPOSE 53/tcp 53/udp
ENTRYPOINT ["/usr/sbin/named", "-f", "-g"]
If you are familiar with the structure of the Dockerfiles, you might spot that this one is fairly easy. We just install “bind” application on top of the Alpine Linux, open the DNS ports in the container and launch the application in foreground to prevent the Docker container to stop. Let’s build the image:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Sending build context to Docker daemon 13.31kB
Step 1/7 : FROM alpine:latest
---> cdf98d1859c1
Step 2/7 : LABEL maintainer="anton@karneliuk.com"
---> Using cache
---> fb5de72c9375
Step 3/7 : LABEL GitHub="https://github.com/akarneliuk/data_center_fabric"
---> Running in e5c4cdcc7c8f
Removing intermediate container e5c4cdcc7c8f
---> b6fffeb47028
Step 4/7 : ENV REFRESHED_AT 2019-04-16
---> Running in 8d6c07682503
Removing intermediate container 8d6c07682503
---> dcb4ae97d6d0
Step 5/7 : RUN apk update; apk add bind
---> Running in a270041a0f74
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.3-36-g065f287605 [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.3-35-g640443eed6 [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9755 distinct packages available
(1/14) Installing libgcc (8.3.0-r0)
(2/14) Installing krb5-conf (1.0-r1)
(3/14) Installing libcom_err (1.44.5-r0)
(4/14) Installing keyutils-libs (1.6-r0)
(5/14) Installing libverto (0.3.0-r1)
(6/14) Installing krb5-libs (1.15.5-r0)
(7/14) Installing json-c (0.13.1-r0)
(8/14) Installing libxml2 (2.9.9-r1)
(9/14) Installing bind-libs (9.12.3_p4-r2)
(10/14) Installing libcap (2.26-r0)
(11/14) Installing db (5.3.28-r1)
(12/14) Installing libsasl (2.1.27-r1)
(13/14) Installing libldap (2.4.47-r2)
(14/14) Installing bind (9.12.3_p4-r2)
Executing bind-9.12.3_p4-r2.pre-install
Executing busybox-1.29.3-r10.trigger
OK: 15 MiB in 28 packages
Removing intermediate container a270041a0f74
---> a4f3827aea03
Step 6/7 : EXPOSE 53/tcp 53/udp
---> Running in b1b759ebcedb
Removing intermediate container b1b759ebcedb
---> e260aa267cc8
Step 7/7 : ENTRYPOINT ["/usr/sbin/named", "-f", "-g"]
---> Running in cb15f6d7037c
Removing intermediate container cb15f6d7037c
---> a5d686ee7bfb
Successfully built a5d686ee7bfb
Successfully tagged akarneliuk/dcf_dns:latest
Much in the same fashion as DHCP, the DNS application relies a lot on configuration files, which we can store locally on our management host. The first one is “named.conf”, which contains general config and paths to served zones:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
options {
directory "/var/bind";
allow-recursion {
127.0.0.1/32;
192.168.0.0/16;
172.17.0.0/16;
fc00:de::/32;
};
forwarders {
192.168.141.2;
};
listen-on {
any;
};
listen-on-v6 {
any;
};
query-source address * port 53;
allow-transfer { none; };
};
zone "de.karnet.com" IN {
type master;
file "zones/de.karnet.com.zone";
allow-update { none; };
notify no;
};
zone "1.168.192.in-addr.arpa" IN {
type master;
file "zones/1.168.192.in-addr.arpa.zone";
allow-update { none; };
notify no;
};
zone "8.6.1.0.2.9.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.c.f.ip6.arpa" IN {
type master;
file "zones/8.6.1.0.2.9.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.c.f.ip6.arpa.zone";
allow-update { none; };
notify no;
};
As it was explained for DHCP, this file will also be automatically generated by Ansible using information from NetBox. Same is applicable for zones: they are files (including file names and content) are automatically generated. Take a look on a forward zone for instance:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$TTL 86400
@ IN SOA infra-enabler.de.karnet.com. root.de.karnet.com. (
100
3600
900
604800
86400
)
; NS entries
IN NS infra-enabler.de.karnet.com.
; A/AAAA entries
de-bln-leaf-111.de.karnet.com. IN A 192.168.1.21
IN AAAA fc00::192:168:1:21
de-bln-leaf-112.de.karnet.com. IN A 192.168.1.22
IN AAAA fc00::192:168:1:22
de-bln-leaf-211.de.karnet.com. IN A 192.168.1.23
IN AAAA fc00::192:168:1:23
de-bln-leaf-212.de.karnet.com. IN A 192.168.1.24
IN AAAA fc00::192:168:1:24
de-bln-spine-101.de.karnet.com. IN A 192.168.1.25
IN AAAA fc00::192:168:1:25
de-bln-spine-201.de.karnet.com. IN A 192.168.1.26
IN AAAA fc00::192:168:1:26
infra-enabler.de.karnet.com. IN A 192.168.1.1
IN AAAA fc00::192:168:1:1
The reverse zones aren’t provided here for a sake of brevity. Refer to the project’s GitHub page for further details of the zone files.
Once we create the corresponding “named.conf” configuration as well as all zones, we can test the Docker container with DNS using the following launch parameters:
2
3
4
-v $PWD/data/named.conf:/etc/bind/named.conf:ro \
-v $PWD/data/zones:/var/bind/zones \
--name dcf_dns akarneliuk/dcf_dns
By NAT’ed IP here I mean the address of the interface, where we are hearing for DNS requests. We haven’t used it in DHCP, because in DHCP we have in generally used “host” network, without any NAT. This was a kind of workaround, as the traditional solution with NAT was not working for Docker. You also can see that we map both “named.conf” from our local host and the folder with “zones” to the container with DNS application.
To verify the details, what is going in the container, you should use “sudo docker container logs dcf_dns” command.
#1.4. Docker // Container with FTP server
The third service we run on Docker is FTP. There are always a lot of shared files you might need for the operation of your infrastructure, and FTP is a perfect place to store them. In the Data Centre Fabric project we build it using one of the most popular FTP applications called VSFTPD and Alpine Linux. Take a look at the Dockerfile below:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# FTP Container for Data Centre Fabric
FROM alpine:latest
LABEL maintainer="anton@karneliuk.com"
LABEL GitHub="https://github.com/akarneliuk/data_center_fabric"
ENV REFRESHED_AT 2019-04-10
ENV FTP_USERNAME dcf_helper
ENV FTP_PASS aq1sw2de3fr4
RUN apk update; apk add vsftpd
RUN echo "local_enable=YES" >> /etc/vsftpd/vsftpd.conf \
&& echo "chroot_local_user=YES" >> /etc/vsftpd/vsftpd.conf \
&& echo "allow_writeable_chroot=YES" >> /etc/vsftpd/vsftpd.conf \
&& echo "write_enable=YES" >> /etc/vsftpd/vsftpd.conf \
&& echo "local_umask=022" >> /etc/vsftpd/vsftpd.conf \
&& echo "passwd_chroot_enable=yes" >> /etc/vsftpd/vsftpd.conf \
&& echo 'seccomp_sandbox=NO' >> /etc/vsftpd/vsftpd.conf \
&& echo 'pasv_enable=Yes' >> /etc/vsftpd/vsftpd.conf \
&& echo 'pasv_min_port=50000' >> /etc/vsftpd/vsftpd.conf \
&& echo 'pasv_max_port=50050' >> /etc/vsftpd/vsftpd.conf \
&& sed -i "s/anonymous_enable=YES/anonymous_enable=NO/" /etc/vsftpd/vsftpd.conf
RUN mkdir -p "/var/ftp/files"
RUN adduser -h "/var/ftp/files" -s "/sbin/nologin" -D $FTP_USERNAME
RUN echo "$FTP_USERNAME:$FTP_PASS" | /usr/sbin/chpasswd
RUN chown -R $FTP_USERNAME:nogroup "/var/ftp/files"
EXPOSE 20 21 50000-50050
ENTRYPOINT ["/usr/sbin/vsftpd", "/etc/vsftpd/vsftpd.conf"]
You might see that FTP user and password is provided directly in the container, what might be considered as non-secure. You can remove it and create the user afterwards.
This time the file for build is quite big as we do a lot of things:
- We install vsftpd on top of Alpine Linux.
- Then we modify internally the configuration file “/etc/vsftpd/vsftpd.conf”.
- We create internally in the container folder, which will be used for storage of the files
- And we configure ports for passive FTP operation (20-21, 50000-50050).
My Docker image is inspired by another one I’ve found on the GitHub. That one is more feature reach, but in the Data Centre Fabric I need less features, as all the stuff is automated. One of the important topics, where I spent really a lot of time with troubleshooting is the fact, that the FTP must operate in passive mode, if it’s deployed in the container. The reason for that is NAT, which stands by default between the container and outer network, whereas passive FTP helps to overcome the NAT problem.
Let’s build the FTP container:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
Sending build context to Docker daemon 4.608kB
Step 1/14 : FROM alpine:latest
---> cdf98d1859c1
Step 2/14 : LABEL maintainer="anton@karneliuk.com"
---> Using cache
---> fb5de72c9375
Step 3/14 : LABEL GitHub="https://github.com/akarneliuk/data_center_fabric"
---> Using cache
---> b6fffeb47028
Step 4/14 : ENV REFRESHED_AT 2019-04-10
---> Running in 5ac83bc8558f
Removing intermediate container 5ac83bc8558f
---> 65cbfc8cf27d
Step 5/14 : ENV FTP_USERNAME dcf_helper
---> Running in 3dc3d60a04d2
Removing intermediate container 3dc3d60a04d2
---> a3ef504c49fc
Step 6/14 : ENV FTP_PASS aq1sw2de3fr4
---> Running in 54fa6a5154c8
Removing intermediate container 54fa6a5154c8
---> 05f425857302
Step 7/14 : RUN apk update; apk add vsftpd
---> Running in 16224dd16a1f
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.3-36-g065f287605 [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.3-35-g640443eed6 [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9755 distinct packages available
(1/3) Installing libcap (2.26-r0)
(2/3) Installing linux-pam (1.3.0-r0)
(3/3) Installing vsftpd (3.0.3-r6)
Executing vsftpd-3.0.3-r6.pre-install
Executing busybox-1.29.3-r10.trigger
OK: 7 MiB in 17 packages
Removing intermediate container 16224dd16a1f
---> d229010e9fb5
Step 8/14 : RUN echo "local_enable=YES" >> /etc/vsftpd/vsftpd.conf && echo "chroot_local_user=YES" >> /etc/vsftpd/vsftpd.conf && echo "allow_writeable_chroot=YES" >> /etc/vsftpd/vsftpd.conf && echo "write_enable=YES" >> /etc/vsftpd/vsftpd.conf && echo "local_umask=022" >> /etc/vsftpd/vsftpd.conf && echo "passwd_chroot_enable=yes" >> /etc/vsftpd/vsftpd.conf && echo 'seccomp_sandbox=NO' >> /etc/vsftpd/vsftpd.conf && echo 'pasv_enable=Yes' >> /etc/vsftpd/vsftpd.conf && echo 'pasv_min_port=50000' >> /etc/vsftpd/vsftpd.conf && echo 'pasv_max_port=50050' >> /etc/vsftpd/vsftpd.conf && sed -i "s/anonymous_enable=YES/anonymous_enable=NO/" /etc/vsftpd/vsftpd.conf
---> Running in cb763eadf44c
Removing intermediate container cb763eadf44c
---> 0575249f1f62
Step 9/14 : RUN mkdir -p "/var/ftp/files"
---> Running in c531ca797b0b
Removing intermediate container c531ca797b0b
---> 27054505a5ff
Step 10/14 : RUN adduser -h "/var/ftp/files" -s "/sbin/nologin" -D $FTP_USERNAME
---> Running in 42fef38956ee
Removing intermediate container 42fef38956ee
---> 65cc2ee78948
Step 11/14 : RUN echo "$FTP_USERNAME:$FTP_PASS" | /usr/sbin/chpasswd
---> Running in 41f92d7d9a5b
chpasswd: password for 'dcf_helper' changed
Removing intermediate container 41f92d7d9a5b
---> 1a136816dc91
Step 12/14 : RUN chown -R $FTP_USERNAME:nogroup "/var/ftp/files"
---> Running in 0c4b832eb98f
Removing intermediate container 0c4b832eb98f
---> b92a4388eec1
Step 13/14 : EXPOSE 20 21 50000-50050
---> Running in d77553319d9a
Removing intermediate container d77553319d9a
---> fa9cf6471939
Step 14/14 : ENTRYPOINT ["/usr/sbin/vsftpd", "/etc/vsftpd/vsftpd.conf"]
---> Running in 5d61c6018f9f
Removing intermediate container 5d61c6018f9f
---> a2be30dfc51e
Successfully built a2be30dfc51e
Successfully tagged akarneliukdcf_ftp:latest
Theoretically, it should be possible to attach the external file to this Docker container, but for whatever reason I have failed to do that and was constantly getting an error. That’s why all the updates of the FTP configuration file were done directly inside the container.
To launch the container, the following command should be used:
2
3
-v $PWD/data:/var/ftp/files \
--name dcf_ftp akarneliuk/dcf_ftp
Once the Container with FTP application is up and running, we can log to it:
2
3
4
5
6
7
Password:
lftp dcf_helper@192.168.1.1:~> ls
-rw-rw-r-- 1 1000 1000 5 Apr 27 18:13 test.txt
lftp dcf_helper@192.168.1.1:/> cat test.txt
Come
5 bytes transferred
The Docker container with FTP service is working fine as well.
#1.5. Docker // Container with HTTP server
The last infrastructure enabler service we are creating in this lab is the HTTP service. As a HTTP server we are using NGINX application running on top of Alpine Linux.
The following Dockerfile is created for the HTTP service:
2
3
4
5
6
7
8
9
10
11
12
13
14
# HTTP Server for Data Centre Fabric
FROM alpine
LABEL maintainer="anton@karneliuk.com"
ENV REFRESHED_AT 2019-04-19
RUN apk update; apk add nginx
RUN mkdir -p /var/www/html/website
RUN mkdir -p /run/nginx
EXPOSE 80
ENTRYPOINT ["nginx"]
As you see, the structure of this Docker file is very straightforward:
- Install Alpine Linux and NGINX application.
- Create two folders, one would be used for to map website files, whereas the second one is used for some temp files during application run.
- Open port 80/TCP
- Launch the “nginx” application.
The main configuration file for NGINX is the following one. Actually, we left all the parameter as default:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# /etc/nginx/nginx.conf
user nginx;
worker_processes auto;
pcre_jit on;
error_log /var/log/nginx/error.log warn;
include /etc/nginx/modules/*.conf;
daemon off;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
server_tokens off;
client_max_body_size 1m;
keepalive_timeout 65;
tcp_nodelay on;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:2m;
gzip_vary on;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
include /etc/nginx/conf.d/*.conf;
}
The configuration above contains general information about NGINX configuration, but not the website itself. For the configuration of the website there is a dedicated file:
2
3
4
5
6
7
8
9
10
11
12
13
14
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name dcf-http.de.karnet.com;
location / {
root /var/www/html/website;
index index.html;
}
location = /404.html {
internal;
}
}
The most important part is “location /”, what is basically the folder with all the website details. We put is as “root /var/www/html/website”, what equals to the folder we’ve created previously in the Dockerfile. Now it’s time to compile the Docker container
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
Sending build context to Docker daemon 10.75kB
Step 1/8 : FROM alpine
---> cdf98d1859c1
Step 2/8 : LABEL maintainer="anton@karneliuk.com"
---> Running in 6e99457afe4c
Removing intermediate container 6e99457afe4c
---> dc2e9b6ffdfe
Step 3/8 : ENV REFRESHED_AT 2019-04-19
---> Running in d5a44655c7a8
Removing intermediate container d5a44655c7a8
---> ffb0d1837eaa
Step 4/8 : RUN apk update; apk add nginx
---> Running in d45adb150f73
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
v3.9.3-36-g065f287605 [http://dl-cdn.alpinelinux.org/alpine/v3.9/main]
v3.9.3-35-g640443eed6 [http://dl-cdn.alpinelinux.org/alpine/v3.9/community]
OK: 9755 distinct packages available
(1/2) Installing pcre (8.42-r1)
(2/2) Installing nginx (1.14.2-r0)
Executing nginx-1.14.2-r0.pre-install
Executing busybox-1.29.3-r10.trigger
OK: 7 MiB in 16 packages
Removing intermediate container d45adb150f73
---> 1ebc94535d84
Step 5/8 : RUN mkdir -p /var/www/html/website
---> Running in 812af6ebf5a0
Removing intermediate container 812af6ebf5a0
---> 5367e078a45d
Step 6/8 : RUN mkdir -p /run/nginx
---> Running in 1522707e3e86
Removing intermediate container 1522707e3e86
---> 0932e6a9df72
Step 7/8 : EXPOSE 80
---> Running in f99022960944
Removing intermediate container f99022960944
---> 28944bf670d6
Step 8/8 : ENTRYPOINT ["nginx"]
---> Running in b7cb24e23e84
Removing intermediate container b7cb24e23e84
---> 9312ae0890e6
Successfully built 9312ae0890e6
Successfully tagged akarneliuk/dcf_http:latest
Once the container is build, we launch it as follows:
2
3
4
5
-v $PWD/data/nginx.conf:/etc/nginx/nginx.conf:ro \
-v $PWD/data/default.conf:/etc/nginx/conf.d/default.conf:ro \
-v $PWD/data/website:/var/www/html/website \
--name dcf_http akarneliuk/dcf_http
As you can see, we map the config files of the NGINX and of the server, as well as the folder with the website files. Once the container is running, we can check that the default webpage is available:
2
3
4
5
6
7
8
9
10
11
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<title>HTML5</title>
</head>
<body>
Data Centre HTTP Server is ready!
</body>
</html>
#1.6. Docker // short summary
So far, all four containers (DHCP, DNS, FTP and HTTP) are up and running, or can be launched later, when necessary. So our data centre infrastructure In any case, we know how to launch them properly, what we will do later using Ansible.
All the containers are published at hub.docker.com in my profile, so that you can download them directly from Ansible or using “sudo docker pull” command.
#2.1. Ansible // Overall project structure
To implement the scenario, we’ve described in the beginning, we have created the following playbook structure:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
+--containers
| +--dhcp
| | +--Dockerfile
| +--dns
| | +--Dockerfile
| +--ftp
| | +--Dockerfile
| +--http
| +--Dockerfile
+--ansible
+--ansible.cfg
+--data_center_fabric.yml
+--files
| +--ftp
| | +--test.txt
| +--http
| +--index.html
+--group_vars
| +--linux
| +--auth.yml
| +--main.yml
+--inventory
| +--hosts
+--README.md
+--roles
+--cloud_enabler
+--tasks
| +--collection_loop.yml
| +--container_dcf_dhcp.yml
| +--container_dcf_dns.yml
| +--container_dcf_ftp.yml
| +--container_dcf_http.yml
| +--main.yml
+--templates
+--cumulus-ztp.j2
+--dhcpd.j2
+--dns_forward_zone.j2
+--dns_reverse_zone_ipv4.j2
+--dns_reverse_zone_ipv4_name.j2
+--dns_reverse_zone_ipv6.j2
+--dns_reverse_zone_ipv6_name.j2
+--named.j2
+--nginx_app.j2
+--nginx_default_server.j2
This is just a subset of the project files, for all of them refer to the project page on Github.
There are quite a lot of files, but in reality, the structure is straightforward, once you understand that. Later in this article, you will learn about all the components in a greater details, but for now we’ll just go through the most important pieces:
- Folder “containers”, as you can see, contains further folders with “Dockerfile” files to create proper Docker image per each service we run: DHCP, DNS, FTP and HTTP.
- Folder “ansible” contains files related to Ansible playbook.
- File “ansible/data_center_fabric.yml” is a main playbook, what should be launched with proper tags as explained in “ansible/README.md”.
- Folder “ansible/files” contains two further folders called “ftp” and “http”, where some static files are stored to be copied to the corresponding container folders during the process, when Ansible build these services.
- Folder “ansible/group_vars/linux” stores all the parameters used either for authentication with Linux host and NetBox and network functions in “auth.yml” or general parameters including overall domain name in “main.yml”.
- Folder “ansible/inventory” contains Ansible inventory file called “hosts”, which lists all the hosts the Ansible playbook executed against.
- File “README.md” provides details how you should launch playbooks to get the desired result.
- Folder “roles” has all the roles (only relevant is shown), whereas for our role “roles/cloud_enabler” there are two further folders “tasks” and “templates” are created.
If you want to jump into details of each file, you are welcomed at the project page in GitHub.
Let’s take a closer look on the master playbook:
2
3
4
5
6
7
8
9
10
---
- hosts: localhost
connection: local
gather_facts: yes
roles:
- dc_underlay
- cloud_docker
- cloud_enabler
...
As you see, the playbook is run against a single host. Let’s double check in the inventory:
2
3
[linux]
localhost
That’s one of the advantages, at least for me: we don’t need to construct fixed or dynamic inventory as the playbook is run just against our management host.
But as you know, when we work with Ansible roles, the most important actions are happening in another place. This place is the “main.yml” playbook of the “cloud_enabler” role:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
---
- name: CREATING FOLDER FOR ALL CONTAINERS
file:
dest: "{{ docker.path_to_containers }}"
state: directory
tags:
- infra_enabler_install
- name: GETTING PWD
command: pwd
register: pwd
tags:
- infra_enabler_install
- infra_enabler_ops
- name: SETTING PWD VARIABLE
set_fact:
pwd_actual: "{{ pwd.stdout }}"
tags:
- infra_enabler_install
- infra_enabler_ops
- name: STARTING COLLECTION LOOP ...
include_tasks: collection_loop.yml
tags:
- infra_enabler_install
- infra_enabler_ops
- name: BUILDING CONTAINERS ...
include_tasks: "container_{{ container_item }}.yml"
loop:
- dcf_dhcp
- dcf_dns
- dcf_ftp
- dcf_http
loop_control:
loop_var: container_item
tags:
- infra_enabler_install
- infra_enabler_ops
...
The followings things are happening in the playbook above:
- The folder “ansible\containers” is created to store the files used by containers.
- Ansible gets your local context using “pwd” command.
- Collected path is converted into a variable.
- All the relevant information from NetBox is extracted and stored as variables
- The containers are created and launched.
As the first three tasks are quite self-explanatory, we’ll focus on the last two till the end of the Chapter.
#2.2. Ansible // Data extraction from NetBox
In the previous article about NetBox, we have already extracted some information from NetBox. Basically, we reuse the approach we deployed there, just add more requests to collect more information:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
---
- name: COLLECTION LOOP // GETTING INFO ABOUT NETBOX CONTAINER
shell: "sudo docker container port {{ netbox_frontend_container }}"
become: yes
register: netbox_frontend_container_port
tags:
- infra_enabler_install
- infra_enabler_ops
- name: COLLECTION LOOP // SETTING API PORT
set_fact:
netbox_port: "{{ netbox_frontend_container_port.stdout | regex_replace('^.*:(\\d+)$', '\\1') }}"
tags:
- infra_enabler_install
- infra_enabler_ops
- name: COLLECTION LOOP // GETTING LIST OF DEVICES
uri:
url: "http://{{ netbox_host }}:{{ netbox_port }}/api/dcim/devices/"
method: GET
return_content: yes
headers:
accept: "application/json"
Authorization: "Token {{ netbox_token }}"
register: get_devices
tags:
- infra_enabler_install
- infra_enabler_ops
- name: COLLECTION LOOP // GETTING LIST OF ALL INTERFACES
uri:
url: "http://{{ netbox_host }}:{{ netbox_port }}/api/dcim/interfaces/?device={{ item.name }}"
method: GET
return_content: yes
headers:
accept: "application/json"
Authorization: "Token {{ netbox_token }}"
register: get_all_interfaces
loop: "{{ get_devices.json.results }}"
tags:
- infra_enabler_install
- infra_enabler_ops
- name: COLLECTION LOOP // GETTING LIST OF ALL IP ADDRESSES
uri:
url: "http://{{ netbox_host }}:{{ netbox_port }}/api/ipam/ip-addresses/?device={{ item.name }}"
method: GET
return_content: yes
headers:
accept: "application/json"
Authorization: "Token {{ netbox_token }}"
register: get_all_ip
loop: "{{ get_devices.json.results }}"
tags:
- infra_enabler_install
- infra_enabler_ops
- name: COLLECTION LOOP // GETTING LIST OF PREFIXES
uri:
url: "http://{{ netbox_host }}:{{ netbox_port }}/api/ipam/prefixes/"
method: GET
return_content: yes
headers:
accept: "application/json"
Authorization: "Token {{ netbox_token }}"
register: get_all_ip_prefixes
tags:
- infra_enabler_install
- infra_enabler_ops
...
One of the major assumptions is that NetBox is running on the same management host, where we are deploying our infrastructure stack. It is not necessary, as the Ansible extracts info using REST API, but this is how my playbooks are created. That’s why, in the first two tasks we are looking for the TCP port, what NetBox frontend listen to. In the last four tasks we just collect from the NetBox:
- List of all devices
- List of all interfaces of all devices
- List of all IP addresses of all devices
- All the prefixes
It’s possible to filter the node in REST path so that we collect only certain nodes or interfaces, but we don’t do that. For now it’s more advantageous to have all the information extracted and stored as variables, but that might change in future.
#2.3. Ansible // Automated creation and launch of the container with infrastructure service
Once the information is collected, we can start doing the core part of the lab that is the launch of the Docker containers with the proper application. As you might have noticed on the point 2.1, we call the proper additional playbooks in a cycle. More or less, all the playbooks are structured in the same way, that’s why I will explain the logic only of one of them. The rest are fairly the same. Let’s take a look on the Ansible playbook to launch the Docker container with DHCP service:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
---
- name: CONTAINER WITH {{ container_item }} // CREATING FOLDER
file:
dest: "{{ docker.path_to_containers }}/{{ container_item }}"
state: directory
tags:
- infra_enabler_install
- name: CONTAINER WITH {{ container_item }} // CREATING STORAGE FOLDER
file:
dest: "{{ docker.path_to_containers }}/{{ container_item }}/data"
state: directory
tags:
- infra_enabler_install
- name: CONTAINER WITH {{ container_item }} // TEMPLATING CONFIG
template:
src: dhcpd.j2
dest: "{{ docker.path_to_containers }}/{{ container_item }}/data/dhcpd.conf"
tags:
- infra_enabler_install
- name: CONTAINER WITH {{ container_item }} // LAUNCHING CONTAINER
docker_container:
name: "{{ container_item }}"
image: "{{ docker.repo }}/{{ container_item }}"
state: started
network_mode: host
volumes:
- "{{ pwd_actual }}/{{ docker.path_to_containers }}/{{ container_item }}/data/dhcpd.conf:/etc/dhcp/dhcpd.conf:ro"
become: yes
tags:
- infra_enabler_install
- infra_enabler_ops
...
The following actions are happening there:
- The folder is created to store appropriate external files to the container.
- Additional subfolder is created.
- Based on the information from NetBox the “dhcpd.conf” is created using Jinja2 template.
- The Docker container is launched using the parameters explained earlier in the corresponding part.
The very important thing, besides the launch of the container itself, is the templating of the config. The following template is used:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
# dhcpd.conf
# private DHCP options
option cumulus-provision-url code 239 = text;
default-lease-time 600;
max-lease-time 7200;
authoritative;
{% for ip_prefix in get_all_ip_prefixes.json.results %}{# f-001: looping all prefixes #}
{% if ip_prefix.site.slug == 'bln' and ip_prefix.role.slug == 'oob-management' and ip_prefix.family.value == 4 %}{# i-001: matching IPv4 OOB subent for Berlin #}
# {{ ip_prefix.description }} // dynamic lease
subnet {{ ip_prefix.prefix | ipaddr('network') }} netmask {{ ip_prefix.prefix | ipaddr('netmask') }} {
range {{ ip_prefix.prefix | ipaddr('network') | ipmath (129) }} {{ ip_prefix.prefix | ipaddr('network') | ipmath (254) }};
option domain-name-servers {% for device in get_devices.json.results %}{% if device.site.slug == 'bln' and 'infra-srv' in device.name %}{{ device.primary_ip4.address | ipaddr('address') }}{% endif %}{% endfor %};
option domain-name "{{ domain_name }}";
option routers {{ ip_prefix.prefix | ipaddr('network') | ipmath (1) }};
option broadcast-address {{ ip_prefix.prefix | ipaddr('broadcast') }};
}
# {{ ip_prefix.description }} // fixed lease
{% for device_all_ips in get_all_ip.results %}{# f-002: list all IPs from all devices #}
{% for device_ip in device_all_ips.json.results %}{# f-003: list all IPs per device #}
{% if device_ip.address | ipaddr('network') == ip_prefix.prefix | ipaddr('network') %}{# i-002: looking for interfaces matching OOB subnet #}
{% for device_all_interfaces in get_all_interfaces.results %}{# f-004: list all interfaces from all devices #}
{% for device_interface in device_all_interfaces.json.results %}{# f-005: list all interfaces per device #}
{% if device_ip.interface.device.name == device_interface.device.name and device_interface.mac_address != None %}{# i-003: checking there is MAC provided #}
host {{ device_ip.interface.device.name }} {
hardware ethernet {{ device_interface.mac_address}};
fixed-address {{ device_ip.address | ipaddr('address') }};
option host-name "{{ device_ip.interface.device.name }}";
{# Adding vendor-specific entries for ZTP #}
{% for device in get_devices.json.results %}{# f-006: loopling all devices #}
{% if device_ip.interface.device.name == device.name %}{# i-004: matching proper device #}
{% if device.platform.slug == 'cumulus-linux' %}{# i-005: adding vendor-specific stuff #}
option cumulus-provision-url "http://{% for d_in in get_devices.json.results %}{% if d_in.site.slug == 'bln' and 'infra-srv' in d_in.name %}{{ d_in.primary_ip4.address | ipaddr('address') }}{% endif %}{% endfor %}/cumulus-ztp.sh";
{% elif device.platform.slug == 'arista-eos' %}{# i-005: adding vendor-specific stuff #}
option bootfile-name "http://{% for d_in in get_devices.json.results %}{% if d_in.site.slug == 'bln' and 'infra-srv' in d_in.name %}{{ d_in.primary_ip4.address | ipaddr('address') }}{% endif %}{% endfor %}/bootstrap";
{% endif %}{# i-005: adding vendor-specific stuff #}
{% endif %}{# i-004: matching proper device #}
{% endfor %}{# f-006: loopling all devices #}
}
{% endif %}{# i-003: checking there is MAC provided #}
{% endfor %}{# f-005: list all interfaces per device #}
{% endfor %}{# f-004: list all interfaces from all devices #}
{% endif %}{# i-002: looking for interfaces matching OOB subnet #}
{% endfor %}{# f-003: list all IPs per device #}
{% endfor %}{# f-002: list all IPs from all devices #}
{% endif %}{# i-001: matching IPv4 OOB subent for Berlin #}
{% endfor %}{# f-001: looping all prefixes #}
Frankly speaking, the template is quite heavy, though I tried to add a lot of comments for almost each condition or loop part. In a nutshell, we create an IPv4 pool for the OOB subnet for the Berlin data centre and then, if there is any documented IP/MAC pair with IP matching OOB subnet, the static DHCP entry is created. Additionally, if the vendor is “Cumulus” or “Arista”, we add specific string for zero touch provision script.
#3. Infrastructure stack verification
Now it’s time to execute our Ansible playbook to check that the automation works properly:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
PLAY [localhost] *******************************************************************************************************************************************************
TASK [Gathering Facts] *************************************************************************************************************************************************
ok: [localhost]
TASK [cloud_enabler : CREATING FOLDER FOR ALL CONTAINERS] **************************************************************************************************************
ok: [localhost]
TASK [cloud_enabler : GETTING PWD] *************************************************************************************************************************************
changed: [localhost]
TASK [cloud_enabler : SETTING PWD VARIABLE] ****************************************************************************************************************************
ok: [localhost]
TASK [cloud_enabler : STARTING COLLECTION LOOP ...] ********************************************************************************************************************
included: /home/aaa/network_fabric/ansible/roles/cloud_enabler/tasks/collection_loop.yml for localhost
!OUTPUT IS OMITTED
TASK [cloud_enabler : CONTAINER WITH dcf_http // TEMPLATING NGINX APP CONFIG] ******************************************************************************************
changed: [localhost]
TASK [cloud_enabler : CONTAINER WITH dcf_http // TEMPLATING NGINX WEBSITE CONFIG] **************************************************************************************
changed: [localhost]
TASK [cloud_enabler : CONTAINER WITH dcf_http // COPYING WEBSITE CONTENT] **********************************************************************************************
changed: [localhost]
TASK [cloud_enabler : CONTAINER WITH dcf_http // TEMPLATING CUMULUS ZTP SCRIPT] ****************************************************************************************
changed: [localhost]
TASK [cloud_enabler : CONTAINER WITH dcf_http // LAUNCHING CONTAINER] **************************************************************************************************
changed: [localhost]
PLAY RECAP *************************************************************************************************************************************************************
localhost : ok=39 changed=26 unreachable=0 failed=0
As you see, all the tasks are executed successfully. Additionally we can check the list of running Docker containers:
2
3
4
5
6
7
8
9
10
11
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b20135edea0b akarneliuk/dcf_http "nginx" 5 minutes ago Up 5 minutes 0.0.0.0:80->80/tcp dcf_http
21643a51e959 akarneliuk/dcf_ftp "/usr/sbin/vsftpd /e…" 6 minutes ago Up 6 minutes 0.0.0.0:20-21->20-21/tcp, 0.0.0.0:50000-50050->50000-50050/tcp dcf_ftp
af0ab2af92e9 akarneliuk/dcf_dns "/usr/sbin/named -f …" 6 minutes ago Up 6 minutes 192.168.1.1:53->53/tcp, 192.168.1.1:53->53/udp dcf_dns
cc3f5ee89e36 akarneliuk/dcf_dhcp "/usr/sbin/dhcpd -4 …" 6 minutes ago Up 6 minutes dcf_dhcp
32cc947ec3d3 nginx:1.15-alpine "nginx -c /etc/netbo…" 3 weeks ago Up 10 minutes 80/tcp, 0.0.0.0:32768->8080/tcp netboxdocker_nginx_1
1e8885a71190 netboxcommunity/netbox:latest "/opt/netbox/docker-…" 3 weeks ago Up 10 minutes netboxdocker_netbox_1
d43ef0cd565b netboxcommunity/netbox:latest "python3 /opt/netbox…" 3 weeks ago Up 10 minutes netboxdocker_netbox-worker_1
bdc68612f29d postgres:10.4-alpine "docker-entrypoint.s…" 3 weeks ago Up 10 minutes 5432/tcp netboxdocker_postgres_1
5d815c016171 redis:4-alpine "docker-entrypoint.s…" 3 weeks ago Up 10 minutes 6379/tcp netboxdocker_redis_1
All the Docker containers that have “dcf_” in the name were automatically created using Ansible playbook. As the Docker images were stored at the “hub.docker.com”, there are automatically download upon creation and stored locally afterwards:
2
3
4
5
6
REPOSITORY TAG IMAGE ID CREATED SIZE
akarneliuk/dcf_ftp latest aeca611f7115 6 days ago 8MB
akarneliuk/dcf_http latest 6576f5f3062e 6 days ago 8.27MB
akarneliuk/dcf_dns latest d87a6908693f 6 days ago 16.1MB
akarneliuk/dcf_dhcp latest 11acac54c145 6 days ago 11.7MB
Once the infrastructure enable stack is up and running, we can test our zero touch provisioning for all the vendors.
#3.1. Zero touch provisioning // Cumulus Linux
Let’s start with the Cumulus Linux. Actually, being Debian Linux based platform the Zero Touch Provision is an bash script. Let’s check it on our HTTP server running as a Docker container:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#!/bin/bash
# CUMULUS-AUTOPROVISIONING
sleep 5
net del dns nameserver ipv4 192.168.1.1
net commit
sleep 5
net add vrf mgmt
net add time zone
net add dns nameserver ipv4 192.168.1.1 vrf mgmt
net commit
adduser aaa --disabled-password --gecos "Network Admin User Account"
adduser aaa sudo
echo "aaa:aaa" | chpasswd
exit 0
The details of the Cumulus Zero Touch Provision I’ve checked on the official website.
Now, let’s verify the content of the “dhcpd.conf” file genereated based on NetBox to search for an entry for any Cumulus Linux host:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# dhcpd.conf
# private DHCP options
option cumulus-provision-url code 239 = text;
default-lease-time 600;
max-lease-time 7200;
authoritative;
# de-bln // OOB management IPv4 // dynamic lease
subnet 192.168.1.0 netmask 255.255.255.0 {
range 192.168.1.129 192.168.1.254;
option domain-name-servers 192.168.1.1;
option domain-name "de.karnet.com";
option routers 192.168.1.1;
option broadcast-address 192.168.1.255;
}
# de-bln // OOB management IPv4 // fixed lease
host de-bln-leaf-111 {
hardware ethernet 00:50:56:2D:3B:89;
fixed-address 192.168.1.21;
option host-name "de-bln-leaf-111";
option cumulus-provision-url "http://192.168.1.1/cumulus-ztp.sh";
}
! FURTHER OUTPUT IS OMITTED
You see that general domain part as well as the host entry for “de-bln-leaf-111” is generated, meaning that leaf switch with the Cumulus Linux should have MAC address “00:50:56:2D:3B:89”.
Following the details we’ve provided earlier in the KVM cheat sheet, we create a VM for Cumulus VX using MAC address documented in NetBox:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> --name=VX1 \
> --description "VX1 VM" \
> --os-type=Linux \
> --ram=1024 \
> --vcpus=1 \
> --boot hd,cdrom,menu=on \
> --disk path=/var/lib/libvirt/images/VX1.qcow2,bus=ide,size=4 \
> --import \
> --graphics vnc \
> --serial tcp,host=0.0.0.0:2261,mode=bind,protocol=telnet \
> --network=bridge:br0,mac=00:50:56:2D:3B:89,model=virtio \
> --network=bridge:br1,mac=52:54:00:06:02:01,model=virtio \
> --network=bridge:br2,mac=52:54:00:06:02:02,model=virtio
WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.
Starting install...
In the example above there is less interfaces than documents in NetBox, but it doesn’t matter, as we focus primarily on OOB interface.
When the router is booted, we see already the ZTP script applied. We can understand it by tracking that management VRF is created and DNS server IP address is mapped to that VRF:
2
3
4
5
6
7
8
9
10
11
12
13
Cumulus VX (TM) is a community supported virtual appliance designed for
experiencing, testing and prototyping Cumulus Networks' latest technology.
For any questions or technical support, visit our community site at:
http://community.cumulusnetworks.com
The registered trademark Linux (R) is used pursuant to a sublicense from LMI,
the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide
basis.
Last login: Sun Apr 28 14:22:19 2019
cumulus@de-bln-leaf-111:mgmt-vrf:~$ net show configuration commands | grep 'nameserver'
net add dns nameserver ipv4 192.168.1.1 vrf mgmt
Also, we can check if the DNS server performs proper resolution:
2
3
4
5
6
7
8
9
10
ping: Warning: source address might be selected on device other than mgmt.
PING de-bln-infra-srv.de.karnet.com (192.168.1.1) from 192.168.1.21 mgmt: 56(84) bytes of data.
64 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=1 ttl=64 time=0.200 ms
64 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=2 ttl=64 time=0.564 ms
64 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=3 ttl=64 time=0.674 ms
^C
--- de-bln-infra-srv.de.karnet.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.200/0.479/0.674/0.203 ms
Hurray, hurray! As we can see the Zero Touch Provisioning is working for Cumulus Linux based switch as expected using our infrastructure enabler Docker cloud.
#3.2. Zero touch provisioning // Arista EOS
The next in the raw is the zero touch provision of the Arista EOS based device. Being Fedora Linux based OS, Arista EOS can utilize either zero touch provisioning script created in BASH or Python, or just copy the start-up config. Let’s take a look into “dhcpd.conf” entry for the Arista EOS (you remember, it’s generated automatically using NetBox info):
2
3
4
5
6
7
8
9
10
# dhcpd.conf
! OUTPUT IS OMITTED
host de-bln-leaf-211 {
hardware ethernet 00:50:56:28:57:01;
fixed-address 192.168.1.23;
option host-name "de-bln-leaf-211";
option bootfile-name "http://192.168.1.1/arista-ztp.conf";
}
! FURTHER OUTPUT IS OMITTED
We see the MAC address, which we need to have on our VM with Arista EOS in order to get fully automated provisioning. Let’s briefly check that the Arista ZTP config is available on the Docker container with HTTP:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
!
switchport default mode access
!
ip name-server vrf default 192.168.1.1
ip domain-name de.karnet.com
!
logging console debugging
!
username aaa privilege 15 secret aaa
!
interface Management1
ip address dhcp
!
management api netconf
transport ssh def
!
end
All variable parameters in the output above are collected either from NetBox or Ansible vars.
Now, let’s start the VM with Arista vEOS lab taking into account proper MAC on the management interface:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
> --name=vEOS1 \
> --description "vEOS VM" \
> --os-type=Linux \
> --ram=2048 \
> --vcpus=2 \
> --boot cdrom,hd,menu=on \
> --cdrom /tmp/Aboot-veos-8.0.0.iso \
> --livecd \
> --disk path=/var/lib/libvirt/images/vEOS1.qcow2,bus=ide,size=4 \
> --graphics vnc \
> --serial tcp,host=0.0.0.0:2281,mode=bind,protocol=telnet \
> --network=bridge:br0,mac=00:50:56:28:57:01,model=virtio \
> --network=bridge:br1,mac=52:54:00:08:02:01,model=virtio \
> --network=bridge:br2,mac=52:54:00:08:02:02,model=virtio
[sudo] password for aaa:
WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.
Starting install...
If we connect to the serial port of the Arista vEOS router during the boot-up process, we can even see how the zero touch provisioning happens:
2
3
4
5
Apr 28 17:54:40 localhost ConfigAgent: %ZTP-6-DHCPv4_SUCCESS: DHCPv4 response received on Management1 [ Ip Address: 192.168.1.23/24/24; Hostname: de-bln-leaf-211; Nameserver: 192.168.1.1; Domain: de.karnet.com; Gateway: 192.168.1.1; Boot File: http://192.168.1.1/arista-ztp.conf ]
Apr 28 17:54:45 de-bln-leaf-211 ConfigAgent: %ZTP-6-CONFIG_DOWNLOAD: Attempting to download the startup-config from http://192.168.1.1/arista-ztp.conf
Apr 28 17:54:45 de-bln-leaf-211 ConfigAgent: %ZTP-6-CONFIG_DOWNLOAD_SUCCESS: Successfully downloaded startup-config from http://192.168.1.1/arista-ztp.conf
Apr 28 17:54:45 de-bln-leaf-211 ConfigAgent: %ZTP-6-RELOAD: Rebooting the system
Once the Arista vEOS router is fully booted, we see the applied configuration:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
! Command: show running-config
! device: localhost (vEOS, EOS-4.21.1.1F)
!
! boot system flash:/vEOS-lab.swi
!
transceiver qsfp default-mode 4x10G
!
logging console debugging
!
ip name-server vrf default 192.168.1.1
ip domain-name de.karnet.com
!
spanning-tree mode mstp
!
no aaa root
!
username aaa privilege 15 secret sha512 $6$nxPX8LSfC/e5w.dg$5Y97P30vpsV.9Ohs11KBsIRBJus/rmscQk64vkT75AQZXRCd2m8Z3pjA5qOI2rXePU6/a7I/MWNSnNH9ilpHY0
!
interface Ethernet1
!
interface Ethernet2
!
interface Management1
ip address dhcp
!
no ip routing
!
management api netconf
transport ssh def
!
end
The last check, how DNS is working:
2
3
4
5
6
7
8
9
10
11
PING de-bln-infra-srv.de.karnet.com (192.168.1.1) 72(100) bytes of data.
80 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=1 ttl=64 time=4.00 ms
80 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=2 ttl=64 time=4.00 ms
80 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=3 ttl=64 time=0.000 ms
80 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=4 ttl=64 time=0.000 ms
80 bytes from de-bln-infra-srv.de.karnet.com (192.168.1.1): icmp_seq=5 ttl=64 time=0.000 ms
--- de-bln-infra-srv.de.karnet.com ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 16ms
rtt min/avg/max/mdev = 0.000/1.600/4.000/1.959 ms, ipg/ewma 4.000/2.679 ms
So, the Arista Zero Touch Provisioning (ZTP) works fine as well.
#3.3. Zero touch provisioning // Nokia SR OS
The next one is Nokia. Though there are not too much things what is possible to do on the OOB, as Nokia SR OS doesn’t support DHCP on the OOB, we still can utilize a couple of services, like
- The FTP server to store the licenses
- The DNS server for name resolution
It is also possible to put the startup config on the FTP server, but we don’t do that. That’s why we launch the Nokia VSR as follows:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
--name=SR1 \
--description "SR1 VM" \
--os-type=Linux \
--sysinfo type='smbios',system_product='TIMOS:address=fc00:de:1:ffff::A1/112@active address=192.168.1.101/24@active static-route=fc00:ffff:1::/64@fc00:de:1:ffff::1 license-file=ftp://dcf_helper:aq1sw2de3fr4@192.168.1.1/sros16.lic slot=A chassis=SR-1 card=iom-1 mda/1=me6-100gb-qsfp28'\
--ram=4096 \
--vcpus=2 \
--boot hd \
--disk path=/var/lib/libvirt/images/SR1.qcow2,bus=virtio,size=4 \
--import \
--graphics vnc \
--serial tcp,host=0.0.0.0:3301,mode=bind,protocol=telnet \
--network=bridge:br0,mac=52:54:00:02:02:00,model=virtio \
--network=bridge:br1,mac=52:54:00:02:02:01,model=virtio \
--network=bridge:br2,mac=52:54:00:02:02:02,model=virtio
As you can see, the FTP server address equals to the IP address of our management host, as well as access credentials equal to the defined during the build of the Docker container.
When the router is booted, we can add the DNS address. Unfortunately, it is impossible to provide DNS-server in “sysinfo” during the VM creation, though it is part of the BOF, and some other BOF parameters such as IP addresses or the license files could be defined. Nevertheless, let’s defined the parameters manually and then test it:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
A:vSIM>bof# primary-dns 192.168.1.1
*A:vSIM>bof#
*A:vSIM# ping de-bln-infra-srv.de.karnet.com
PING 192.168.1.1 56 data bytes
No route to destination. Address: 192.168.1.1, Router: Base
No route to destination. Address: 192.168.1.1, Router: Base
No route to destination. Address: 192.168.1.1, Router: Base
^C
ping aborted by user
---- 192.168.1.1 PING Statistics ----
3 packets transmitted, 0 packets received, 100% packet loss
*A:vSIM# ping de-bln-leaf-112.de.karnet.com
PING 192.168.1.22 56 data bytes
No route to destination. Address: 192.168.1.22, Router: Base
No route to destination. Address: 192.168.1.22, Router: Base
The DNS resolution works properly, but the OOB interface isn’t used for data plane forwarding, that’s why we see errors. But generally we see that the infrastructure enabler stack does its job for Nokia VSR as well.
#3.4. Zero touch provisioning // Cisco IOS XR
The last, but not least, is the zero touch provisioning for Cisco IOS XR router. I have found a link explaining that ZTP works there as well. The main problem, at least with Cisco IOS XRv, that all the ports are shut down by default, therefore even having the infrastructure enabler stack we will have to manually make port up.
Nevertheless, let’s test what we can do. First of all, we check in the DHCP container, what info for Cisco Spine we’ve extracted from the NetBox:
2
3
4
5
6
7
8
9
# dhcpd.conf
! OUTPUT IS OMITTED
host de-bln-spine-101 {
hardware ethernet 00:50:56:2B:FE:41;
fixed-address 192.168.1.25;
option host-name "de-bln-spine-101";
}
! FURTHER OUTPUT IS OMITTED
We don’t provide any additional specific line, just the fixed IP, hostname, etc. So we simply starts the VM with Cisco IOS XRv normally using proper MAC on the management interface:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
> --name=XR1 \
> --description "XR1 VM" \
> --os-type=Linux \
> --ram=3072 \
> --vcpus=1 \
> --boot hd,cdrom,menu=on \
> --disk path=/var/lib/libvirt/images/XR1.qcow2,bus=ide,size=4 \
> --import \
> --graphics vnc \
> --serial tcp,host=0.0.0.0:2251,mode=bind,protocol=telnet \
> --network=bridge:br0,mac=00:50:56:2B:FE:41,model=virtio \
> --network=bridge:br1,mac=52:54:00:05:02:01,model=virtio \
> --network=bridge:br2,mac=52:54:00:05:02:02,model=virtio
WARNING No operating system detected, VM performance may suffer. Specify an OS with --os-variant for optimal results.
Starting install...
Once the VM boots, we change the configuration of the management interface to have an IP address using DHCP:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Sun Apr 28 18:40:19.350 UTC
Building configuration...
!! IOS XR Configuration 6.5.1.34I
vrf mgmt
address-family ipv4 unicast
!
address-family ipv6 unicast
!
!
interface MgmtEth0/0/CPU0/0
vrf mgmt
ipv4 address dhcp
no shutdown
!
end
After the DHCP process is done, we can see that our Cisco IOS XR router has an IP address assigned:
2
3
4
5
6
7
Sun Apr 28 18:41:14.646 UTC
Interface IP-Address Status Protocol Vrf-Name
MgmtEth0/0/CPU0/0 192.168.1.25 Up Up mgmt
GigabitEthernet0/0/0/0 unassigned Shutdown Down default
GigabitEthernet0/0/0/1 unassigned Shutdown Down default
We also can apply the DNS config:
2
3
4
5
6
7
Sun Apr 28 18:43:23.897 UTC
Building configuration...
!! IOS XR Configuration 6.5.1.34I
domain vrf mgmt name-server 192.168.1.1
domain name de.karnet.com
end
And perform check that it’s working:
2
3
4
5
6
Sun Apr 28 18:44:24.183 UTC
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
That’s basically it. Our infrastructure enabler stack of DHCP, DNS, FTP and HTTP services are up and running as the Docker containers.
All the lab files including Dockerfile and Ansible playbooks you can find on my GitHub page.
Lessons learned
ZTP is different for different vendors or different device types. For Arista and Cumulus, which are traditional Data Centre vendors, the ZTP process is very straightforward and is based using Linux native stuff, which ease the life for initial provisioning. For Cisco IOS XR and Nokia SR OS, which are traditional Service Provider operation systems, the ZTP functionality is less popular, though they are also possible at some extend.
Conclusion
Zero touch provisioning is very important step towards fully automated network (data centre, service provider network, etc). It brings the necessary level of speed and agility into network like by making device available to be fully configured using automated framework, like Ansible in conjunction with InfluxData TICK stack. Later on we’ll try to build trully fully automated data centre. Take care and good bye!
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk