Hello my friend,
After the release of the previous article outlining the data and control plane security for IPv4 in Cisco, Nokia and Mellanox/Cumulus (link) I’ve got several requests about the security in IPv6. The requests were fair enough and with this article we close this gap.
2
3
4
5
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
Thanks
Special thanks for Avi Alkobi from Mellanox and Pete Crocker and Attilla de Groot from Cumulus for providing me the Mellanox switch and Cumulus license for the tests.
Disclaimer
This is the fourth article in the series about the Mellanox/Cumulus switch. The three previous are:
- Building lab with Mellanox SN2010
- Configuring Segment Routing on Mellanox/Cumulus, Cisco IOS XR and Nokia SR OS and connecting physical and virtual networks
- Data and control plane protection in Mellanox/Cumulus, Cisco IOS XR and Nokia SR OS
Brief description
The importance of the security for the network in terms of the control and data plane protection was explained in the previous article (link), so there is no necessity to write about that again. In this blogpost we’ll talk about the same principles and technologies, but we’ll focus on IPv6.
Despite the fact that IPv6 is more than 20 years old as it was initially published in RFC 2460 back in 1998, it is still considered to be innovative in many companies. There is one more interesting thing about IPv6: as an official self-contained standard it was released only 2 years ago in 2017 in RFC 8200, which obsoletes RFC 2460 and incorporates all the additions done along the path of IPv6 development.
It could easily be that you already use IPv6 for the communication, even if you are unaware of this fact. For instance, there are multiple communication service providers, which use IPv6 as a single mechanism to provide the Internet access for consumer segment and use some translation techniques such as MAP-T or NAT64 at the edge of their infrastructure so that customers terminals (e.g. ADSL or cable modems) have IPv6 addresses, but the customers can still access IPv4 Internet resources.
The IPv6 is also coming (or has already come) into the enterprise and data centre world, where growing army of servers, VMs and containers need to communicate to each other and to outside world. In such a scenario, IPv6 is the only option to satisfy that growth
That’s why, IPv6 is no longer an option. It is a mandatory way of building the applications’ communications, where the network must be able to facilitate that communication.
What are we going to test?
In the same fashion we did it in the previous article, we’ll dig into two aspects of the traffic flowing through the network:
- The traffic running through the network function and its control, which is an objective for the data plane protection.
- The traffic destined to the network function itself, which is handled by the control plane protection.
Software version
We continue using the same set of the software, we have used during the whole cycle.
Management host:
- CentOS 7.5.1804 with python 2.7.5
- Ansible 2.8.0
- Docker-CE 18.09
- Containerized NetBox
Enabler and monitoring infrastructure:
- Base Linux image for container: Alpine Linux 3.9
- DHCP: ISC DHCP 4.4.1-r2
- DNS: ISC BIND 9.12.3_p4-r2
- FTP: VSFTPD 3.0.3-r6
- HTTP: NGINX 1.14.2-r1
The Data Centre Fabric:
- Mellanox SN 2010
- Cumulus Linux 3.7.10
- Cisco IOS XR 6.5.1 [guest VNF]
- Nokia SR OS (VSR) 19.5.R1 [guest VNF]
All these SWs were used in the Data Centre Fabric project.
Topology
The physical topology of this lab follows the setup done in the previous lab:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
| |
| /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ |
| / Docker cloud \ |
| (c)karneliuk.com / +------+ \ |
| \ +---+ DHCP | / |
| Mellanox/Cumulus lab / | .2+------+ \ |
| \ | / |
| \ 172.17.0.0/16 | +------+ \ |
| +-----------+ \+------------+ +---+ DNS | / |
| | | | +----+ .3+------+ \ |
| | Mellanox | 169.254.255.0/24 | Management |.1 | / |
| | SN2010 | fc00:de:1:ffff::/64 | host | | +------+ \ |
| | | | | +---+ FTP | / |
| | mlx+cl | eth0 enp2s0f1 | carrier |\ | .4+------+ \ |
| | | .21 .1 | | \ | / |
| | | :21 :1 | +------+ | / | +------+ \ |
| | +------------------------+ br0 | | \ +---+ HTTP | / |
| | | | +------+ | \ .5+------+ \ |
| | | | | \ / |
| | | swp1 ens2f0 | +------+ +--+ \/\/\/\/\/\/\/ |
| | +------------------------+ br1 +-+ | |
| | | | +------+ | +-------------+ |
| | | | | | |
| | | swp7 ens2f1 | +------+ +---+ +-------+ | |
| | +------------------------+ br2 | +--+ | | |
| | | | +------+--+ | SR1 | | |
| | | | | +--+ | | |
| +-----------+ | +------+ | | +---+---+ | |
| | | br3 +-----+ | | |
| | +------+ | +---+---+ | |
| | +-----+ | | |
| | +------+ | XR1 | | |
| | | br4 +--------+ | | |
| | +------+ +-------+ | |
| | | |
| +-----------------------------+ |
| |
+-----------------------------------------------------------------------+
Details about the connection of the Mellanox SN 2010 to the HP lab server you can find in the previous lab. DHCP for IPv6 on the management interfaces is out of scope for this article.
Like it was explained in the previous articles, the Mellanox/Cumulus SN2010 device is connected to the two VNFs (virtual network functions) running Cisco IOS XR and Nokia SR OS.
You can use any hypervisor of your choice (KVM, VMWare Player/ESXI, etc) to run guest VNFs. For KVM you can use corresponding cheat sheet for VM creation.
The logical topology for this lab is also same as the previous lab with the a single difference, which is the IP addresses. In the current lab all the IPv4 addresses (besides OOB management interfaces) are replaced by IPv6 addresses:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
| |
| +----------------+eBGP(IPv6/Unicast)+----------+ |
| v v |
| |
| +-----------+ fc00::A:0/1201 +-----------+ |
| sys: fc00::1:1/128 | | to_XR1 g0/0/0/1 | | Lo0: fc00::1:3/128 |
| Lo1: fc00::2:1/128 | SR1 +----------------------------------+ XR1 | Lo1: fc00::2:3/128 |
| Lo2: fc00::3:1/128 | ASN:65001 | :1 :3 | ASN:65003 | Lo2: fc00::3:3/128 |
| +-----+-----+ +-----+-----+ |
| | | |
| ^ :1 | to_SR1 g0/0/0/0 | :3 ^ |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | fc00::A:100/120 +-----------+fc00::A:200/120 | | |
| | | | | | | |
| | +-----------------+ MLX+CL +----------------+ | |
| | :2 | ASN:65002 | :2 | |
| +eBGP(IPv6/Unicast)+-> +-----------+ <+eBGP(IPv6/Unicast)+ |
| |
| lo: fc00::1:2/128 |
| fc00::2:2/128 |
| fc00::3:2/128 |
| |
| (c)2019 karneliuk.com |
| |
+----------------------------------------------------------------------------------------------------+
In the same way we did in the previous lab, we will run BGP as a routing protocol for the fabric, but, obviously, this time we’ll run IPv6 unicast flavour. For the loopback interfaces we use three IPv6 ranges: fc00::1:0/112, fc00::2:0/112 and fc00::3:0/112. The IPv6 addresses of each router out of those three prefixes are advertised to the peers. For link addressing we use fc00::a:0/112 subnetted into three prefixes: fc00::a:0/120, fc00::a:100/120 and fc00::a:200/120. As you can see, the routers establish external BGP sessions to each other using the IPv6 addresses out of the provided ranges.
The topology and the configuration files you can find on my GitHub page.
Data plane protection in IPv6
Before you will see how to limit/control the communication between various IPv6 addresses, we need to make sure that the communication is working in general:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
PING fc00::1:1(fc00::1:1) from fc00::1:2 : 56 data bytes
64 bytes from fc00::1:1: icmp_seq=1 ttl=64 time=2.10 ms
--- fc00::1:1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.101/2.101/2.101/0.000 ms
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::2:1 -I fc00::2:2 -c 1
PING fc00::2:1(fc00::2:1) from fc00::2:2 : 56 data bytes
64 bytes from fc00::2:1: icmp_seq=1 ttl=64 time=1.38 ms
--- fc00::2:1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.386/1.386/1.386/0.000 ms
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::3:1 -I fc00::3:2 -c 1
PING fc00::3:1(fc00::3:1) from fc00::3:2 : 56 data bytes
64 bytes from fc00::3:1: icmp_seq=1 ttl=64 time=1.63 ms
--- fc00::3:1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.630/1.630/1.630/0.000 ms
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::1:3 -I fc00::1:2 -c 1
PING fc00::1:3(fc00::1:3) from fc00::1:2 : 56 data bytes
64 bytes from fc00::1:3: icmp_seq=1 ttl=64 time=2.10 ms
--- fc00::1:3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.101/2.101/2.101/0.000 ms
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::2:3 -I fc00::2:2 -c 1
PING fc00::2:3(fc00::2:3) from fc00::2:2 : 56 data bytes
64 bytes from fc00::2:3: icmp_seq=1 ttl=64 time=1.38 ms
--- fc00::2:3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.386/1.386/1.386/0.000 ms
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::3:1 -I fc00::3:2 -c 1
PING fc00::3:3(fc00::3:3) from fc00::3:2 : 56 data bytes
64 bytes from fc00::3:3: icmp_seq=1 ttl=64 time=1.63 ms
--- fc00::3:3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.630/1.630/1.630/0.000 ms
The connectivity check is provided from the Mellanox/Cumulus prospective. For Cisco IOS XR based VNF, as well as for Nokia SR OS based VNF, the connectivity is done in the same way.
As long the connectivity is OK, let’s start working on its control.
#1.0. Testing methodology
For the data plane protection, the idea is to control the traffic passing through the router. To achieve that we temporary will be shutting down some links. The following matrix summarizes the test scenarios:
| Test traffic generator | Test traffic receiver | Device under test | Link down |
| MLX-CL | XR1 | SR1 | MLX-CL <-> XR1 |
| SR1 | MLX-CL | XR1 | SR1 <-> MLX-CL |
| XR1 | SR1 | MLX-CL | XR1 <-> SR1 |
Detailed explanation how that works you can find in the previous article.
#1.1. Nokia SR OS data plane protection for IPv6
As the primary goal of the data plane protection is to control the traffic passing through the network function, the security measures must be constructed in a way to accomplish this. The security measure is the IPv6 access-list (IPv6 ACL). To explore the topic, we will deploy the following scenario:
- Allow any traffic from fc00::2:0/112 to fc00::2:0/112 IPv6 addresses.
- Deny any other traffic to fc00::2:0/112 IPv6 addresses.
- Allow any traffic to fc00::3:0/112 IPv6 addresses on the TCP port 80.
- Deny any other traffic to fc00::3:0/112 IPv4 addresses.
- Allow any other traffic.
In Nokia SR OS these rules can be implemented using the following ipv6-filter:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
A:admin@SR1# compare
configure {
+ filter {
+ ipv6-filter "IPV6_ACL_DP_PROTECTION" {
+ default-action accept
+ filter-id 10
+ entry 10 {
+ description "ALLOW ANYTHING FROM fc00::2:0/112 TO fc00::2:0/112"
+ match {
+ src-ip {
+ address fc00::2:0/112
+ }
+ dst-ip {
+ address fc00::2:0/112
+ }
+ }
+ action {
+ accept
+ }
+ }
+ entry 20 {
+ description "BLOCK ANYTHING TO fc00::2:0/112"
+ match {
+ dst-ip {
+ address fc00::2:0/112
+ }
+ }
+ action {
+ drop
+ }
+ }
+ entry 30 {
+ description "ALLOW HTTP TO fc00::3:0/112"
+ match {
+ next-header tcp
+ dst-ip {
+ address fc00::3:0/112
+ }
+ dst-port {
+ eq 80
+ }
+ }
+ action {
+ accept
+ }
+ }
+ entry 40 {
+ description "BLOCK ANYTHING TO fc00::3:0/112"
+ match {
+ dst-ip {
+ address fc00::3:0/112
+ }
+ }
+ action {
+ drop
+ }
+ }
+ }
+ }
}
*(gl)[]
A:admin@SR1# commit
The knob ipv6-fitler is the IPv6 ACL in Nokia SR OS world. All the key words, as well as the logic of this config is the same as it was for IPv4 ACL. The processing of the packets are done in a sequence defined by entry numbers starting from the lowest value. If no specific actions were taken (no match), then default-action is performed.
After the IPv6 ACL is created, we can apply it to the proper interfaces:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
A:admin@SR1# compare
configure {
router "Base" {
interface "to_MLX-CL" {
+ ingress {
+ filter {
+ ipv6 "IPV6_ACL_DP_PROTECTION"
+ }
+ }
}
interface "to_XR1" {
+ ingress {
+ filter {
+ ipv6 "IPV6_ACL_DP_PROTECTION"
+ }
+ }
}
}
}
*(gl)[]
A:admin@SR1# commit
Once the commit is successfully performed, we can check the port details:
2
3
4
5
6
Egress Filter : none Ingress Filter : none
Egr IPv6 Flt : none Ingr IPv6 Flt : 10
Ingress stats : Disabled IPv6 DAD : Enabled
Ingress FP QGrp : (none) Egress Port QGrp : (none)
Ing FP QGrp Inst : (none) Egr Port QGrp Inst: (none)
You can spot that the Ingr IPv6 Flt has a strange value of 10, which we haven’t configured. Each ACL is assigned such an ID upon creation, which is coming from old days of the Nokia SR OS, when the ACLs didn’t have names, but only IDs. The ID is still visible within the ACL itself:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
A:admin@SR1# show filter ipv6 "IPV6_ACL_DP_PROTECTION"
===============================================================================
IPv6 Filter
===============================================================================
Filter Id : 10 Applied : Yes
Scope : Template Def. Action : Forward
Type : Normal
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 4
Description : (Not Specified)
Filter Name : IPV6_ACL_DP_PROTECTION
-------------------------------------------------------------------------------
Filter Match Criteria : IPv6
-------------------------------------------------------------------------------
Entry : 10
Description : ALLOW ANYTHING FROM fc00::2:0/112 TO fc00::2:0/112
Log Id : n/a
Src. IP : fc00::2:0/112
Src. Port : n/a
Dest. IP : fc00::2:0/112
Dest. Port : n/a
Next Header : Undefined Dscp : Undefined
ICMP Type : Undefined ICMP Code : Undefined
Sampling : Off Int. Sampling : On
Tcp-flag : (Not Specified)
Fragment : Off
HopByHop Opt : Off Routing Type0 : Off
Auth Hdr : Off ESP header : Off
Flow-label : n/a Flow-label Mask: n/a
Egress PBR : Disabled
Primary Action : Forward
Ing. Matches : 0 pkts
Egr. Matches : 0 pkts
Now we can test if the ACL is working properly:
2
3
4
5
6
7
PING fc00::2:3(fc00::2:3) from fc00::2:2 : 56 data bytes
64 bytes from fc00::2:3: icmp_seq=1 ttl=64 time=1.60 ms
^C
--- fc00::2:3 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.607/1.607/1.607/0.000 ms
In the previous lab we were shutting down the interfaces between the network functions to make sure the traffic is passing the router by. In this blog we continue doing this, but we don’t show that to save the time and space. Refer to the previous article for details.
The traffic between hosts out of fc00::2:0/112 between MLX-CL and XR3 via SR1 works as expected. What about the traffic that should be dropped? Let’s verify it:
2
3
4
5
PING fc00::2:3(fc00::2:3) from fc00::1:2 : 56 data bytes
^C
--- fc00::2:3 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 1999ms
Let’s check the counters in the IPv6 ACL table:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
A:admin@SR1# show filter ipv6 "IPV6_ACL_DP_PROTECTION"
===============================================================================
IPv6 Filter
===============================================================================
Filter Id : 10 Applied : Yes
Scope : Template Def. Action : Forward
Type : Normal
System filter : Unchained
Radius Ins Pt : n/a
CrCtl. Ins Pt : n/a
RadSh. Ins Pt : n/a
PccRl. Ins Pt : n/a
Entries : 4
Description : (Not Specified)
Filter Name : IPV6_ACL_DP_PROTECTION
-------------------------------------------------------------------------------
Filter Match Criteria : IPv6
-------------------------------------------------------------------------------
Entry : 10
Description : ALLOW ANYTHING FROM fc00::2:0/112 TO fc00::2:0/112
! OUTPUT IS TRUNCATED
Ing. Matches : 1 pkts (122 bytes)
Egr. Matches : 0 pkts
Entry : 20
Description : BLOCK ANYTHING TO fc00::2:0/112
! OUTPUT IS TRUNCATED
Ing. Matches : 3 pkts (366 bytes)
Egr. Matches : 0 pkts
! OUTPUT IS TRUNCATED
Second set of the checks is related to HTTP (and other) traffic to hosts in fc00::3:0/112 subnet. We do that as follows:
2
3
4
5
6
7
8
9
curl: (7) Failed to connect to fc00::3:3 port 80: Connection refused
cumulus@MLX-CL:mgmt-vrf:~$ ping6 fc00::3:3 -I fc00::1:2
PING fc00::3:3(fc00::3:3) from fc00::1:2 : 56 data bytes
^C
--- fc00::3:3 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
In Cumulus Linux we rely on curl tool to make an HTTP request and also ping to generate other type of traffic. The fact that the connection is refused just means that there is no HTTP service running on Cisco IOS XR, which we try to connect to. Let’s check the counters in the ACLs again:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
A:admin@SR1# show filter ipv6 "IPV6_ACL_DP_PROTECTION"
===============================================================================2.16.2.2
IPv6 Filter
===============================================================================
Filter Id : 10 Applied : Yes
Scope : Template Def. Action : Forward
! OUTPUT IS TRUNCATED
Entry : 30
Description : ALLOW HTTP TO fc00::3:0/112
! OUTPUT IS TRUNCATED
Ing. Matches : 3 pkts (294 bytes)
Egr. Matches : 0 pkts
Entry : 40
Description : BLOCK ANYTHING TO fc00::3:0/112
! OUTPUT IS TRUNCATED
Ing. Matches : 2 pkts (244 bytes)
Egr. Matches : 0 pkts
===============================================================================
The counters show the proper amount of hits based on the tests we’ve done so far.
To save the space, only the relevant configuration is provided in next two chapters. For detailed explanation refer to the previous article.
#1.2. Cisco IOS XR data plane protection for IPv6
The scenario described above is implemented in Cisco IOS XR using the following configuration:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
Sat Nov 2 22:51:48.350 UTC
Building configuration...
!! IOS XR Configuration 6.5.1.34I
ipv6 access-list IPV6_ACL_DP_PROTECTION
9 remark "ALLOW ANYTHING FROM fc00::2:0/112 TO fc00::2:0/112"
10 permit ipv6 fc00::2:0/112 fc00::2:0/112
19 remark "BLOCK ANYTHING TO fc00::2:0/112"
20 deny ipv6 any fc00::2:0/112
29 remark "ALLOW HTTP TO fc00::3:0/112"
30 permit tcp any fc00::3:0/112 eq www
39 remark "BLOCK ANYTHING TO fc00::3:0/112"
40 deny ipv6 any fc00::3:0/112
50 permit ipv6 any any
!
interface GigabitEthernet0/0/0/0
ipv6 access-group IPV6_ACL_DP_PROTECTION ingress
!
interface GigabitEthernet0/0/0/1
ipv6 access-group IPV6_ACL_DP_PROTECTION ingress
!
end
To check that IPv6 access-list is attached to the interface use the following command:
2
3
4
Sat Nov 2 22:56:40.550 UTC
Outgoing access list is not set
Inbound common access list is not set, access list is IPV6_ACL_DP_PROTECTION
To check the status of the IPv6 ACL including counters use the following command:
2
3
4
5
6
7
8
9
10
11
12
Sat Nov 2 22:59:39.658 UTC
ipv6 access-list IPV6_ACL_DP_PROTECTION
9 remark "ALLOW ANYTHING FROM fc00::2:0/112 TO fc00::2:0/112"
10 permit ipv6 fc00::2:0/112 fc00::2:0/112
19 remark "BLOCK ANYTHING TO fc00::2:0/112"
20 deny ipv6 any fc00::2:0/112
29 remark "ALLOW HTTP TO fc00::3:0/112"
30 permit tcp any fc00::3:0/112 eq www
39 remark "BLOCK ANYTHING TO fc00::3:0/112"
40 deny ipv6 any fc00::3:0/112
50 permit ipv6 any any (361 matches)
#1.3. Cumulus/Mellanox data plane protection for IPv6
In Cumulus Linux the configuration is done using ip6tables, much in the way it was done in the previous lab for iptables:
2
3
4
5
6
[ip6tables]
-A FORWARD --in-interface swp+ -s fc00::2:0/112 -d fc00::2:0/112 -j ACCEPT
-A FORWARD --in-interface swp+ -d fc00::2:0/112 -j DROP
-A FORWARD --in-interface swp+ -d fc00::3:0/112 -p tcp --dport 80 -j ACCEPT
-A FORWARD --in-interface swp+ -d fc00::3:0/112 -j DROP
Once you have created the ACL in a proper file, you need to install it HW, what is Mellanox or Broadcom specific stuff:
2
3
4
5
6
7
8
9
Reading rule file /etc/cumulus/acl/policy.d/00control_plane.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/00control_plane.rules ...
Reading rule file /etc/cumulus/acl/policy.d/50_data_plane_protection.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/50_data_plane_protection.rules ...
Reading rule file /etc/cumulus/acl/policy.d/99control_plane_catch_all.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/99control_plane_catch_all.rules ...
Installing acl policy
done.
To verify the counters as well as the operation of the ACLs as such, use the following commands:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
--------------------------------
Listing rules of type ip6tables:
--------------------------------
TABLE filter :
Chain INPUT (policy ACCEPT 431 packets, 62272 bytes)
pkts bytes target prot opt in out source destination
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:3785 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:3784 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:4784 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE ospf swp+ any anywhere anywhere POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE tcp swp+ any anywhere anywhere tcp dpt:bgp POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE tcp swp+ any anywhere anywhere tcp spt:bgp POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp router-solicitation POLICE mode:pkt rate:100 burst:100 class:2
39 4758 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp router-advertisement POLICE mode:pkt rate:500 burst:500 class:2
375 33750 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp neighbour-solicitation POLICE mode:pkt rate:400 burst:400 class:2
157 14130 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp neighbour-advertisement POLICE mode:pkt rate:400 burst:400 class:2
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 130 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 131 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 132 POLICE mode:pkt rate:200 burst:100 class:6
12 1528 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 143 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere POLICE mode:pkt rate:64 burst:40 class:2
0 0 POLICE udp swp+ any anywhere anywhere udp dpts:dhcpv6-client:dhcpv6-server POLICE mode:pkt rate:100 burst:100 class:2
0 0 POLICE tcp swp+ any anywhere anywhere tcp dpts:dhcpv6-client:dhcpv6-server POLICE mode:pkt rate:100 burst:100 class:2
0 0 POLICE all swp+ any anywhere anywhere ADDRTYPE match dst-type LOCAL POLICE mode:pkt rate:1000 burst:1000 class:2
0 0 POLICE all swp+ any anywhere anywhere ADDRTYPE match dst-type IPROUTER POLICE mode:pkt rate:400 burst:100 class:2
0 0 SETCLASS all swp+ any anywhere anywhere SETCLASS class:0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all swp+ any fc00::2:0/112 fc00::2:0/112
0 0 DROP all swp+ any anywhere fc00::2:0/112
0 0 ACCEPT tcp swp+ any anywhere fc00::3:0/112 tcp dpt:http
0 0 DROP all swp+ any anywhere fc00::3:0/112
Chain OUTPUT (policy ACCEPT 10950 packets, 885K bytes)
pkts bytes target prot opt in out source destination
The data plane for IPv6 is protected by this point.
Control plane protection
If the data plane protection is related to the connected customers, then control plane protection is related to the traffic destined to the router itself. This is mainly the traffic related to the network management (such as SSH, SNMP, etc) or so-called control-plane traffic (such as BGP, OSPF, etc).
In our scenario, we will do the following:
- Deny any SSH traffic on the data plane interfaces for IPv6
- Allow BGP only from link IPv6 subnets
- Block all the rest BGP traffic
- Allow all the rest traffic
#2.1. Nokia SR OS control plane protection
To implement the scenario, the following configuration is applied in Nokia SR OS:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
A:admin@SR1# compare
configure {
system {
security {
cpm-filter {
+ ipv6-filter {
+ admin-state enable
+ entry 10 {
+ description "BLOCK SSH TRAFFIC"
+ match {
+ next-header tcp
+ dst-port {
+ eq 22
+ }
+ }
+ action {
+ drop
+ }
+ }
+ entry 20 {
+ description "ALLOW LINK BGP"
+ match {
+ next-header tcp
+ src-ip {
+ address fc00::a:0/112
+ }
+ dst-ip {
+ address fc00::a:0/112
+ }
+ dst-port {
+ eq 179
+ }
+ }
+ action {
+ accept
+ }
+ }
+ entry 30 {
+ description "BLOCK REST BGP"
+ match {
+ next-header tcp
+ dst-port {
+ eq 170
+ }
+ }
+ action {
+ drop
+ }
+ }
+ }
}
}
}
}
*(gl)[]
A:admin@SR1# commit
You can verify its operation using this command:
2
3
4
5
6
7
8
9
10
11
12
13
14
A:admin@SR1# show system security cpm-filter ipv6-filter
===============================================================================
CPM IPv6 Filter (applied)
===============================================================================
Id Dropped Forwarded Description
-------------------------------------------------------------------------------
10 0 0 BLOCK SSH TRAFFIC
20 0 52 ALLOW LINK BGP
30 0 0 BLOCK REST BGP
-------------------------------------------------------------------------------
Num CPM IPv6 filter entries: 3
===============================================================================
#2.2. Cisco IOS XR control plane protection
From Cisco IOS XR point, of view, the following configuration should be applied:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
Sat Nov 2 23:34:49.303 UTC
Building configuration...
!! IOS XR Configuration 6.5.1.34I
control-plane
management-plane
out-of-band
vrf mgmt
interface MgmtEth0/0/CPU0/0
allow all
!
!
!
!
ipv6 access-list IPV6_ACL_DP_PROTECTION
49 remark "ALLOW LINK BGP"
50 permit tcp fc00::a:0/112 fc00::a:0/112 eq bgp
59 remark "BLOCK OTHER BGP"
60 deny tcp any any eq bgp
999 permit ipv6 any any
!
ssh server v2
ssh server vrf mgmt
end
RP/0/0/CPU0:XR1(config)#commit
The verification is the same as the data plane given we have extended the same ACL.
#2.3. Cumulus/Mellanox control plane protection
Generally speaking, we the approach in Cumulus would be the same as the data plane, but we need to change the proper ACL application (from chain FORWARD to INPUT). To manage them separately, we create another file with ACL rules:
2
3
4
5
[ip6tables]
-A INPUT --in-interface swp+ -p tcp --dport 22 -j DROP
-A INPUT --in-interface swp+ -s fc00::a:0/112 -d fc00::a:0/112 -p tcp --dport 179 -j ACCEPT
-A INPUT --in-interface swp+ -p tcp --dport 179 -j DROP
Sequentially this rule would be applied before data plane protection based on the number in the beginning of the file.
Don’t forget to install rules in HW:
2
3
4
5
6
7
8
9
10
11
Reading rule file /etc/cumulus/acl/policy.d/00control_plane.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/00control_plane.rules ...
Reading rule file /etc/cumulus/acl/policy.d/40_control_plane_protection.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/40_control_plane_protection.rules ...
Reading rule file /etc/cumulus/acl/policy.d/50_data_plane_protection.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/50_data_plane_protection.rules ...
Reading rule file /etc/cumulus/acl/policy.d/99control_plane_catch_all.rules ...
Processing rules in file /etc/cumulus/acl/policy.d/99control_plane_catch_all.rules ...
Installing acl policy
done.
The verification is the same as it was for the data plane:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
--------------------------------
Listing rules of type ip6tables:
--------------------------------
TABLE filter :
Chain INPUT (policy ACCEPT 431 packets, 62272 bytes)
pkts bytes target prot opt in out source destination
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:3785 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:3784 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE udp swp+ any anywhere anywhere udp dpt:4784 POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE ospf swp+ any anywhere anywhere POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE tcp swp+ any anywhere anywhere tcp dpt:bgp POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE tcp swp+ any anywhere anywhere tcp spt:bgp POLICE mode:pkt rate:2000 burst:2000 class:7
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp router-solicitation POLICE mode:pkt rate:100 burst:100 class:2
48 5856 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp router-advertisement POLICE mode:pkt rate:500 burst:500 class:2
457 41130 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp neighbour-solicitation POLICE mode:pkt rate:400 burst:400 class:2
196 17640 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmp neighbour-advertisement POLICE mode:pkt rate:400 burst:400 class:2
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 130 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 131 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 132 POLICE mode:pkt rate:200 burst:100 class:6
12 1528 POLICE ipv6-icmp swp+ any anywhere anywhere ipv6-icmptype 143 POLICE mode:pkt rate:200 burst:100 class:6
0 0 POLICE ipv6-icmp swp+ any anywhere anywhere POLICE mode:pkt rate:64 burst:40 class:2
0 0 POLICE udp swp+ any anywhere anywhere udp dpts:dhcpv6-client:dhcpv6-server POLICE mode:pkt rate:100 burst:100 class:2
0 0 POLICE tcp swp+ any anywhere anywhere tcp dpts:dhcpv6-client:dhcpv6-server POLICE mode:pkt rate:100 burst:100 class:2
0 0 DROP tcp swp+ any anywhere anywhere tcp dpt:ssh
0 0 ACCEPT tcp swp+ any fc00::a:0/112 fc00::a:0/112 tcp dpt:bgp
0 0 DROP tcp swp+ any anywhere anywhere tcp dpt:bgp
0 0 POLICE all swp+ any anywhere anywhere ADDRTYPE match dst-type LOCAL POLICE mode:pkt rate:1000 burst:1000 class:2
0 0 POLICE all swp+ any anywhere anywhere ADDRTYPE match dst-type IPROUTER POLICE mode:pkt rate:400 burst:100 class:2
0 0 SETCLASS all swp+ any anywhere anywhere SETCLASS class:0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all swp+ any fc00::2:0/112 fc00::2:0/112
0 0 DROP all swp+ any anywhere fc00::2:0/112
0 0 ACCEPT tcp swp+ any anywhere fc00::3:0/112 tcp dpt:http
0 0 DROP all swp+ any anywhere fc00::3:0/112
So far we have protected the control plane as well. You can find the final configuration files on the corresponding GitHub page.
Lessons learned
Building the environment for IPv6 switching with Linux bridges could be quite a painful exercise, as the bridges have by default IPv6 link-local addresses. This fact implies that the Linux host also takes part in IPv6 routing (e.g. answering to network discovery packets), so there are plenty of hacks might be necessary to allow the lab working properly.
Conclusion
That was a fourth article expanding the initial series of the 3 articles about Cumulus/Mellanox. IPv6 is equally important as IPv4 today, so having security for both of them is equally important as well. Take care and goodbye!
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message (https://karneliuk.com/contact/). Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk