Hello my friend,
Today we will talk about one of the most popular monitoring protocol over past decade or two, which is SNMP. In the world of cyber-attacks and risks, we will review the most protected version of SNMP, which is version 3 with authentication and privacy in multivendor-environment with Arista EOS, Cisco IOS XR, Cumulus Linux and Nokia (Alcatel-Lucent) SR OS.
2
3
4
5
retrieval system, or transmitted in any form or by any
means, electronic, mechanical or photocopying, recording,
or otherwise, for commercial purposes without the
prior permission of the author.
Disclaimer
You can ask me, why I need to review SNMP configuration in multivendor network, if I have already shown the model-driven telemetry, which provides information that is much more detailed. Well, the question is legitimate and here are the answers. First, it will take a while before all the network functions will support model-driven telemetry. For instance, Cumulus Linux doesn’t support it yet. Second, SNMP will continue playing important role in the network monitoring as well-known, time-proven and stable monitoring protocol.
Brief description
Obviously monitoring is very important in any kind of network. Whatever network we have in mind, like service providers, data centers or enterprise, it must be controlled. On the other hand, there is no control without measurements and monitoring. And that’s exactly the stage for SNMP, which provides possibility to collect valuable information (mainly counters, but not only) about network health.
SNMP also allows to perform basic configuration of the network, but it is out of the scope for this article.
The first SNMP version was realized back in 1990 in RFC 1157 and since time developed a lot adding such capabilities as Bulk requests, configuration of network elements and security features. The latest actual version of SNMP is 3 and described in RFC 3410.
What are we going to test?
We will enable SNMP version 3 all the provider (EOS1, EOS2, SR1, XR1) and customer (VX4) network functions from our service provider fabric (link):
- Create user with authentication and privacy security mechanism
- Read-only view of “iso” tree
The success criteria for this article is to be able to collect interesting SNMP data using “snmpwalk” from management host.
Software version
The following software components are used in this lab:
- CentOS 7.5.1804 with python 2.7.5
- Ansible 2.7.0
- Nokia SR OS 16.0.R5 [guest VNF]
- Arista EOS 4.21.1.1F [guest VNF]
- Cisco IOS XR 6.5.1 [guest VNF]
- Cumulus Linux 3.7.2 [guest VNF]
More details about Service Provider Fabric you may find in the previous articles.
Topology
This article continues series about the service provider fabric, that’s why we use already known topology:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| |
| +-------------+ +-----------------+ |
| | Internet GW | | management host | |
| +------+------+ +--------+--------+ |
| | eth1 | ens33 |
| | .2 | .131 OOB: 192.168.141.0/24 |
| | | |
| +---+-----------+--------------+--------------+-------------+ |
| | | | | | |
| | BOF (Eth) | MgmtEth0 | Management1 | Management1 | eth0 |
| | .101 | .51 | .71 | .72 | .84 |
| +---+---+ +---+---+ +---+---+ +---+---+ +---+---+ |
| | SR1 | | XR1 | | EOS1 | | EOS2 | | VX4 | |
| +-------+ +-------+ +-------+ +-------+ +-------+ |
| |
+---------------------------------------------------------------------------+
You can use any hypervisor of your choice (KVM, VMWare Player/ESXI, etc). For KVM you can use corresponding cheat sheet for VM creation.
The logical topology is also traditional one for the service provider fabric:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
| |
| L0: |
| 10.0.0.22/32 |
| fc00::10:0:0:22/128 10.1.22.0/24 |
| (Metric: 10) fc00::10:1:22:0/112 |
| +---------------+ (Metric: 10) |
| | EOS1 (SID:22) |(Eth2)+----------------------+ |
| +---------------+ .22/:22 | |
| (Eth1) | |
| + .22/:22 + .1/:1 10.1.44.0/24 |
| | (Eth1) fc00::10:1:44:0/112 |
| | 10.22.33.0/24 +--------------+ (Metric: 10) +----------------+ |
| | fc00::10:22:33:0/112 | EOS2 (SID:1) |(Eth3)+--------------+(Eth3)| SR1 (SID: 44) | |
| | (Metric: 100) +--------------+ .1:/1 .44/:44 +----------------+ |
| | (Eth2) |
| + .33/:33 + .1/:1 L0: system: |
| (G0/0/0/0) | 10.0.0.1/32 10.0.0.44/32 |
| +---------------+ | fc00::10:0:0:1/128 fc00::10:0:0:44/128 |
| | XR1 (SID:33) |(G0/0/0/1)+------------------+ (Metric: 10) (Metric: 10) |
| +---------------+ .33/:33 (Metric: 10) |
| 10.1.33.0/24 |
| L0: fc00::10:1:33:0/112 |
| 10.0.0.33/32 |
| fc00::10:0:0:33/128 |
| (Metric: 10) ISIS 0 (CORE) / level 2 / Segment Routing |
| |
| ASN: 65000 |
| |
| Full iBGP mesh for VPNV4 UNICAST, VPNV6 UNICAST |
| and L2VPN EVPN between loopbacks of all PEs |
| |
+-------------------------------------------------------------------------------------------------------------+
We are not configuring any new routing protocol or service, but rather enable the existing with possibility to be monitored.
The existing state of the Service Provider Fabric you can find on my GitHub page.
General algorithm for configuration of SNMP version 3
Comparing to earlier versions of SNMP, like version 1, 2 or even 2c, SNMPv3 has plenty of specifics, which you need to know in order to configure it successfully. The steps below aim to help you with the process:
- Configure “engineID”, what is a unique identification of the network function (in our case we will use loopback 0/system interface IPv4 address adding all leading zeros: IPv4 10.0.0.1 -> engineID 010000000001) within SNMP version 3 world, and it is used for hashing of passwords.
- Create “view”, what defines the part of the SNMP tree will be available for certain user (“iso” in our case).
- Create “group”, which associates certain security policy (both authentication and encryption in our case) with earlier created “view”.
- Create “user”, which is associated with created “group” and provide access credentials.
Now we will follow these steps for each vendor network elements from our service provider fabric.
The next list of vendors does not show any particular sympathy, just an alphabetic order.
#1. Configuration of SNMP version 3 in Arista EOS
Knowing exact steps what to do makes the configuration relatively simple. For the PE network function EOS1 based on Arista EOS we implement the following configuration:
2
3
4
5
6
snmp-server engineID local 010000000022
snmp-server source-interface Management1
snmp-server view all-items iso included
snmp-server group TG v3 priv read all-items
snmp-server user Collector TG v3 auth sha SUPER_AUTH priv aes SUPER_PASS
For P network function EOS2, which is also based on Arista EOS, the configuration is the same besides “engineID”:
2
3
4
5
6
snmp-server engineID local 010000000001
snmp-server source-interface Management1
snmp-server view all-items iso included
snmp-server group TG v3 priv read all-items
snmp-server user Collector TG v3 auth sha SUPER_AUTH priv aes SUPER_PASS
If you change engineID, you need you to re-enter the credentials, because hashing of passwords are dependent on engineID.
You can do some local verifications on any Arista EOS based boxes to confirm that SNMP is enabled:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
Local SNMP EngineID: 010000000001
EOS2#show snmp
Chassis: EOS2-Arista-vEOS-lab-4.21.1.1F
19143 SNMP packets input
0 Bad SNMP version errors
0 Unknown community name
0 Illegal operation for community name supplied
0 Encoding errors
178509 Number of requested variables
0 Number of altered variables
951 Get-request PDUs
478 Get-next PDUs
0 Set-request PDUs
19147 SNMP packets output
0 Too big errors
0 No such name errors
0 Bad value errors
0 General errors
19137 Response PDUs
10 Trap PDUs
Access Control
1 Users
1 Groups
1 Views
SNMP logging: enabled
Logging to 192.168.141.135.162
SNMP agent enabled in VRFs: default
SNMP source interfaces:
Management1 in VRF default
#2. Configuration of SNMP version 3 in Cisco IOS XR
The configuration on Cisco IOS XR is quite similar to one we’ve just provided for Arista EOS, that’s why the following config is applied to PE network function XR1:
2
3
4
5
6
7
Sat Feb 23 18:24:30.997 UTC
Building configuration...
snmp-server engineID local 010000000033
snmp-server user Collector TG v3 auth sha SUPER_AUTH priv aes 128 SUPER_PRIV
snmp-server view all-items iso included
snmp-server group TG v3 priv notify all-items read all-items
On Cisco IOS XR you can utilize the same verification commands, which were previously used for Arista EOS:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Sat Feb 23 18:30:22.612 UTC
SNMP engineID 0100000000330050562d7b4a
RP/0/0/CPU0:XR1#show snmp
Sat Feb 23 18:32:09.385 UTC
Chassis: XR1-Cisco-XRv-6.5.1.34I
1483 SNMP packets input
0 Bad SNMP version errors
0 Unknown community name
0 Illegal operation for community name supplied
0 Encoding errors
0 Number of requested variables
0 Number of altered variables
0 Get-request PDUs
118 Get-next PDUs
0 Set-request PDUs
1483 SNMP packets output
0 Too big errors (Maximum packet size 1500)
0 No such name errors
0 Bad values errors
0 General errors
1483 Response PDUs
0 Trap PDUs
SNMP logging: Disabled
Point out how the engineID looks in Cisco IOS XR like. It’s partially automatically generated, so you need to be aware of this.
#3. Configuration of SNMP version 3 in Cumulus Linux
The configuration in Cumulus Linux is even a bit easier, as there is no concept of the group and you put view directly to the user:
2
3
4
net add snmp-server listening-address 192.168.141.84 vrf mgmt
net add snmp-server viewname all-items included iso
net add snmp-server username Collector auth-sha SUPER_AUTH encrypt-aes SUPER_PASS view all-items
Unfortunately, I haven’t found, where I can configure “engineID” in Cumulus Linux in the official guide. If anybody advise me how to do it, I will be very grateful.
The verification of the local status of the SNMP on the Cumulus Linux looks like as follows:
2
3
4
5
6
7
8
9
10
11
Simple Network Management Protocol (SNMP) Daemon.
--------------------------------- ----------------------------------
Current Status active (running)
Reload Status enabled
Listening IP Addresses localhost, 192.168.141.84 vrf mgmt
Main snmpd PID 2632
Version 1 and 2c Community String Not Configured
Version 3 Usernames Configured
--------------------------------- ----------------------------------
#4. Configuration of SNMP version 3 in Nokia (Alcatel-Lucent) SR OS
Nokia (Alcatel-Lucent) SR OS has one specific regarding configuration of SNMPv3 credentials. As we said earlier, for hashing the passwords “engineID”, as the password is stored using hash as described in RFC 3414. Arista EOS, Cisco IOS XR and Cumulus Linux calculates this hash automatically, whereas in Nokia (Alcatel-Lucent) SR OS you need to perform this activity manually. The easiest way is to use “snmpkey” tool in Linux:
2
3
4
5
6
7
$ sudo yum install –y perl-Crypt-Rijndael
$ snmpkey sha SUPER_AUTH 010000000044 aes SUPER_PASS
authKey: 0xbd864d281d446e52124d15faedf4a24b9606908d
privKey: 0xffce81a56c63902127f00be7032d5553
Thanks to my colleague Frank Bauer from Vodafone, who shew me this tool.
To get “snmpkey” we need to install the following two modules. The first one enables usage of “snmpkey” in general and the second one adds support for “aes” encryption.
Once you have hashed values for two passwords, you can proceed with the configuration of network function based on Nokia (Alcatel-Lucent) SR OS:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
A:admin@SR1# compare
configure {
system {
management-interface {
snmp {
admin-state enable
engine-id 010000000044
}
}
security {
snmp {
access "TG" context "_sros_none" security-model usm security-level privacy {
read "iso"
notify "iso"
}
}
user-params {
local-user {
user "Collector" {
access {
snmp true
}
snmp {
group "TG"
authentication {
authentication-protocol sha
authentication-key bd864d281d446e52124d15faedf4a24b9606908d
privacy {
privacy-protocol aes-128-cfb
privacy-key ffce81a56c63902127f00be7032d5553
}
}
}
}
}
}
}
}
}
Nokia has different contexts, where you need to enable SNMP, but generally, you are performing the same steps provided above: configuring “engineID”, configure “view” and “group”, associate “user” with “group” and provide credentials.
You can locally verify the operation of SNMP using the following command at Nokia (Alcatel-Lucent) SR OS device:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
===============================================================================
System Information
===============================================================================
System Name : SR1
System Type : 7750 SR-1
Chassis Topology : Standalone
System Version : B-16.0.R5
System Contact :
System Location :
System Coordinates :
System Up Time : 0 days, 19:08:00.39 (hr:min:sec)
Configuration Mode Cfg : model-driven
Configuration Mode Oper: model-driven
Last Mode Changed : 2019/02/23 14:33:26 Duration: 0d 00:00:01
SNMP Port : 161
SNMP Engine ID : 010000000044
SNMP Engine Boots : 10
SNMP Max Message Size : 1500
SNMP Admin State : Enabled
SNMP Oper State : Enabled
SNMP Index Boot Status : Persistent
SNMP Sync State : N/A
! FURHTER OUTPUT IS OMITTED
Verification of SNMP version 3 work from management host
In the beginning of the article we have defined the success criteria, which is the possibility to collect SNMP information using Linux tool called “snmpwalk”. By default, it isn’t installed, but we can easily do it:
Once it’s installed, we are able it use it. The tool has very extensive built-in help, which allows you to build proper requests:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
USAGE: snmpwalk [OPTIONS] AGENT [OID]
Version: 5.7.2
Web: http://www.net-snmp.org/
Email: net-snmp-coders@lists.sourceforge.net
OPTIONS:
-h, --help display this help message
-H display configuration file directives understood
-v 1|2c|3 specifies SNMP version to use
-V, --version display package version number
SNMP Version 1 or 2c specific
-c COMMUNITY set the community string
SNMP Version 3 specific
-a PROTOCOL set authentication protocol (MD5|SHA)
-A PASSPHRASE set authentication protocol pass phrase
-e ENGINE-ID set security engine ID (e.g. 800000020109840301)
-E ENGINE-ID set context engine ID (e.g. 800000020109840301)
-l LEVEL set security level (noAuthNoPriv|authNoPriv|authPriv)
-n CONTEXT set context name (e.g. bridge1)
-u USER-NAME set security name (e.g. bert)
-x PROTOCOL set privacy protocol (DES|AES)
-X PASSPHRASE set privacy protocol pass phrase
-Z BOOTS,TIME set destination engine boots/time
General communication options
-r RETRIES set the number of retries
-t TIMEOUT set the request timeout (in seconds)
! FURHTER OUTPUT IS OMITTED
Take some to review the snippet above so that you can understand, how we structure the request. To test SNMP we will use the one of the most popular SNMP MIBs, which is supported in all the vendors including Arista, Cisco, Cumulus and Nokia. This MIB is called “IF-MIB” and it’s typically included in any Linux distributive (at least in CentOS 7, which I’m using as management host):
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
IF-MIB::ifName.1 = STRING: Ethernet1
IF-MIB::ifName.2 = STRING: Ethernet2
IF-MIB::ifName.3 = STRING: Ethernet3
IF-MIB::ifName.4 = STRING: Ethernet4
IF-MIB::ifName.999001 = STRING: Management1
IF-MIB::ifName.5000000 = STRING: Loopback0
! FURHTER OUTPUT IS OMITTED
$ snmpwalk -v 3 -l authPriv -u Collector -a SHA -A SUPER_AUTH -x AES -X SUPER_PASS EOS1 IF-MIB::ifXTable
IF-MIB::ifName.1 = STRING: Ethernet1
IF-MIB::ifName.2 = STRING: Ethernet2
IF-MIB::ifName.3 = STRING: Ethernet3
IF-MIB::ifName.4 = STRING: Ethernet4
IF-MIB::ifName.999001 = STRING: Management1
IF-MIB::ifName.5000000 = STRING: Loopback0
! FURHTER OUTPUT IS OMITTED
$ snmpwalk -v 3 -l authPriv -u Collector -a SHA -A SUPER_AUTH -x AES -X SUPER_PASS XR1 IF-MIB::ifTable
IF-MIB::ifIndex.2 = INTEGER: 2
IF-MIB::ifIndex.3 = INTEGER: 3
IF-MIB::ifIndex.4 = INTEGER: 4
IF-MIB::ifIndex.5 = INTEGER: 5
IF-MIB::ifIndex.6 = INTEGER: 6
IF-MIB::ifIndex.7 = INTEGER: 7
IF-MIB::ifIndex.8 = INTEGER: 8
! FURHTER OUTPUT IS OMITTED
$ snmpwalk -v 3 -l authPriv -u Collector -a SHA -A SUPER_AUTH -x AES -X SUPER_PASS VX4 IF-MIB::ifXTable
IF-MIB::ifName.1 = STRING: lo
IF-MIB::ifName.2 = STRING: eth0
IF-MIB::ifName.3 = STRING: swp1
IF-MIB::ifName.4 = STRING: swp2
IF-MIB::ifName.5 = STRING: swp3
IF-MIB::ifName.6 = STRING: swp4
! FURHTER OUTPUT IS OMITTED
$ snmpwalk -v 3 -l authPriv -u Collector -a SHA -A SUPER_AUTH -x AES -X SUPER_PASS SR1 IF-MIB::ifXTable
IF-MIB::ifName.1 = STRING: system
IF-MIB::ifName.2 = STRING: noc_1/1/c1/1
IF-MIB::ifName.1610899520 = STRING: 1/1/c1
IF-MIB::ifName.1610899521 = STRING: 1/1/c1/1
IF-MIB::ifName.1610899522 = STRING: 1/1/c1/2
! FURHTER OUTPUT IS OMITTED
All the network functions from our service provider fabric (PE routers, P router, CE routers) have responded to SNMP requests, so we can states that the configuration is successfully done.
Upgrade of Service Provider Fabric to incorporate SNMP version 3 configuration
Like with all previous parts of the Service Provider Fabric universe, configuration of SNMP on Arista EOS, Cisco IOS XE, Cumulus Linux and Nokia (Alcatel-Lucent) SR OS is automated. The SNMP details are added to “{{ inventory_hostname }}_underlay.yml” configuration file, like the example below:
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
---
! OUTPUT IS OMITTED
nms:
- id: 1
type: snmp
snmp:
version: 3
engineID: "010000000033"
security:
type: authPriv
auth: sha
priv: aes
view:
- iso
...
The user credentials for SNMP is stored in separated file (you can make it Ansible vault for better security if you wish):
2
3
4
5
6
7
---
snmp_credentials:
user: Collector
auth_pass: SUPER_AUTH
priv_pass: SUPER_PASS
...
The configuration itself is stored in newly created role “roles/all_snmp”, where you can find the details. Frankly, it’s just very easy template and per-vendor CLI module.
The execution of this role is done using standard playbook for service provider fabric or for customer equipment:
2
3
4
$ ansible-playbook customer_equipment.yml --inventory=hosts --tags=nms_snmp
The final configuration you can find on my GitHub page.
Lessons learned
I’ve observed interesting behavior of “sros_config” Ansible module in case using with SR OS 16.0.R* version. The behavior is such that even having failures in CLI it sends back “changed” code, not “failed”. This makes the playbook being executed further and report ultimate success, though it isn’t.
So, beware by troubleshooting and check templated config in CLI to verify it’s correct.
Conclusion
Probably in some time model-driven telemetry will overtake the main role for network monitoring from SNMP, and that would be good achievement given that all the vendors implement same YANG model (like OpenConfig (link)) or any other, but common model. In the example above we have configured Arista EOS, Cisco IOS XR, Cumulus Linux and Nokia (Alcatel-Lucent) SR OS for monitoring using SNMP version 3 and collected information using standard ISO tree, which is implemented in all the vendors. So, monitoring and unification is done. The next time we’ll speak about how to make use of this SNMP version 3 monitoring with visualization using the best available today tools.
Take care and good bye!
Support us
P.S.
If you have further questions or you need help with your networks, I’m happy to assist you, just send me message. Also don’t forget to share the article on your social media, if you like it.
BR,
Anton Karneliuk
engine-id configuration in Cumulus Linux is done via the net add snmp-server command. just add “engine-id ” parameter. It is (now) documented under the link you already posted:
https://docs.cumulusnetworks.com/display/DOCS/Simple+Network+Management+Protocol+%28SNMP%29+Monitoring
Hi Gerold,
Thanks a lot for pointing me to the proper source!
Cheers,
Anton