Network automation and telemetry demo with EOS devices
This repository has Network automation demo and telemetry demo with EOS devices
Table of content
We are using this Ubuntu VM for this workshop:
$ lsb_release -aNo LSB modules are available.Distributor ID: UbuntuDescription: Ubuntu 18.04.1 LTSRelease: 18.04Codename: bionic
Run these commands to update the VM and install tools
sudo apt-get updatesudo apt-get -y upgradesudo apt-get install tree snmp python3-pip build-essential libssl-dev libffi-dev python3-dev -ysudo apt install jqpip3 install napalm netmiko jsonrpclib-pelix pyang pyangbind ansible==2.9.15 influxdb pygnmi
Check
pip3 list
$ python3 -VPython 3.6.9
ansible --versionpyang --version
Then install also:
Check
docker versiondocker-compose versiongnmic version
Clone this repository
And then move to the local directory
cd automation_and_telemetry_demo
Configure all EOS devices for gNMI and SNMP and eAPI
snmp-server community public rosnmp-server vrf MGMT
username arista secret 0 arista
management api gnmitransport grpc defvrf MGMTprovider eos-native
management api http-commandsprotocol httpno shutdown
and verify
DC1-LEAF1A#sho management http-serverSSL Profile: noneFIPS Mode: NoQoS DSCP: 0Log Level: noneCSP Frame Ancestor: NoneTLS Protocols: 1.0 1.1 1.2VRF Server Status Enabled Services---------- --------------------- ----------------MGMT HTTPS: port 443 http-commands
DC1-LEAF1A#sho management api gnmiOcta: enabledEnabled: YesServer: running on port 6030, in MGMT VRFSSL Profile: noneQoS DSCP: none
no need to enable RESTCONF and NETCONF for this demo
DC1-LEAF1A#sho management api restconfEnabled: NoServer: Not yet runningSSL Profile: noneQoS DSCP: none
DC1-LEAF1A#sho management api netconfEnabled: NoServer: Not yet runningDC1-LEAF1A#
Netmiko is a python library to simplify SSH connections to network devices.
So we can use it even when API is disabled on EOS devices (default). As example we can use it to enable API on EOS devices.
From the root of this repository, move to the netmiko directory
cd netmiko
python3 collect_show_commands.pymore "show version.txt"more "show ip interface brief.txt"
more config.txtpython3 configure_from_file.py
Once the API is enabled, the switch accepts HTTP(S) requests containing a list of EOS commands, and responds with machine-readable output serialized in JSON (served over HTTP or HTTPS).
From the root of this repository, move to the eAPI directory
cd eapi
python3 test1.pypython3 test2.py
We will use the ansible module eos_command and eAPI to run show commands on EOS devices.
From the root of the repository, move to the Ansible directory
cd ansible
Update the inventory.yml file
Update the variables group_vars and host_vars directories
ansible-playbook playbooks/print_version_and_models.yml
show commands from the devicesUpdate the list of show commands you want to collect (this is an ansible variable currently defined in the group_vars directory) and execute this playbook:
ansible-playbook playbooks/snapshots.yml
The output of the show commands is saved in the directory snaphots
tree snapshots
To run all the tests (NTP, LLDP, interfaces state, temperature, …):
ansible-playbook playbooks/tests.yml
This will generate this markdown report and this CSV report
ls reportsmore reports/POC-state.mdmore reports/POC-state.csv
To run all only some tests, use ansible tags.
Examples:
ansible-playbook playbooks/tests.yml --tags lldpansible-playbook playbooks/tests.yml --tags "hardware, ntp, reload_cause, arbgp"
pyang is a python program.
We can use it to:
pip3 freeze | grep pyang
We need YANG files.
From the root of this repository:
git clone https://github.com/openconfig/public.git
ls public
cp public/release/models/*.yang yang_modules/.cp -R public/release/models/*/*.yang yang_modules/.cp public/third_party/ietf/*.yang yang_modules/.
It has all the YANG files published on the OpenConfig repository
cd yang_modules/ls
pyang openconfig-bgp.yangpyang openconfig-interfaces.yang
A YANG module can be translated into an XML syntax called YIN. The translated module is called a YIN module. The YANG and YIN formats contain equivalent information using different notations: YIN is YANG in XML. A YANG module can be translated into YIN syntax without losing any information.
Example (openconfig-bgp.yin is the YIN equivalent of openconfig-bgp.yang)
pyang openconfig-bgp.yang -f yin -o openconfig-bgp.yinls *.yin
pyang openconfig-interfaces.yang -f treepyang openconfig-interfaces.yang -f tree --tree-path=/interfaces/interface/statepyang openconfig-interfaces.yang -f tree --tree-depth=4
pyang openconfig-bgp.yang -f tree --tree-path=/bgp/neighbors --tree-depth=4pyang openconfig-bgp.yang -f tree --tree-path=/bgp/neighbors/neighbor/configpyang openconfig-bgp.yang -f tree --tree-path=/bgp/neighbors/neighbor/state --tree-depth=5pyang openconfig-bgp.yang -f tree --tree-path=/bgp/neighbors/neighbor/afi-safis --tree-depth=6
pyang openconfig-network-instance.yang -f tree --tree-depth=4pyang openconfig-network-instance.yang -f tree --tree-path=/network-instances/network-instance/protocols/protocol/bgp --tree-depth=6pyang openconfig-network-instance.yang -f tree --tree-path=/network-instances/network-instance/protocols/protocol/isis --tree-depth=6
PyangBind is a pyang plugin.
It generates Python classes from a YANG module: It converts YANG module into a Python module, such that Python can be used to generate data which conforms with the data model defined in YANG.
pip3 freeze | grep pyang
From the root of this repository:
cd yang_modules/
pyang --plugindir $HOME/.local/lib/python3.6/site-packages/pyangbind/plugin/ -f pybind -o oc_bgp.py openconfig-bgp.yang
The above command generated the python module oc_bgp.py from the openconfig-bgp.yang file
ls oc_bgp.py
The file pyangbind_demo.py uses the new python module oc_bgp.py to generate this OpenConfig configuration file
more pyangbind_demo.pypython3 pyangbind_demo.py
more ../gnmi/test.json
This configuration will be loaded later on a switch using the gNMI Set RPC
We will use gNMIc (an open source gNMI client)
gnmic version
From the root of this repository, move to the gNMI directory
cd gnmi/
Lets use the following gNMI RPC: Capabilities, Get, Set, Subscribe.
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure capabilities
Retrieve a snapshot for a path
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "/interfaces/interface[name=Ethernet2]/config/description"gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "/interfaces/interface[name=Ethernet1]/config/enabled"
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "/network-instances/network-instance[name=default]/protocols/protocol[name=BGP]/bgp"gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path '/network-instances/network-instance[name=default]/protocols/protocol[name=BGP]/bgp/neighbors'gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "network-instances/network-instance[name=default]/protocols/protocol[identifier=BGP]/bgp[afi-safi-name=IPV4_UNICAST]"gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "network-instances/network-instance[name=default]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=172.31.255.8]/afi-safis/afi-safi"
The Set RPC is used to modify states.
The SetRequest message uses the following fields:
gnmic -a 10.73.1.107:6030 --insecure -u arista -p arista get --path "/interfaces/interface[name=Ethernet1]/config/description"gnmic -a 10.73.1.107:6030 --insecure -u arista -p arista set --update-path "/interfaces/interface[name=Ethernet1]/config/description" --update-value "gnmi-example"gnmic -a 10.73.1.107:6030 --insecure -u arista -p arista get --path "/interfaces/interface[name=Ethernet1]/config/description"sh run int et1
gnmic -a 10.73.1.107:6030 --insecure -u arista -p arista set --update "/interfaces/interface[name=Ethernet3]/config/enabled:::bool:::false"gnmic -a 10.73.1.107:6030 --insecure -u arista -p arista get --path "/interfaces/interface[name=Ethernet3]/config/enabled"sh run int et3
gnmic -a 10.73.1.117:6030 --insecure -u arista -p arista get --path '/network-instances/network-instance[name=default]/protocols/protocol[name=BGP]/bgp'
sh run sec bgp
more test.json
gnmic -a 10.73.1.117:6030 --insecure -u arista -p arista set --replace-path '/network-instances/network-instance[name=default]/protocols/protocol[name=BGP]/bgp' --replace-file test.jsongnmic -a 10.73.1.117:6030 --insecure -u arista -p arista get --path '/network-instances/network-instance[name=default]/protocols/protocol[name=BGP]/bgp'
sh run sec bgp
Request to the target to stream values for an OpenConfig path
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path '/interfaces/interface[name=Ethernet1]/state/counters'gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path '/network-instances/network-instance/protocols/protocol/bgp/'gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path '/network-instances/network-instance/protocols/protocol/bgp/neighbors/neighbor[neighbor-address=::133:0:0:2]/state'gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path '/network-instances/network-instance[name=default]/protocols/protocol/bgp/neighbors/neighbor[neighbor-address=172.31.255.8]/state'gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path '/network-instances/network-instance[name=Tenant_B_WAN_Zone]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=10.255.254.5]/state'
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --stream-mode "sample" --sample-interval "5s" --path '/network-instances/network-instance[name=default]/protocols/protocol/bgp/neighbors/neighbor[neighbor-address=172.31.255.8]/state'
Request to the target to stream values for an EOS native path
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure sub --path "eos_native:/Sysdb/routing/bgp/export/"
Get an EOS show command via gNMI
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "cli:/show version"
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure get --path "cli:/show ip route summary" | jq '.[0].updates[0].values."show ip route summary".totalRoutes'
The above RPC works if the device has this YANG file
You can check this using the Capabilities RPC:
gnmic -a 10.73.1.107:6030 -u arista -p arista --insecure capabilities | grep arista-cli
For more examples about EOS commands and gNMI you can refer to this gist
ls -lamore .gnmic.yml
then
gnmic --config .gnmic.yml subscribe
or
gnmic subscribe
then
more gnmi_output.txt
cd ../yang_modules/gnmic path --file openconfig-bgp.yanggnmic path --file openconfig-bgp.yang | wc -lgnmic path --file openconfig-bgp.yang --path-type gnmignmic path --file openconfig-bgp.yang --types
pyGNMI is a Python implementation of the gNMI client
From the root of this repository, move to the pygnmi directory
cd pygnmi
python3 capabilities.py
python3 get.py
python3 sub.py
python3 update.py
python3 delete.py
Telegraf is an open source collector written in GO.
Telegraf collects data and writes them into a database.
It is plugin-driven (it has input plugins, output plugins, …)
InfluxDB is an open source time series database written in GO.
Grafana is an open source tool used to visualize time series data.
It supports InfluxDB and other backends.
It runs as a web application.
It is written in GO.
A TIG stack uses:
This Telegraf uses the following plugins:
Use this telegraf fork in order to have Telegraf to overwrite the gnmi timestamp by its local time
more details https://gist.github.com/ksator/e36a1be086da6c2239c2c2c0eb9fe300
From the root of this repository:
git clone https://github.com/rski/telegrafcd telegrafmake docker-imagedocker images
We already tested gNMI.
Let’s test SNMP:
snmpwalk --versionsnmpwalk -v 2c -c public 10.73.1.107 .1.3.6.1.2.1.1.3.0
From the root of this repository, move to the TIG directory
cd TIG
Update the input.yml with the devices details:
vi input.yml
Execute this python script to generate:
python3 render.pymore docker-compose.ymlls telegraf.d/
docker-compose up -d
docker-compose psdocker psdocker images
docker logs telegraf
docker exec -it telegraf bash
root@d35fed5663c0:/# ls /etc/telegrafroot@d35fed5663c0:/# more /etc/telegraf/telegraf.confroot@d35fed5663c0:/# ls /etc/telegraf/telegraf.droot@d35fed5663c0:/# exit
Start an interactive session
docker exec -it influxdb bash
influx
List databases
SHOW DATABASES
Select a database
use arista
List measurements
SHOW MEASUREMENTS
Query ifcounters measurement
SHOW TAG KEYS FROM "ifcounters"
SHOW TAG VALUES FROM "ifcounters" with KEY = "device"SHOW TAG VALUES FROM "ifcounters" with KEY = "name" WHERE ("device" = 'leaf1')
SELECT * FROM "ifcounters" WHERE "device" = 'leaf1' ORDER BY DESC LIMIT 3SELECT "in_octets","out_octets", "name" FROM "ifcounters" WHERE "device" = 'leaf1' ORDER BY DESC LIMIT 3SELECT "in_octets","out_octets", "name" FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name"='Ethernet2' AND time >= now() - 120s)SELECT "in_octets","out_octets", "name" FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name"=~/Ethernet.*/ AND time >= now() - 120s) GROUP BY "name"SELECT mean("in_octets")*8 FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name" = 'Ethernet2' AND time >= now() - 10m)SELECT mean("in_octets")*8 FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name" = 'Ethernet2' AND time >= now() - 10m) GROUP BY time(1m)SELECT derivative(mean("in_octets"), 1s) *8 FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name" = 'Ethernet2' AND time >= now() - 10m) GROUP BY time(1m)SELECT derivative(mean("in_unicast_pkts"), 1s) FROM "ifcounters" WHERE ("device" = 'leaf1' AND "name" = 'Ethernet2') AND (time >= now() - 10m) GROUP BY time(1m)SELECT stddev("in_octets") FROM "ifcounters" WHERE ("device" = 'leaf1' AND ("name" = 'Ethernet1' OR "name" = 'Ethernet2') AND (time >= now() - 10m)) GROUP BY time(1m)SELECT derivative(stddev("out_octets"), 1s) / 8 FROM "ifcounters" WHERE ("device" =~ /lea.*/ AND "name" =~ /Ethernet[1|2]/ AND (time >= now() - 10m)) GROUP BY time(1m), "device"
Query openconfig_bgp measurement
SHOW TAG KEYS FROM "openconfig_bgp"
SHOW TAG VALUES FROM "openconfig_bgp" WITH KEY = "device"SHOW TAG VALUES FROM "openconfig_bgp" WITH KEY = "name"SHOW TAG VALUES FROM "openconfig_bgp" WITH KEY = "name" WHERE "device"='leaf1'SHOW TAG VALUES FROM "openconfig_bgp" WITH KEY = "neighbor_address" WHERE "device"='leaf1'SHOW TAG VALUES FROM "openconfig_bgp" WITH KEY = "neighbor_address" WHERE ("device"='leaf1' AND "name" = 'default')
SELECT LAST("neighbors/neighbor/state/session_state") FROM "openconfig_bgp" WHERE ("device"='leaf1' AND "neighbor_address" = '10.255.254.1')SELECT LAST("neighbors/neighbor/state/session_state") FROM "openconfig_bgp" WHERE ("device"='leaf1') GROUP BY neighbor_addressSELECT LAST("neighbors/neighbor/state/session_state") FROM "openconfig_bgp" WHERE ("device"='leaf1' AND "name" = 'default') GROUP BY neighbor_addressSELECT LAST("neighbors/neighbor/state/session_state") AS session_state FROM "openconfig_bgp" WHERE "name" = 'default' GROUP BY "device", "neighbor_address"SELECT "device", "neighbor_address", LAST("neighbors/neighbor/state/session_state") AS session_state FROM "openconfig_bgp" WHERE ("device"='leaf1' AND "neighbor_address" = '10.255.254.1')
SELECT "neighbors/neighbor/afi_safis/afi_safi/state/prefixes/received" FROM "openconfig_bgp" WHERE ("device" = 'leaf1' AND "neighbor_address"='10.255.254.1' AND "afi_safi_name"='IPV4_UNICAST') LIMIT 15SELECT mean("neighbors/neighbor/afi_safis/afi_safi/state/prefixes/sent") FROM "openconfig_bgp" WHERE ("device" = 'leaf1' AND "afi_safi_name" = 'IPV4_UNICAST' AND "name" = 'default') GROUP BY time(1m), "neighbor_address"SELECT mean("neighbors/neighbor/afi_safis/afi_safi/state/prefixes/sent") FROM "openconfig_bgp" WHERE ("device" = 'leaf1' AND "name" = 'default') GROUP BY time(1m), "neighbor_address", "afi_safi_name"
SELECT mean("neighbors/neighbor/state/messages/sent/UPDATE") FROM "openconfig_bgp" WHERE ("device" = 'leaf1' AND "name" = 'default' AND time >= now() - 10m) GROUP BY "neighbor_address",time(1m)
SELECT COUNT(*) FROM (SELECT LAST("neighbors/neighbor/config/neighbor_address") FROM "openconfig_bgp" GROUP BY "neighbor_address") GROUP BY "device"
exitexit
pip3 freeze | grep influxdb
from influxdb import InfluxDBClientinflux_client = InfluxDBClient('localhost',8086)influx_client.query('show databases')influx_client.query('show measurements', database='arista')points = influx_client.query("""SELECT "in_octets" FROM "ifcounters" WHERE ("device"='leaf1' AND "name"='Ethernet1') ORDER BY DESC LIMIT 3""", database='arista').get_points()for point in points:print(point['in_octets'])exit()
The datasource is already configured. It uses InfluxDB.
We loaded ready to use dashboards.
We loaded a plugin.
docker exec -it grafana bash
bash-5.0$ more /etc/grafana/provisioning/datasources/datasource.yamlbash-5.0$ more /etc/grafana/provisioning/dashboards/dashboards.yamlbash-5.0$ ls /var/tmp/dashboards/bash-5.0$ ls /var/lib/grafana/pluginsbash-5.0$ exit
We can now use the Grafana GUI.
The default username and password are admin/admin, but we changed them to arista/arista.
The default port is 3000.
We can use the dashboards we already loaded or we can create new dashborads querying influxDB.
docker-compose down
docker ps