flutter>> pipeline-gnmi>> 返回
项目作者： cisco-ie

项目描述 ：
A Model-Driven Telemetry collector based on the open-source tool pipeline
高级语言： Go
项目地址: git://github.com/cisco-ie/pipeline-gnmi.git
创建时间： 2018-11-02T13:43:24Z
项目社区：https://github.com/cisco-ie/pipeline-gnmi
开源协议：Apache License 2.0
下载


pipeline-gnmi Go Report Card Build Status

NOTE: For a more recently developed collector with more output flexibility and support, please evaluate usage of the following Telegraf plugins for your use case: cisco_telemetry_mdt and cisco_telemetry_gnmi.

A Model-Driven Telemetry collector based on the open-source tool pipeline including enhancements and bug fixes.

pipeline-gnmi is a Model-Driven Telemetry (MDT) collector based on the open-source tool pipeline which has gNMI support and fixes for maintainability (e.g. Go modules) and compatibility (e.g. Kafka version support). It supports MDT from IOS XE, IOS XR, and NX-OS enabling end-to-end Cisco MDT collection for DIY operators.

The original pipeline README is included here for reference.

Usage

pipeline-gnmi is written in Go and targets Go 1.11+. Windows and MacOS/Darwin support is experimental.

1) pipeline-gnmi binaries may be downloaded from Releases
2) Built from source:

  1. git clone https://github.com/cisco-ie/pipeline-gnmi
  2. cd pipeline-gnmi
  3. make build

3) Acquired via go get github.com/cisco-ie/pipeline-gnmi to be located in $GOPATH/bin

Configuration

pipeline configuration support is maintained and detailed in the original README. Sample configuration is supplied as pipeline.conf.

gNMI Support

This project introduces support for gNMI.
gNMI is a standardized and cross-platform protocol for network management and telemetry. gNMI does not require prior sensor path configuration on the target device, merely enabling gRPC/gNMI is enough. Sensor paths are requested by the collector (e.g. pipeline). Subscription type (interval, on-change, target-defined) can be specified per path.

Filtering of retrieved sensor values can be done directly at the input stage through selectors in the configuration file,
by defining all the sensor paths that should be stored in a TSDB or forwarded via Kafka. Regular metrics filtering through metrics.json files is ignored and not implemented, due to the lack of user-friendliness of the configuration.

  1. [mygnmirouter]
  2. stage = xport_input
  3. type = gnmi
  4. server = 10.49.234.114:57777
  6. # Sensor Path to subscribe to. No configuration on the device necessary
  7. # Appending an @ with a parameter specifies subscription type:
  8. #   @x where x is a positive number indicates a fixed interval, e.g. @10 -> every 10 seconds
  9. #   @change indicates only changes should be reported
  10. #   omitting @ and parameter will do a target-specific subscriptions (not universally supported)
  11. #
  12. path1 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters@10
  13. #path2 = /interfaces/interface/state@change
  15. # Whitelist the actual sensor values we are interested in (1 per line) and drop the rest.
  16. # This replaces metrics-based filtering for gNMI input - which is not implemented.
  17. # Note: Specifying one or more selectors will drop all other sensor values and is applied for all paths.
  18. #select1 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters/packets-sent
  19. #select2 = Cisco-IOS-XR-infra-statsd-oper:infra-statistics/interfaces/interface/latest/generic-counters/packets-received
  21. # Suppress redundant messages (minimum hearbeat interval)
  22. # If set and 0 or positive, redundant messages should be suppressed by the server
  23. # If greater than 0, the number of seconds after which a measurement should be sent, even if no change has occured
  24. #heartbeat_interval = 0
  26. tls = false
  27. username = cisco
  28. password = ...

Kafka 2.x Support

This project supports Kafka 2.x by requiring the Kafka version (kafkaversion) to be specified in the config file stage. This is a requirement of the underlying Kafka library and ensures that the library is communicating with the Kafka brokers effectively.

  1. [kafkaconsumer]
  2. topic=mdt
  3. consumergroup=pipeline-gnmi
  4. type=kafka
  5. stage=xport_input
  6. brokers=kafka-host:9092
  7. encoding=gpb
  8. datachanneldepth=1000
  9. kafkaversion=2.1.0

Docker Environment Variables

This project has improved Docker support. The Dockerfile uses multi-stage builds and
builds Pipeline from scratch. The configuration file can now be created from environment variables directly,
e.g.

  1. PIPELINE_default_id=pipeline
  2. PIPELINE_mygnmirouter_stage=xport_input
  3. PIPELINE_mygnmirouter_type=gnmi

is translated into a pipeline.conf with following contents:

  1. [default]
  2. id = pipeline
  4. [mygnmirouter]
  5. stage = xport_input
  6. type = gnmi

If the special variable _password is used, the value is encrypted using the pipeline RSA key before being written to
the password option. Similarly _secret can be used, then the value is read from the file whose name is given as
value, encrypted using the pipeline RSA key and then written as password option. If the Pipeline RSA key is not
given or does not exist it is created upon creation of the container.

Additionally, existing replays of sensor data can be fed in efficiently using xz-compressed files.

Licensing

pipeline-gnmi is licensed with Apache License, Version 2.0, per pipeline.

Help!

For support, please open a GitHub Issue or email ie@cisco.com"">cisco-ie@cisco.com.

Special Thanks

Chris Cassar for implementing pipeline used by anyone interested in MDT, Steven Barth for gNMI plugin development, and the Cisco teams implementing MDT support in the platforms.