cancel
Showing results for 
Search instead for 
Did you mean: 
Bart_Van_Bos
F5 Employee
F5 Employee

Introduction

This article explains how to use the Telemetry Streaming component (TS) of the Automation Tool chain (ATC) for integration with Grafana through StatsD and Graphite. To get more information on the Push Consumer supported by the F5 Networks ATC, more in particular the TS component, please refer to the official documentation on CloudDocs here.

BIG-IP Configuration

In order to configure the TS component of the ATC correctly for integration with Grafana, we will need to post the following JSON blob to your BIG-IP TS API endpoint at https://<BIG-IP-ADDRESS>:8443/mgmt/shared/telemetry/declare

{
 "class": "Telemetry",
 "MyTelemetrySystem": {
   "class": "Telemetry_System",
   "allowSelfSignedCert": true,
   "systemPoller": {
     "interval": 60
   }
 },
 "GraphiteConsumer": {
   "class": "Telemetry_Consumer",
   "type": "Graphite",
   "host": "10.0.0.55",
   "protocol": "http",
   "port": 80
 },
 "StatsdConsumer": {
  "class": "Telemetry_Consumer",
  "type": "Statsd",
  "host":"10.0.0.55",
  "protocol": "udp",
  "port": 8125
 },
 "MyTelemetryListener": {
   "class": "Telemetry_Listener",
   "port": 6514
 }
}

The above 4 JSON stanzas are the following

  • A Telemetry System class, that sets up the system poller. More information here.
  • Two Push Consumers classes, that will push the metrics or data externally. In this case to Graphite and StatsD. More information here.
  • A Telemetry Listener class, that sets an Event Listener )both TCP and UDP protocols) and can accept events in a specific format and process them. More information here

 

Note that in this example, Graphite and StatsD are running on the same host, because we used a docker container to host them as follows

# docker run -d \
  --name graphite \
  --restart=always \
  -p 80:80 \
  -p 2003-2004:2003-2004 \
  -p 2023-2024:2023-2024 \
  -p 8125:8125/udp \
  -p 8126:8126 \
  graphiteapp/graphite-statsd

Telemetry data

Let's have a look at the TS telemetry data being produced and send over to Graphite. StatsD is used for metrics, Graphite is being used for events

StatsD metrics

StatsD is supporting 3 main metric types: gauges, timers and counters. The TS StatsD integration is using Gauges. We can use Netcat to have a look at the format of these gauge based metrics

# echo "gauges" | nc 10.0.0.55 8126
{
 'statsd.timestamp_lag': 0,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.1-0.counters-bitsIn': 297895992,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.1-0.counters-bitsOut': 0,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.mgmt.counters-bitsIn': 248764520,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.mgmt.counters-bitsOut': 134973160,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.clientSideTraffic-bitsIn': 62854192,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.clientSideTraffic-bitsOut': 229153456,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.serverSideTraffic-bitsIn': 62432120,
 'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.serverSideTraffic-bitsOut': 228977008,
 ...

We can also see the same gauge metrics inside the Graphite admin UI

0151T000002dhKQQAY.png

The structure and path of this telemetry data is important when you create you own dashboards

 

Graphite events

As mentioned earlier, the TS Graphite integration uses events to send the data to Graphite. You can observe those events by going to the /events endpoint on your Graphite admin UI

0151T000002dhKOQAY.png

The details of such an event are as follows

0151T000002dhKNQAY.png

Grafana

In order to be able to use and display the data now collected in Graphite, one needs to set-up Graphite as a data source and import a Grafana dashboard that uses this data

 

Graphite data source

Let's add Graphite as a data source

0151T000002dhKMQAY.png

Grafana BIG-IP TS dashboard

Let's import an example dashboard that used the data available. This sample dashboard is also available in the Grafana dashboard collection online here

0151T000002dhKLQAY.png

This sample dashboard will make use of dashboard variables, so users can filter on parameters like Device (which BIG-IP), Tenant (which BIG-IP partition), Application, Virtual Server and Pool. For the sake of demonstration, there is also a filter for Profile

 

For more information and screenshots on the dashboard itself, refer to the Grafana website where the dashboard is downloadable. The dashboard contains separate rows for

  • application health status: 4xx and 5xx responses. You can add slow responses as well as a matter of excercise
  • device system statistics: CPU, memory, TTM traffic in/out, interface traffic in/out
  • virtual server traffic in/out and server connections
  • pool traffic in/out and server connections
  • members traffic in/out and server connections
  • profile details statistics

 

The variable queries used for this dashboard are as follows, based on the structure of the metrics data you will find in the Graphite admin UI

0151T000002dhKkQAI.png

Conclusion

In this article we have demonstrated how the F5 Automation Tool Chain, and more in particular also its Telemetry Streaming component, is a perfect match for integration into popular DevOps telemetry solutions. For a fully automated scenario, demonstrating the usage of Declarative Onboarding (DO), Application Services 3 (AS3) and Telemetry Streaming (TS) with automated Grafana integration, you can refer to the following Github repo.

Comments
Donamato
MVP
MVP

great, info. I have all this working fine with the exception of the virtual server clientside-curConns. This always show 0. Any advise on the issue would be greatly appreciated .

Bart_Van_Bos
F5 Employee
F5 Employee

It might be that the name of the metric has been changed (or that it has become available in a different branch of the whole tree of metrics). You can use graphite to browse the tree and verify if the path is still matching the one configured in the Grafana graph for this particular metric.

Donamato
MVP
MVP

the values graphite are 0.0691T00000BSKxhQAH.pngAs shown above, bitsOut show data but curConns for all virtual servers show 0

 

Bart_Van_Bos
F5 Employee
F5 Employee

This might be a SW issue.

 

You can create an issue on our Github repository for TS at https://github.com/F5Networks/f5-telemetry-streaming

 

PS: are you sure that the value is 0 (and not 1 for example, with a load generator using that same single connection)?

Donamato
MVP
MVP

thanks, I'll log an issue.

 

Yes, value is 0, although I can see connections in double figures on the F5.

Version history
Last update:
‎07-May-2020 08:48
Updated by:
Contributors