on 07-May-2020 08:48
This article explains how to use the Telemetry Streaming component (TS) of the Automation Tool chain (ATC) for integration with Grafana through StatsD and Graphite. To get more information on the Push Consumer supported by the F5 Networks ATC, more in particular the TS component, please refer to the official documentation on CloudDocs here.
In order to configure the TS component of the ATC correctly for integration with Grafana, we will need to post the following JSON blob to your BIG-IP TS API endpoint at https://<BIG-IP-ADDRESS>:8443/mgmt/shared/telemetry/declare
{
"class": "Telemetry",
"MyTelemetrySystem": {
"class": "Telemetry_System",
"allowSelfSignedCert": true,
"systemPoller": {
"interval": 60
}
},
"GraphiteConsumer": {
"class": "Telemetry_Consumer",
"type": "Graphite",
"host": "10.0.0.55",
"protocol": "http",
"port": 80
},
"StatsdConsumer": {
"class": "Telemetry_Consumer",
"type": "Statsd",
"host":"10.0.0.55",
"protocol": "udp",
"port": 8125
},
"MyTelemetryListener": {
"class": "Telemetry_Listener",
"port": 6514
}
}
The above 4 JSON stanzas are the following
Note that in this example, Graphite and StatsD are running on the same host, because we used a docker container to host them as follows
# docker run -d \
--name graphite \
--restart=always \
-p 80:80 \
-p 2003-2004:2003-2004 \
-p 2023-2024:2023-2024 \
-p 8125:8125/udp \
-p 8126:8126 \
graphiteapp/graphite-statsd
Let's have a look at the TS telemetry data being produced and send over to Graphite. StatsD is used for metrics, Graphite is being used for events
StatsD is supporting 3 main metric types: gauges, timers and counters. The TS StatsD integration is using Gauges. We can use Netcat to have a look at the format of these gauge based metrics
# echo "gauges" | nc 10.0.0.55 8126
{
'statsd.timestamp_lag': 0,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.1-0.counters-bitsIn': 297895992,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.1-0.counters-bitsOut': 0,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.mgmt.counters-bitsIn': 248764520,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.networkInterfaces.mgmt.counters-bitsOut': 134973160,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.clientSideTraffic-bitsIn': 62854192,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.clientSideTraffic-bitsOut': 229153456,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.serverSideTraffic-bitsIn': 62432120,
'f5telemetry.ip-10-0-0-130-eu-west-1-compute-internal.system.tmmTraffic.serverSideTraffic-bitsOut': 228977008,
...
We can also see the same gauge metrics inside the Graphite admin UI
The structure and path of this telemetry data is important when you create you own dashboards
As mentioned earlier, the TS Graphite integration uses events to send the data to Graphite. You can observe those events by going to the /events endpoint on your Graphite admin UI
The details of such an event are as follows
In order to be able to use and display the data now collected in Graphite, one needs to set-up Graphite as a data source and import a Grafana dashboard that uses this data
Let's add Graphite as a data source
Let's import an example dashboard that used the data available. This sample dashboard is also available in the Grafana dashboard collection online here
This sample dashboard will make use of dashboard variables, so users can filter on parameters like Device (which BIG-IP), Tenant (which BIG-IP partition), Application, Virtual Server and Pool. For the sake of demonstration, there is also a filter for Profile
For more information and screenshots on the dashboard itself, refer to the Grafana website where the dashboard is downloadable. The dashboard contains separate rows for
The variable queries used for this dashboard are as follows, based on the structure of the metrics data you will find in the Graphite admin UI
In this article we have demonstrated how the F5 Automation Tool Chain, and more in particular also its Telemetry Streaming component, is a perfect match for integration into popular DevOps telemetry solutions. For a fully automated scenario, demonstrating the usage of Declarative Onboarding (DO), Application Services 3 (AS3) and Telemetry Streaming (TS) with automated Grafana integration, you can refer to the following Github repo.
great, info. I have all this working fine with the exception of the virtual server clientside-curConns. This always show 0. Any advise on the issue would be greatly appreciated .
It might be that the name of the metric has been changed (or that it has become available in a different branch of the whole tree of metrics). You can use graphite to browse the tree and verify if the path is still matching the one configured in the Grafana graph for this particular metric.
the values graphite are 0.As shown above, bitsOut show data but curConns for all virtual servers show 0
This might be a SW issue.
You can create an issue on our Github repository for TS at https://github.com/F5Networks/f5-telemetry-streaming
PS: are you sure that the value is 0 (and not 1 for example, with a load generator using that same single connection)?
thanks, I'll log an issue.
Yes, value is 0, although I can see connections in double figures on the F5.