cancel
Showing results for 
Search instead for 
Did you mean: 

Retrieving Configuration Utility's stats data instead of polling

epaalx
Cirrus
Cirrus
Hello F5 users,

 

I'd like to examine (raw) data used by Configuration Utility's stats that it displays (in Performance tab). I'd appreciate if someone can give the path for these files (- I'm hoping they're in ASCII).

 

The intention is to reuse data instead of polling LTM's SNMP Agent.

 

Regards,

 

Alex

 

 

6 REPLIES 6

hooleylist
Cirrostratus
Cirrostratus
Hi Alex,

 

 

The performance graph data is stored in rrd data files under /var/rrd/. You can export them to another host and use rrdtool to process them:

 

 

http://oss.oetiker.ch/rrdtool/

 

 

Aaron

epaalx
Cirrus
Cirrus
Thank you.

 

epaalx
Cirrus
Cirrus
For those who are interested...

 

 

RRD database files are /var/rrd/. One such file is "connections":

 

 

[admin@foo-dc-6900-1:Active] rrd cat /var/rrd/connections.info DO NOT DELETE OR MODIFY this autogenerated file. This file contains the contents of the RRDFILE section of the statsd.conf file used to generate the rrdfile: '/var/rrd/connections' To restore the /config/statsd.conf contents to match this rrdfile, replace the existing rrdfile entry for /var/rrd/connections with the contents of this file RRDFILE "/var/rrd/connections" { DATASOURCE "sslcurclientconns" { EXPR "sslcurclientconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "sslcurserverconns" { EXPR "sslcurserverconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "ssltotnatclient" { EXPR "ssltotnatclient" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "ssltotcomclient" { EXPR "ssltotcomclient" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "ssltotnatserver" { EXPR "ssltotnatserver" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "ssltotcomserver" { EXPR "ssltotcomserver" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "curclientconns" { EXPR "curclientconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "curserverconns" { EXPR "curserverconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "totclientconns" { EXPR "totclientconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "totserverconns" { EXPR "totserverconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "pvacurclientconns" { EXPR "pvacurclientconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "pvacurserverconns" { EXPR "pvacurserverconns" DSTYPE GAUGE MIN 0 MAX UNKNOWN } DATASOURCE "pvatotclientconns" { EXPR "pvatotclientconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "pvatotserverconns" { EXPR "pvatotserverconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "globalaccepts" { EXPR "globalaccepts" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "globalconnects" { EXPR "globalconnects" DSTYPE DERIVE MIN 0 MAX UNKNOWN } DATASOURCE "httprequests" { EXPR "http_requests" DSTYPE DERIVE MIN 0 MAX UNKNOWN } RRA { CF AVERAGE XFF 0.500000 STEPS 1 ROWS 1080 } non-averaged, keep data for duration of (1080 samples ´300s=) 90hours RRA { CF AVERAGE XFF 0.500000 STEPS 3 ROWS 2880 } averaged over 3 raw sample reads; keep data for duration (3´2880 samples ´300s=) 30 days RRA { CF AVERAGE XFF 0.500000 STEPS 6 ROWS 10080 } averaged over 6 raw sample reads; keep data for duration (6´10080 samples ´300s=) 210 days RRA { CF AVERAGE XFF 0.500000 STEPS 60 ROWS 4320 } averaged over 60 raw sample reads; keep data for duration (60´4320 samples ´300s=) 900 days } CRC 4177393880

 

 

for clarity, the re-formatting and commenting is my own - alas, I can't get this post to display it properly.

 

Those not familiar with RRDTool should realise that

 

1. the data that is kept in database is post-processed ("CDPs"), not raw (sampled) data;

 

2. this data is stored to display a certain view which is defined during original database creation and not alterable

 

- those RRAs definitions defined both- what's stored and retrievable.

 

 

A few ways to examine RRDs:

 

Show timestamps of first and last updates

 

[admin@m5F5:Active] rrd rrdtool first connections | gawk '{print($0strftime(" - %a %b %d %H:%M:%S %Z %Y",$0))}' 1313559530 - Wed Aug 17 15:38:50 EST 2011 [admin@m5F5:Active] rrd rrdtool last connections | gawk '{print($0strftime(" - %a %b %d %H:%M:%S %Z %Y",$0))}' 1313570325 - Wed Aug 17 18:38:45 EST 2011

 

 

Show most recent update of an RRD

 

[admin@m5F5:Active] rrd rrdtool lastupdate connections sslcurclientconns sslcurserverconns ssltotnatclient ssltotcomclient ssltotnatserver ssltotcomserver curclientconns curserverconns totclientconns totserverconns pvacurclientconns pvacurserverconns pvatotclientconns pvatotserverconns globalaccepts globalconnects httprequests 1313570325: 0 0 0 0 0 0 66 66 18973315 18253872 0 0 0 0 1532570 1532129 1532286

 

 

Graph statistics over last 3600s (1 hours) into connections.png

 

[admin@m5F5:Active] rrd rrdtool graph /tmp/connections.png --start -3600 DEF:outconns=connections:totclientconns:AVERAGE LINE1:outconns0000FF:"totclientconns" 481x168

Hamish
Cirrocumulus
Cirrocumulus
A slightly cleaner (Easier? IMO anyway 🙂 way may be to use iControl to get the stats...

 

 

System::Statistics::get_performance_table_statistics

 

System::Statistics::get_performance_table_list

 

System::Statistics::get_performance_graph_list

 

System::Statistics::get_performance_graph_csv_statistics

 

 

H

Randy_Black_758
Nimbostratus
Nimbostratus
is there a way via statsd.conf to sent the data via UDP to another host, say a carbon server?

JJMan
Nimbostratus
Nimbostratus

Hi, To have the data under /var/rrd, i'm using the command below:

 

 

rrdtool xport --start now-2592000 --end now-10 DEF:d1=memory:Rtotal:AVERAGE DEF:d2=memory:Rfree:AVERAGE DEF:d3=memory:Rtmmmemory:AVERAGE DEF:d4=memory:Rtmmused:AVERAGE DEF:d5=rollupcpu:Ruser:AVERAGE DEF:d6=rollupcpu:Rsystem:AVERAGE DEF:d7=rollupcpu:Ridle:AVERAGE DEF:d8=throughput:tput_bytes_in:AVERAGE DEF:d9=throughput:tput_bytes_out:AVERAGE XPORT:d1 XPORT:d2 XPORT:d3 XPORT:d4 XPORT:d5 XPORT:d6 XPORT:d7 XPORT:d8 XPORT:d9

 

 

For my daily KPI, i'm using the bash script below:

 

 

../..
F5_START=$(date -d `date -d 'yesterday' "+%Y-%m-%d"` +%s)
F5_END=$(date -d `date -d 'now' "+%Y-%m-%d"` +%s)
GREP_CMD=`which grep`;
SED_CMD=`which sed`;
AWK_CMD=`which awk`;
RRDTOOL_CMD=`which rrdtool`;

DATA=`$RRDTOOL_CMD xport --start $F5_START --end $F5_END DEF:d3=memory:Rtmmmemory:AVERAGE DEF:d4=memory:Rtmmused:AVERAGE DEF:d5=rollupcpu:Ruser:AVERAGE DEF:d6=rollupcpu:Rsystem:AVERAGE DEF:d7=rollupcpu:Ridle:AVERAGE DEF:d8=throughput:tput_bytes_in:AVERAGE DEF:d9=throughput:tput_bytes_out:AVERAGE XPORT:d3 XPORT:d4 XPORT:d5 XPORT:d6 XPORT:d7 XPORT:d8 XPORT:d9 | $GREP_CMD "" | $SED_CMD -e 's/<\/t>/;/g' | $SED_CMD -e 's/<\/v>/;/g' | $SED_CMD -e 's///g' | $SED_CMD -e 's/<\/v><\/row>//g' |$SED_CMD -e 's/ //g' | $GREP_CMD -v "NaN" | $AWK_CMD -F';' '{if (tput <= $7+$8) {date=$1;mem=$3*100/$2; cpu=($4+$5)*100/($4+$5+$6); tput=$7+$8}} END{printf "%.0f;%.2f;%.2f;%.2f",date,mem,cpu,tput}'`

date_a=$(date -d @`echo $DATA | awk -F';' '{print $1}'` "+%Y%m%d %H:%M:%S");
CPU_RATIO=`echo $DATA | awk -F';' '{print $3}'`;
MEM_RATIO=`echo $DATA | awk -F';' '{print $2}'`;
THROUGHPUT_TTAL_BYTES=`echo $DATA | awk -F';' '{print $4}'`;
THROUGHPUT_TTAL=$(awk "BEGIN {printf \"%.2f\",${THROUGHPUT_TTAL_BYTES}*8}");
THROUGHPUT_RATIO=$(awk "BEGIN {printf \"%.2f\",${THROUGHPUT_TTAL}*100/${F5_LICTPUT}}");

../..

 

Hope it helps.