Retrieving Configuration Utility's stats data instead of polling

Hi Alex,

 

 

The performance graph data is stored in rrd data files under /var/rrd/. You can export them to another host and use rrdtool to process them:

 

 

http://oss.oetiker.ch/rrdtool/

 

 

Aaron

Thank you.

 

For those who are interested...

RRD database files are /var/rrd/. One such file is "connections":

[admin@foo-dc-6900-1:Active] rrd  cat /var/rrd/connections.info
 DO NOT DELETE OR MODIFY this autogenerated file.
 This file contains the contents of the RRDFILE section of the statsd.conf file used to generate the rrdfile: '/var/rrd/connections'
 To restore the /config/statsd.conf contents to match this rrdfile, replace the existing rrdfile entry for /var/rrd/connections with the contents of this file
RRDFILE "/var/rrd/connections" {
    DATASOURCE "sslcurclientconns"    { EXPR "sslcurclientconns" DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "sslcurserverconns"    { EXPR "sslcurserverconns" DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "ssltotnatclient"    { EXPR "ssltotnatclient"   DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "ssltotcomclient"    { EXPR "ssltotcomclient"   DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "ssltotnatserver"    { EXPR "ssltotnatserver"   DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "ssltotcomserver"    { EXPR "ssltotcomserver"   DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "curclientconns"    { EXPR "curclientconns"    DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "curserverconns"     { EXPR "curserverconns"    DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "totclientconns"     { EXPR "totclientconns"    DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "totserverconns"     { EXPR "totserverconns"    DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "pvacurclientconns"    { EXPR "pvacurclientconns" DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "pvacurserverconns"    { EXPR "pvacurserverconns" DSTYPE GAUGE  MIN 0 MAX UNKNOWN }
    DATASOURCE "pvatotclientconns"    { EXPR "pvatotclientconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "pvatotserverconns"    { EXPR "pvatotserverconns" DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "globalaccepts"     { EXPR "globalaccepts"     DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "globalconnects"     { EXPR "globalconnects"    DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    DATASOURCE "httprequests"     { EXPR "http_requests"     DSTYPE DERIVE MIN 0 MAX UNKNOWN }
    RRA { CF AVERAGE XFF 0.500000 STEPS 1 ROWS 1080 }     non-averaged, keep data for duration of (1080 samples ´300s=) 90hours
    RRA { CF AVERAGE XFF 0.500000 STEPS 3 ROWS 2880 }     averaged over 3 raw sample reads; keep data for duration (3´2880 samples ´300s=) 30 days
    RRA { CF AVERAGE XFF 0.500000 STEPS 6 ROWS 10080 }     averaged over 6 raw sample reads; keep data for duration (6´10080 samples ´300s=) 210 days
    RRA { CF AVERAGE XFF 0.500000 STEPS 60 ROWS 4320 }     averaged over 60 raw sample reads; keep data for duration (60´4320 samples ´300s=) 900 days
}
CRC 4177393880

for clarity, the re-formatting and commenting is my own - alas, I can't get this post to display it properly.

Those not familiar with RRDTool should realise that

1. the data that is kept in database is post-processed ("CDPs"), not raw (sampled) data;

2. this data is stored to display a certain view which is defined during original database creation and not alterable

- those RRAs definitions defined both- what's stored and retrievable.

A few ways to examine RRDs:

Show timestamps of first and last updates

 

[admin@m5F5:Active] rrd  rrdtool first connections | gawk '{print($0strftime(" - %a %b %d %H:%M:%S %Z %Y",$0))}'
1313559530 - Wed Aug 17 15:38:50 EST 2011
[admin@m5F5:Active] rrd  rrdtool last connections  | gawk '{print($0strftime(" - %a %b %d %H:%M:%S %Z %Y",$0))}' 
1313570325 - Wed Aug 17 18:38:45 EST 2011

Show most recent update of an RRD

[admin@m5F5:Active] rrd  rrdtool lastupdate connections
        sslcurclientconns    sslcurserverconns    ssltotnatclient    ssltotcomclient    ssltotnatserver    ssltotcomserver    curclientconns    curserverconns    totclientconns    totserverconns    pvacurclientconns    pvacurserverconns    pvatotclientconns    pvatotserverconns    globalaccepts    globalconnects    httprequests
1313570325:    0            0            0        0        0        0        66        66        18973315    18253872     0            0            0            0            1532570        1532129        1532286

 

 

Graph statistics over last 3600s (1 hours) into connections.png

[admin@m5F5:Active] rrd  rrdtool graph /tmp/connections.png --start -3600 DEF:outconns=connections:totclientconns:AVERAGE LINE1:outconns0000FF:"totclientconns"
481x168

A slightly cleaner (Easier? IMO anyway :) way may be to use iControl to get the stats...

 

 

System::Statistics::get_performance_table_statistics

 

System::Statistics::get_performance_table_list

 

System::Statistics::get_performance_graph_list

 

System::Statistics::get_performance_graph_csv_statistics

 

 

H

is there a way via statsd.conf to sent the data via UDP to another host, say a carbon server?

Hi, To have the data under /var/rrd, i'm using the command below:

 

 

rrdtool xport --start now-2592000 --end now-10 DEF:d1=memory:Rtotal:AVERAGE DEF:d2=memory:Rfree:AVERAGE DEF:d3=memory:Rtmmmemory:AVERAGE DEF:d4=memory:Rtmmused:AVERAGE DEF:d5=rollupcpu:Ruser:AVERAGE DEF:d6=rollupcpu:Rsystem:AVERAGE DEF:d7=rollupcpu:Ridle:AVERAGE DEF:d8=throughput:tput_bytes_in:AVERAGE DEF:d9=throughput:tput_bytes_out:AVERAGE XPORT:d1 XPORT:d2 XPORT:d3 XPORT:d4 XPORT:d5 XPORT:d6 XPORT:d7 XPORT:d8 XPORT:d9

 

 

For my daily KPI, i'm using the bash script below:

 

 

../..
F5_START=$(date -d `date -d 'yesterday' "+%Y-%m-%d"` +%s)
F5_END=$(date -d `date -d 'now' "+%Y-%m-%d"` +%s)
GREP_CMD=`which grep`;
SED_CMD=`which sed`;
AWK_CMD=`which awk`;
RRDTOOL_CMD=`which rrdtool`;

DATA=`$RRDTOOL_CMD xport --start $F5_START --end $F5_END DEF:d3=memory:Rtmmmemory:AVERAGE DEF:d4=memory:Rtmmused:AVERAGE DEF:d5=rollupcpu:Ruser:AVERAGE DEF:d6=rollupcpu:Rsystem:AVERAGE DEF:d7=rollupcpu:Ridle:AVERAGE DEF:d8=throughput:tput_bytes_in:AVERAGE DEF:d9=throughput:tput_bytes_out:AVERAGE XPORT:d3 XPORT:d4 XPORT:d5 XPORT:d6 XPORT:d7 XPORT:d8 XPORT:d9 | $GREP_CMD "" | $SED_CMD -e 's/<\/t>/;/g' | $SED_CMD -e 's/<\/v>/;/g' | $SED_CMD -e 's///g' | $SED_CMD -e 's/<\/v><\/row>//g' |$SED_CMD -e 's/ //g' | $GREP_CMD -v "NaN" | $AWK_CMD -F';' '{if (tput <= $7+$8) {date=$1;mem=$3*100/$2; cpu=($4+$5)*100/($4+$5+$6); tput=$7+$8}} END{printf "%.0f;%.2f;%.2f;%.2f",date,mem,cpu,tput}'`

date_a=$(date -d @`echo $DATA | awk -F';' '{print $1}'` "+%Y%m%d %H:%M:%S");
CPU_RATIO=`echo $DATA | awk -F';' '{print $3}'`;
MEM_RATIO=`echo $DATA | awk -F';' '{print $2}'`;
THROUGHPUT_TTAL_BYTES=`echo $DATA | awk -F';' '{print $4}'`;
THROUGHPUT_TTAL=$(awk "BEGIN {printf \"%.2f\",${THROUGHPUT_TTAL_BYTES}*8}");
THROUGHPUT_RATIO=$(awk "BEGIN {printf \"%.2f\",${THROUGHPUT_TTAL}*100/${F5_LICTPUT}}");

../..

 

Hope it helps.