Graphing Performance Data with bigsuds

I was asked internally about some performance graphs with python, and while I've done some work with python and iControl over the years, but I've yet to take on graphing. Most of the python scripts I've written output results to the console, so there was no need for fancy things like graphics. I was thinking of a web-based app, but decided to stick with a console app and use the matplotlib 2D plotting libary to generate the graphs. In addition to the normal modules I use for a python script (suds, bigsuds, getpass, argparse) interacting with the BIG-IP, there are few additions:

• base64 - used to decode the iControl stats data returned from BIG-IP
• time - used to reformat the date/time information returned from BIG-IP from epoch to date-time)
• csv - used to read the csv file data to pull out the time field to update it
• numpy - used to convert the string data from the csv file to arrays of float data for use with matplotlib.
• matplotlib - used for generating the graphs and for the graph date formatting of the timestamps. Two modules that are not imported directly in the script but are required nonetheless for matplotlib: dateutil and pyparsing, so you'll need to install those as well before getting started.

Thankfully, Joe wrote on performance graphs for many moons ago in his icontrol 101 #11 article. So the only real work I had to do was python-ize his code, and do my best to one-up his solution!

Grabbing the Data

This section of the main loop consists of importing all the modules we'll need, initializing the BIG-IP, asking the user what graph they'd like to see, and grabbing that data from the BIG-IP via an iControl call.

    import bigsuds as pc
    import getpass
    import argparse
    import matplotlib.pyplot as plt
    import matplotlib.dates as mdates
    import numpy as np
    import csv, time, base64

    parser = argparse.ArgumentParser()

    parser.add_argument("-s",  "--system", required=True)
    parser.add_argument("-u", "--username", required=True)

    args = vars(parser.parse_args())

    print "%s, enter your " % args['username'],
    upass = getpass.getpass()

    b = pc.BIGIP(args['system'], args['username'], upass)
    stats = b.System.Statistics

    graphs = stats.get_performance_graph_list()
    graph_obj_name = []
    count = 1
    print "Graph Options"
    for x in graphs:
        graph_obj_name.append(x['graph_name'])
        print "%d> %s, %s" % (count, x['graph_description'], x['graph_name'])
        count +=1

    num_select = int(raw_input("Please select the number for the desired graph data: "))

    graphstats = stats.get_performance_graph_csv_statistics(
        [{'object_name': graph_obj_name[num_select - 1],
          'start_time': 0,
          'end_time': 0,
          'interval': 0,
          'maximum_rows': 0}])

In the for loop, I'm just iterating through the data returned by the get_performance_graph_list method and printing it to the console so the user can select the graph they want. Next, I take the user's selection and graph the performance data for that graph with the get_performance_graph_csv_statistics method.

Storing and Manipulating the Data

Now that I have the data from the BIG-IP, I store it in a .csv file just in case the user would like to use it afterward. Before writing it to said .csv however, I need to b64decode the data as show below.

    statsFile = "%s_rawStats.csv" % graph_obj_name[num_select-1]
    f = open(statsFile, "w")
    f.write(base64.b64decode(graphstats[0]['statistic_data']))
    f.close()

This results in cvs data that looks like this:

timestamp,"Total Phys Memory","OS Used Memory","TMM Alloc Memory","TMM Used Memory","OS Used Swap"
1387226160,             nan,             nan,             nan,             nan,             nan
1387226400,4.1607127040e+09,4.1204910763e+09,1.7070817280e+09,6.0823478033e+08,2.4167403520e+08
1387226640,4.1607127040e+09,4.1187021824e+09,1.7070817280e+09,6.0832352860e+08,2.4183606613e+08

After storing the raw data, I re-open that file and read it in line by line, reformatting the timestamp and writing that to a new file:

    outputFile = "%s_modStats.csv" % graph_obj_name[num_select-1]
    with open(statsFile) as infile, open(outputFile, "w") as outfile:
        r = csv.reader(infile)
        w = csv.writer(outfile)
        w.writerow(next(r))
        for row in r:
            fl = float(row[0])
            row[0] = time.strftime('%Y-%m-%d %H:%M:%S', time.localtime(fl))
            w.writerow(row)

This results in updated csv data.

timestamp,Total Phys Memory,OS Used Memory,TMM Alloc Memory,TMM Used Memory,OS Used Swap
2013-12-16 14:36:00,             nan,             nan,             nan,             nan,             nan
2013-12-16 14:40:00,4.1607127040e+09,4.1204910763e+09,1.7070817280e+09,6.0823478033e+08,2.4167403520e+08
2013-12-16 14:44:00,4.1607127040e+09,4.1187021824e+09,1.7070817280e+09,6.0832352860e+08,2.4183606613e+08

I made sure that each graph would end up with its own set of csv files. I'm sure there's a smarter more pythonic way to handle this, but hey, it works.

Calling the Graph Functions

This is the simplest section of code, an if / elif chain! There is no switch statement in python like in Tcl, but there are other solutions available. The if / elif works just fine, so it remains.

    if num_select == 1:
        graph_memory(outputFile)
    elif num_select == 2:
        graph_cpu(outputFile)
    elif num_select == 3:
        graph_active_connections(outputFile)
    elif num_select == 4:
        graph_new_connections(outputFile)
    elif num_select == 5:
        graph_throughput(outputFile)
    elif num_select == 7:
        graph_http_requests(outputFile)
    elif num_select == 8:
        graph_ramcache_util(outputFile)
    elif num_select == 12:
        graph_active_ssl(outputFile)
    else:
        print "graph function not yet defined, please select 1, 2, 3, 4, 5, 7, 8, or 12."

The else is there for the myriad of graphs I have not generated, I'll leave that as an exercise for you!

Generating the Graphs

On my system, collecting the graphs from the get_performance_graph_list method results in 38 different graphs. Many of these graphs have different number of columns of data, so consolidating all these functions into one and working out the data passing and error correction was not an exercise I was willing to take on. So...lots of duplicate code for each graph function, but again, it works. The graphs I have built functions for are memory, cpu, active and new connections, throughput (bits), http requests, ramcache utilization, and active ssl connections. The function is essentially the same for all of them, the changes are related to number of plot lines, labels, etc. The memory graph function is below.

def graph_memory(csvdata):
    timestamp, total_mem, os_used, tmm_alloc, tmm_used, swap_used = np.genfromtxt(csvdata, delimiter=',',
                                                           skip_header=2, skip_footer=2, unpack=True,
                                                           converters= {0: mdates.strpdate2num('%Y-%m-%d %H:%M:%S')})
    fig = plt.figure()
    plt.plot_date(x=timestamp, y=total_mem, fmt='-')
    plt.plot(timestamp, total_mem, 'k-', label="Total Memory")
    plt.plot(timestamp, tmm_used, 'r-', label="TMM Used")
    plt.plot(timestamp, tmm_alloc, 'c-', label="Tmm Allocated")
    plt.plot(timestamp, os_used, 'g-', label="OS Used")
    plt.plot(timestamp, swap_used, 'b-', label="OS Used Swap")
    plt.legend(title="Context", loc='upper left', shadow=True)
    plt.title('Global Memory')
    plt.ylabel('GB')
    plt.show()

In this function, numpy (np.genfromtxt) is used to pull in the csv data to arrays that matplotlib can use. The rest of the function is just placing and formatting data for the graph.

Running the Script

C:\>python stats1.py -s 192.168.1.1 -u citizen_elah

citizen_elah, enter your Password:


Graph Options. Not all options supported at this time, please select only 1, 2, 3, 4, 5, 7, 8, or 12.


1> Memory Used, memory
2> System CPU Usage %, CPU
3> Active Connections, activecons
4> Total New Connections, newcons
5> Throughput(bits), throughput
6> Throughput(packets), throughputpkts
7> HTTP Requests, httprequests
8> RAM Cache Utilization, ramcache
9> Blade Memory Usage, b0memory
10> Active Connections, detailactcons1
11> Active Connections per Blade, detailactcons2
12> Active SSL Connections, detailactcons3
13> New Connections, detailnewcons1
14> New PVA Connections, detailnewcons2
15> New ClientSSL Profile Connections, detailnewcons3
16> New TCP Accepts/Connects, detailnewcons4
17> Client-side Throughput, detailthroughput1
18> Server-side Throughput, detailthroughput2
19> HTTP Compression Rate, detailthroughput3
20> SSL Transactions/Sec, SSLTPSGraph
21> GTM Requests and Resolutions, GTMGraph
22> GTM Requests, GTMrequests
23> GTM Resolutions, GTMresolutions
24> GTM Resolutions Persisted, GTMpersisted
25> GTM Resolutions Returned to DNS, GTMret2dns
26> GTM Requests per DNS type, GTMRequestBreakdownGraph
27> Active Sessions, act_sessions
28> New Sessions, new_sessions
29> Access Requests, access_requests
30> ACL Actions, acl_stats
31> Active Network Access Connections, NAActiveConnections
32> New Network Access Connections, NANewConnections
33> Network Access Throughput, NAthroughput
34> Rewrite Transactions, rewrite_trans
35> Rewrite Transaction Data, rewrite_bytes
36> GTM requests for IPv4 addresses, GTMarequests
37> GTM requests for IPv6 addresses, GTMaaaarequests
38> CPU Usage, blade0cpuset1


Please select the number for the desired graph data: 1

Wrapping Up

This was a fun walk through the matplotlib and the iControl interface to generate some graphs. The script in full is here in the codeshare. Happy coding!