iRule stats formatter
Problem this snippet solves: When you have a load of iRule stats in text format from your F5 device and need to get them into a nicer format. The following Python 3 script takes in a text file in the following format: ------------------------------------------------------------------------------------------------------ Ltm::Rule Event: /web_testing/test_environment_rule:HTTP_RESPONSE ------------------------------------------------------------------------------------------------------ Priority 12 Executions Total 31686860 Failures 0 Aborts 0 CPU Cycles on Executing Average 404058 Maximum 10703959 Minimum 264201 (raw) ------------------------------------------------------------------------------------------------------ Ltm::Rule Event: /web_testing/test_environment_rule:HTTP_REQUEST ------------------------------------------------------------------------------------------------------ Priority 899 Executions Total 31686860 Failures 0 Aborts 0 CPU Cycles on Executing Average 404058 Maximum 10703959 Minimum 264201 Put through the following python script to output a CSV file for further data manipulation. How to use this snippet: Python3 script, to use run the following (can also add in '--o' to define an output file, if not will replace the file extension '.txt' with '.csv' by default): python statformating.py --i my_irule_stats.txt output will be something like Openning 'my_irule_stats.txt' Saving output csv to 'my_irule_stats.csv' Usage/help output: usage: statformating.py [-h] [--i INPUT] [--o OUTPUT] optional arguments: -h, --help show this help message and exit --i INPUT iRule Stats File input file name --o OUTPUT iRule Stats File output csv file name Code : import re import os import argparse def iruleStatsFormat(inputFile, outputFile): print('Openning \'{}\''.format(inputFile)) iruleStats = open(inputFile, 'rt').read() iruleStats = re.sub(r'[ ]{2,}', ' ', iruleStats) iruleStats = re.sub(r'\n\s\(raw\)\s{1,}', '', iruleStats) iruleStats = re.sub(r'[-]{2,}\n', '', iruleStats) iruleStats = re.sub(r'\n ', r'\n', iruleStats) iruleStats = re.sub(r'CPU Cycles on Executing\n', '', iruleStats) iruleStats = re.sub(r'Executions \n', '', iruleStats) iruleStats = re.sub(r'\nPriority (\d{1,})\nTotal (\d{1,})\nFailures (\d{1,})\nAborts (\d{1,})\nAverage (\d{1,})\nMaximum (\d{1,})\nMinimum (\d{1,})', r'\t\1\t\2\t\3\t\4\t\5\t\6\t\7', iruleStats) iruleStats = re.sub(r'Ltm::Rule Event: /(.*?)/(.*?):(.*?\t)', r'\1\t\2\t\3', iruleStats) iruleStats = re.sub(r'Ltm::Rule Event: (.*?):(.*?\t)', r'Common\t\1\t\2', iruleStats) iruleStats = re.sub(r'\n{2,}', r'\n', iruleStats) iruleStats = re.sub(r'\t', r',', iruleStats) print('Saving output csv to \'{}\''.format(outputFile)) with open(outputFile, 'wt') as f: print(iruleStats, file=f) if __name__=='__main__': parser = argparse.ArgumentParser() parser.add_argument("--i", dest='input', help="iRule Stats File input file name", type=str) parser.add_argument("--o", dest='output', help="iRule Stats File output csv file name", type=str, default="") args = parser.parse_args() if args.input and os.path.isfile(args.input): if not args.output: args.output = args.input[:-3] + 'csv' iruleStatsFormat(args.input, args.output) else: parser.print_help()
F5 Python SDK: How to get virtual server availability?
Due to my limited Python and SDK experience I am finding it difficult to easily obtain the availability state of a virtual server. Given a 'virtual' object, what is the best/easiest way to return the value of that VS' 'Availability' state (offline, available, unknown, etc)? I can do the following: my_virtual = mgmt.tm.ltm.virtuals.virtual.load(name='some_virtual') stats = my_virtual.stats.load() Printing the stats.raw shows that there is a nested dictionary entry keyed 'status.availibityState' that appears to hold the value I'm looking for. Forgive my inexperience, but what is the easiest way to get the availability state of a virtual using the SDK? Perhaps a method I'm missing that retrieves the value above, or the value is easily obtained elsewhere. Appreciate any assitance!
Analytics: How to Expand the "Details" table to get more entries listed?
Version: 11.6.3 Navigation path: Statistics ›› Analytics : HTTP : Transactions The "Details" table lists only 10 items and then an "Overall" entry. The "Total Entries" would run into hundreds. How do I get more than ten entries listed?
How to reset pool statistics using Python API?
I am trying to duplicate the functionality that exists in the GUI of the "Reset" button that exists on the pool statistics page to clear the stats of all members in the current pool. I am able to GET these stats with Python, but cannot figure out how to reset stats such as serverside.bitsIn and serverside.bitsOut. Does anyone have any examples of how this can be accomplished?Play Ball!
...Oh Wait, Let Me Check the Stat-Cloud First! It is like a SAT question: Cincinnati Reds Billy Hamilton has a 10.83 foot lead off first base, can hit a top speed of 21.51 mph and clocked a jump of 0.49 seconds. If the Milwaukee Brewers catcher took 0.667 seconds to to get the ball out of his glove to throw to second and the ball is travelling at 78.81 mph, is Hamilton safe or out? A few weeks ago I wrote about the Internet of Sports, and can't believe I missed this one. But with the MLB playoffs in full gear, I didn't want this to slip through the IoT cracks. Sports analytics has been around for a while but never to this degree. Just like the NFL, Major League Baseball is equipping stadiums with technologies that can track moving players, flying baseballs and accurate throws. More than the RBIs, hits and stolen bases that appear on the back of trading cards, new technology (and software) also gathers stats like pop-fly pursuit range or average ground ball response time. Professional sports teams have always tracked their players' performance and often such milestones are included in the player's contract. Bonus for so many games played, or home runs hit or some other goal. With all this new detailed data, teams can adjust how they train, prepare for games and even player value for personnel moves like trades. For the 2014 season, only 3 stadiums (Mets, Brewers, Twins) had the new Field f/x (Sportvision Inc.) system but the league plans to have all 30 parks complete for the 2015 season. Field f/x can show data such as the angle of elevation of a batted ball, the highest point in its trajectory and the distance covered and top speed attained by a player attempting to field a ball. Of course all this can then be crunched for cool graphics during a replay. Cameras, sensors and software are all now part of the game. So are data centers, clouds and high speed links. All this data needs to be crunched somewhere and more often it is in a cloud environment. Add to that, the connection(s) to the fans and with the fans at the stadium. Levi's Stadium, for instance, has 1200 access points and an app that allows you to order food, watch instant replays and know which bathroom line is the shortest. Our sport stadiums are becoming data centers. Announcer: Welcome to Exclusive Sponsor Data Center Field! Home of the Hypertext Transfer Protocols. Undefeated at home this year, the Prots look to extend their record and secure home field throughout the playoffs. And if you were wondering, Hamilton was Safe. ps Related: New Baseball Season Brings Tech to Track Player Skills Major League Baseball brings new tech to the plate Baseball All-Stars’ Data Gets More Sophisticated With Field F/X The Internet of Sports Are You Ready For Some...Technology!! Is IoT Hype For Real? Technorati Tags: iot,things,baseball,sports,sensors,stats,big data,mlb,nfl,f5,silva Connect with Peter: Connect with F5:BIG-IP Interface Stats in Real Time with a TMSH Script
For the savants among us, calculating bits in and bits out over a delta from two snapshots of the interface counters is a walk in the park. For the rest of us, it's nice to have a tool to look at the current traffic load on an interface while working in the command line interface. This article will walk you through creating a TMSH script to do just that. Source Data You can get at interface data via snmp and icontrol, but is also available with the tmsh show net interface command. --------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out --------------------------------------------------------- 1.1 up 59.4T 5.0T 6.2G 2.4G 0 0 none Yep, that's data. But when you get to terabits, the dial doesn't move quite so quickly, so taking a diff every few seconds won't amount to much. Specifying the raw option on the show net interface command helps out in that regard. (raw) ----------------------------------------------------------------------------------------- Net::Interface Name Status Bits Bits Pkts Pkts Drops Errs Media In Out In Out ----------------------------------------------------------------------------------------- 1.1 up 59485486972968 5080727699544 6291600606 2488751052 0 0 none That's better, but a little more challenging to parse than adding the field-fmt option, which puts it in a nice key value pair list. The bits-in and bits-out counters are the focus of this script. net interface 1.1 { counters.bits-in 59486479580896 counters.bits-out 5080875828888 counters.drops-all 0 counters.errors-all 0 counters.pkts-in 6291722759 counters.pkts-out 2488812198 media-active none name 1.1 status up } Now that we have key value pairs, and already separated by whitespace, this is a simple extraction once we split the entire string by newline. % split $x "\n" net\ interface\ 1.1\ \{ \ { counters.bits-in 59500356294368} \ { counters.bits-out 5082163022832} \ { counters.drops-all 0} \ { counters.errors-all 0} \ { counters.pkts-in 6293231170} \ { counters.pkts-out 2489470246} \ { media-active none} \ { name 1.1} \ { status up} \} \ {} % lindex [split $x "\n"] 1 counters.bits-in 59500356294368 % lindex [split $x "\n"] 2 counters.bits-out 5082163022832 % lindex [lindex [split $x "\n"] 1] 1 59500356294368 % lindex [lindex [split $x "\n"] 2] 1 5082163022832 Now that the data is extracted in proper form, we can move on to the script! Goals & Workflow The goals for this script are simple: take the values from counters.bits-in and counters.bits-out from a specified interface and display them at a specified refresh interval. We'll get from goals to a script by first working through some workflow: The Script Since we need to get data from the user (interface and interval specifications), let's start with the standard input. We'll use the getFeedback proc below. proc getFeedback { question } { puts -nonewline $question flush stdout return [gets stdin] } This proc pulls is then used in the initial script setup as shown next. tmsh::clear_screen if { $tmsh::argc == 1 } { set int [getFeedback "Please enter the interface number (ie, 1.1): "] } else { set int [lindex $tmsh::argv 1] } set l1 [] set l2 [] set interval [getFeedback "Please enter refresh rate for the stats (in seconds): "] set delay [expr $interval * 1000] Here we see the screen has been cleared, and then if the only argument in the script initialization is the script itself, then we ask for the interface name. Otherwise, we take the second argument value and set it as the interface name. Then, we initialize the l1 and l2 variables as lists. Finally, we ask for the desired refresh interval and set that delay for the after command use as it's argument is in milliseconds, not seconds. Next, we need to go ahead and take the data and dump it into the l1 variable we initialized: lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l1 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] It looks a little scary, but this is an exact copy of the structure shown above in the Tcl shell except that we're using the TMSH command output instead of the static "x" variable we used to get the syntax necessary to extract the data. This results in l1 having a list with the bits-in and bits-out values in indexes 0 and 1 respectively. Now, the loop that allows this script to display the bit rate real time. while { true } { after $delay lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 1] 1] lappend l2 [lindex [lindex [split [tmsh::show net interface $int raw field-fmt] "\n"] 2] 1] tmsh::clear_screen set statsIn [expr ([lindex $l2 0] - [lindex $l1 0]) / $interval] set statsOut [expr ([lindex $l2 1] - [lindex $l1 1]) / $interval] puts "Interface\t\tInbound (bps)\t\tOutbound (bps)" puts "$int\t\t\t$statsIn\t\t\t$statsOut" set l1 $l2 unset l2 } This loop will continue until you break it with a ctrl-c. We start the loop condition with our specified delay, then do with the l2 variable what we did with the l1 variable: take a snapshot of the bits-in and bits-out of the interface. After again clearing the screen, now we take the delta of the new snapshot and the old snapshot, and divide by the interval to get the bits transferred in and out on that interface, per second. Next, we display that to the screen with the puts command. Finally, in order to maintain the latest snapshot for the next interval, we set the l2 data to the l1 variable and unset the l2 variable. And that's it. Not that complicated, right? Going Forward This is a very simple throwaway script that needs a lot of work to have "arrived." Error checking, extensibility, etc, are missing, and are all left to the reader to develop for those purposes. This met a very specific troubleshooting need in my environment, and I would be remiss if I didn't share. I'd love to see someone take on error checking, or maybe displaying the bitrates for all interfaces if none is specified, or going a step further, summarizing all interfaces per vlan and showing vlan bitrates. Any takers? The script in its entirety is here in the TMSH codeshare.You Got a Minute?
Like most of us, I try to read the entire internet on a daily basis but for some reason, these slipped through. They both came out in 2011 and I am sure the numbers have changed in many cases. For instance, the graphic shows 70+ domains registered every minute and for Sept 3 (thus far for today), on average 78 per minute have been registered. Yet for twitter, the chart indicates 320 new accounts per minute but my look up today, if my math is correct, shows 94 new twitter accounts every minute but with 546,000 (vs. 98,000+) tweets per minute today. Regardless, the somewhat, slightly dated info is still mind boggling and it is always fun to see historical data. Things that happen on the Internet every 60 Seconds circa 2011. And the products we use: ps Related: 60 Seconds – Things That Happen On Internet Every Sixty Seconds [Infographic] Technorati Tags: data,60 seconds,stats,internet,web,social media,application delivery,silva Connect with Peter: Connect with F5:
