iControl Apps - #07 - System Http Statistics

I was digging through the iControl interfaces the other day and it's still amazing to me, after being around since the beginning of the API, to find new and cool stuff hiding in there.  Today's find was lurking within the System.Statistics interface.  There are literally dozens of method calls to pull out all sorts of useful statistics and the one I picked for today is the System HTTP Statistics which are a rollup of all HTTP Profiles' statistics.  This application will build a report on everything you wanted to know from an HTTP level such as how many 5xx responses have been served to the number of bytes of compressed video and the RAM cache hits and misses.

Usage

The arguments for this application are the address, username, and password of the BIG-IP along with optional argument for the subset of the statistitcs to present.  This is declared in the top of the script with the following param statement.  There is also a Write-Usage function to display the arguments to the user.

param (
  $g_bigip = $null,
  $g_uid = $null,
  $g_pwd = $null,
  $g_subset = $null
);

Set-PSDebug -strict;

function Write-Usage()
{
  Write-Host "Usage: GlobalHttpStats.ps1 host uid pwd [all|http|compression|cache]";
  exit;
}

Initialization

As is with all of my PowerShell scripts, the initialization component will look to see if the iControlSnapIn is loaded into the current PowerShell session.  If not, the Add-PSSnapIn Cmdlet is called to add the snapin into the runtime.  Then a call to the Initialize-F5.iControl cmdlet to setup the connection to the BIG-IP.  If this succeeds, then a call to the Get-GlobalHttpStatistics function is called with the optional subset argument.

function Do-Initialize()
{
  if ( (Get-PSSnapin | Where-Object { $_.Name -eq "iControlSnapIn"}) -eq $null )
  {
    Add-PSSnapIn iControlSnapIn
  }
  $success = Initialize-F5.iControl -HostName $g_bigip -Username $g_uid -Password $g_pwd;
 
  return $success;
}

if ( ($g_bigip -eq $null) -or ($g_uid -eq $null) -or ($g_pwd -eq $null) )
{
  Write-Usage;
}

if ( Do-Initialize )
{
  Get-GlobalHttpStatistics $g_subset;
}
else
{
  Write-Error "ERROR: iControl subsystem not initialized"
}

Global HTTP Statistics

The System.Statistics.get_http_statistics() is the only method that's called here.  It takes no arguements and returns a System.Statistics structure containing the timestamp when the statistics were taken, along with a list of Common.Statistics structures containing the statistic type along with the 64bit value.

To pretty up the presentation, an empty System.Object was created named stat_obj.  The statistics list is iterated upon and each of the statistics that match the subset are added as a property to the stats_obj object using Add-Member Cmdlet.  This way I can use the PowerShell Format-List CmdLet to make a nice columnar display on the console.

You'll notice that all statistics are returned from the method call so the subset feature is purely cosmetic in allowing you to have a more concise output if that is what you desire.

Once all of the statistics have been iterated through, they are written to the console by passing the stats_obj object through the PowerShell pipeline to the Format-List Cmdlet.

function Get-GlobalHttpStatistics()
{
  param($subset = $null);
  if ( $subset -eq $null ) { $subset = "all"; }
 
  Write-Host "Global HTTP Statistics"
 
  $stat_obj = New-Object -TypeName System.Object;
 
  $SystemStatistics = (Get-F5.iControl).SystemStatistics.get_http_statistics();
  $t = Get-TimeFromTimeStamp $SystemStatistics.time_stamp;
  $Statistics = $SystemStatistics.statistics;
 
  $stat_obj | Add-Member -Type noteProperty -Name "Time Stamp" $t;
  $stat_obj | Add-Member -Type noteProperty -Name "--------------------------------" "";

  foreach($Statistic in $Statistics)
  {
    $val = Convert-To64Bit $Statistic.value.high $Statistic.value.low;
    $bDisplay = 0;
    switch ($Statistic.type)
    {
      "STATISTIC_HTTP_COOKIE_PERSIST_INSERTS" {
        $label = "Cookie Persist Header Inserts";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_2XX_RESPONSES" {
        $label = "Responses - 2xx";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_3XX_RESPONSES" {
        $label = "Responses - 3xx";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_4XX_RESPONSES" {
        $label = "Responses - 4xx";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_5XX_RESPONSES" {
        $label = "Responses - 5xx";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_TOTAL_REQUESTS" {
        $label = "Requests  - Total";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_GET_REQUESTS" {
        $label = "Requests  - HTTP GET";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_POST_REQUESTS" {
        $label = "Requests  - HTTP POST";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V9_REQUESTS" {
        $label = "Requests  - v0.9";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V10_REQUESTS" {
        $label = "Requests  - v1.0";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V11_REQUESTS" {
        $label = "Requests  - v1.1";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V9_RESPONSES" {
        $label = "Responses - v0.9";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V10_RESPONSES" {
        $label = "Responses - v1.0";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_V11_RESPONSES" {
        $label = "Responses - v1.1";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_MAXIMUM_KEEPALIVE_REQUESTS" {
        $label = "Requests  - Maximum Keepalive";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_BUCKET_1K_RESPONSES" {
        $label = "Responses - under 1k";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_BUCKET_4K_RESPONSES" {
        $label = "Responses - From 1-4k";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_BUCKET_16K_RESPONSES" {
        $label = "Responses - From 4-16k";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_BUCKET_32K_RESPONSES" {
        $label = "Responses - From 16-32k";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_BUCKET_64K_RESPONSES" {
        $label = "Responses - From 32-64k";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "http"))
      }
      "STATISTIC_HTTP_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_NULL_COMPRESSION_BYTES" {
        $label = "     Compression Bytes - Null";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_HTML_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - HTML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_HTML_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - HTML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_CSS_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - CSS";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_CSS_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - CSS";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_JS_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - JavaScript";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_JS_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - JavaScript";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_XML_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - XML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_XML_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - XML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_SGML_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - SGML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_SGML_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - SGML";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_PLAIN_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Plain";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_PLAIN_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Plain";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_OCTET_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Octet";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_OCTET_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Octet";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_IMAGE_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Image";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_IMAGE_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Image";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_VIDEO_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Video";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_VIDEO_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Video";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_AUDIO_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Audio";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_AUDIO_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Audio";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_OTHER_PRE_COMPRESSION_BYTES" {
        $label = " Pre Compression Bytes - Other";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_OTHER_POST_COMPRESSION_BYTES" {
        $label = "Post Compression Bytes - Other";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "compression"))
      }
      "STATISTIC_HTTP_RAM_CACHE_HITS" {
        $label = "RAM Cache - Hits";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_MISSES" {
        $label = "RAM Cache - Misses";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_TOTAL_MISSES" {
        $label = "RAM Cache - Total Misses";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_HIT_BYTES" {
        $label = "RAM Cache - Hit Bytes";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_MISS_BYTES" {
        $label = "RAM Cache - Miss Bytes";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_TOTAL_MISS_BYTES" {
        $label = "RAM Cache - Total Miss Bytes";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_SIZE" {
        $label = "RAM Cache - Size";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_COUNT" {
        $label = "RAM Cache - Count";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      "STATISTIC_HTTP_RAM_CACHE_EVICTIONS" {
        $label = "RAM Cache - Evictions";
        $bDisplay = (($subset -eq "all") -or ($subset -eq "cache"))
      }
      default {
        $label = $Statistic.type
        $bDisplay = 1;
      }
    }
    if ( $bDisplay -ne 0 )
    {
      $stat_obj | Add-Member -Type noteProperty -Name $label $val;
    }
  }
  $stat_obj | Format-List
}

Utility functions

Several utility functions are included as well that will do the 64 bit math conversion as well as converting the iControl TimeStamp structure to a .Net DateTime.  See the full example in the iControl CodeShare for the content of the Invoke-Inline function.

function Convert-To64Bit()
{
  param($high, $low);
  return Invoke-Inline "result.Add((Convert.ToUInt64($high)<<32) | (Convert.ToUInt64($low)));"
}

function Get-TimeFromTimeStamp()
{
  param ($TimeStamp);
  $dt = new-object -typename System.DateTime
  $dt = $dt.AddYears($TimeStamp.year-1).AddMonths($TimeStamp.month-1).AddDays($TimeStamp.day-1).AddHours($TimeStamp.hour).AddMinutes($TimeStamp.minute).AddSeconds($TimeStamp.second);
  return $dt;
}

Running the Application

The following example shows my somewhat idle BIG-IP's statistics.  While these are not very exciting, the numbers you are likely to see will be more interesting depending on the activity of your HTTP applicationsand whether you are using compression and/or caching.

PS C:\> .\GlobalHttpStats.ps1 bigip_address username password
Global HTTP Statistics

Time Stamp                          : 8/6/2008 12:57:01 PM
--------------------------------    :
Cookie Persist Header Inserts       : 0
Responses - 2xx                     : 80000
Responses - 3xx                     : 0
Responses - 4xx                     : 0
Responses - 5xx                     : 0
Requests  - Total                   : 80000
Requests  - HTTP GET                : 80000
Requests  - HTTP POST               : 0
Requests  - v0.9                    : 0
Requests  - v1.0                    : 80000
Requests  - v1.1                    : 0
Responses - v0.9                    : 0
Responses - v1.0                    : 0
Responses - v1.1                    : 80000
Requests  - Maximum Keepalive       : 1
Responses - under 1k                : 0
Responses - From 1-4k               : 80000
Responses - From 4-16k              : 0
Responses - From 16-32k             : 0
Responses - From 32-64k             : 0
 Pre Compression Bytes              : 0
Post Compression Bytes              : 0
     Compression Bytes - Null       : 0
 Pre Compression Bytes - HTML       : 0
Post Compression Bytes - HTML       : 0
 Pre Compression Bytes - CSS        : 0
Post Compression Bytes - CSS        : 0
 Pre Compression Bytes - JavaScript : 0
Post Compression Bytes - JavaScript : 0
 Pre Compression Bytes - XML        : 0
Post Compression Bytes - XML        : 0
 Pre Compression Bytes - SGML       : 0
Post Compression Bytes - SGML       : 0
 Pre Compression Bytes - Plain      : 0
Post Compression Bytes - Plain      : 0
 Pre Compression Bytes - Octet      : 0
Post Compression Bytes - Octet      : 0
 Pre Compression Bytes - Image      : 0
Post Compression Bytes - Image      : 0
 Pre Compression Bytes - Video      : 0
Post Compression Bytes - Video      : 0
 Pre Compression Bytes - Audio      : 0
Post Compression Bytes - Audio      : 0
 Pre Compression Bytes - Other      : 0
Post Compression Bytes - Other      : 0
RAM Cache - Hits                    : 0
RAM Cache - Misses                  : 0
RAM Cache - Total Misses            : 0
RAM Cache - Hit Bytes               : 0
RAM Cache - Miss Bytes              : 0
RAM Cache - Total Miss Bytes        : 0
RAM Cache - Size                    : 0
RAM Cache - Count                   : 0
RAM Cache - Evictions               : 0

Conclusion

This article illustrated how to pull out HTTP statistics that you probably didn't know were available to you.  There are several other interesting statistics available in the System.Statistics interface that I'll illustrate in future articles so be on the look out!

The entire script can be found in the iControl CodeShare under PsSystemHttpStats.