Creating An iControl PowerShell Monitoring Dashboard With Google Charts

PowerShell is a very extensible scripting language and the fact that it integrates so nicely with iControl means you can do all sorts of fun things with it.  In this tech tip, I'll illustrate how to use just a couple of iControl method calls (3 to be exact) to create a load distribution dashboard for you desktop (with a little help from the Google Chart API).

Usage

The arguments for this application are the address, username, and password for your BIG-IP.

param (
  $g_bigip = $null,
  $g_uid = $null,
  $g_pwd = $null
);

The main control flow then looks for the input parameters and if they are not present, a usage message is displayed to the console indicating the required inputs.  If the connection info is specified, then the standard Do-Initialize function is called which will look to see if the iControl Snapin is installed and the Initialize-F5.iControl cmdlet is called to initialize the connection to the BIG-IP.  If an error occurs during the connection, then an error is logged and the application exits.

function Write-Usage()
{
  Write-Host "Usage: iControlDashboard.ps1 host uid pwd";
  exit;
}

function Do-Initialize()
{
  if ( (Get-PSSnapin | Where-Object { $_.Name -eq "iControlSnapIn"}) -eq $null )
  {
    Add-PSSnapIn iControlSnapIn
  }
  $success = Initialize-F5.iControl -HostName $g_bigip -Username $g_uid -Password $g_pwd;
  
  return $success;
}

#-------------------------------------------------------------------------
# Main Application Logic
#-------------------------------------------------------------------------
if ( ($g_bigip -eq $null) -or ($g_uid -eq $null) -or ($g_pwd -eq $null) )
{
  Write-Usage;
}

if ( Do-Initialize )
{
  Run-Dashboard
}
else
{
  Write-Error "ERROR: iControl subsystem not initialized"
  Kill-Browser
}

Global Variables

This appliction will make use of the Google Chart APIs to generate graphs and as such we need a browser to render it in.  Since we will be interacting with another process (in this case Internet Explorer), it is probably a good idea to gracefully shutdown if an error occurs.  A generic Exception Trap is created to log the error and shutdown the application properly.

Trap [Exception] {
  Write-Host $("TRAPPED: " + $_.Exception.GetType().FullName);
  Write-Host $("TRAPPED: " + $_.Exception.Message);
  Kill-Browser
  Exit;
}

A few global variables are used to make the app more configurable.  You can specify the title that comes up in the browsers header as well as the graph size for each report graph along with the chart type and polling interval.  I opted for a pie chart but other options are available that may or may not be to your liking.    At this point I go ahead and create a empty browser window and point it to the about:blank page giving us a context to manipulate the contents of the browser window.  I make the window visible and set it to full screen theatermode.

$g_title = "iControl PowerShell Dashboard";
$g_graphsize = "300x150";
$g_charttype = "p";
$g_interval = 5;

$g_browser = New-Object -com InternetExplorer.Application;
$g_browser.Navigate2("About:blank");
$g_browser.Visible = $true;
$g_browser.TheaterMode = $true;

Browser Control

The following functions are to control the browser and the data going into it.  The Refresh-Browser function takes as input the HTML to display.  The Document object is then accessed from the InternetExplorer.Application object and from there we can access the DocumentElement.  Then we set the InnerHTML to the input parameter $html_data and that is displayed in the browser window.

#-------------------------------------------------------------------------
# function Refresh-Browser
#-------------------------------------------------------------------------
function Refresh-Browser()
{
  param($html_data);
  
  if ( $null -eq $g_browser )
  {
    Write-Host "Creating new Browser"
    $g_browser = New-Object -com InternetExplorer.Application;
    $g_browser.Navigate2("About:blank");
    $g_browser.Visible = $true;
    $g_browser.TheaterMode = $true;
  }
  $docBody = $g_browser.Document.DocumentElement.lastChild;
  $docBody.InnerHTML = $html_data;
}

#-------------------------------------------------------------------------
# function Kill-Browser
#-------------------------------------------------------------------------
function Kill-Browser()
{
  if ( $null -ne $g_browser )
  { 
    $g_browser.TheaterMode = $false;
    $g_browser.Quit();
    $g_browser = $null;
  }
}

Main Application Loop

The main logic for this application is a little infinite loop where we call the Get-Data function, refresh the browser with the newly acquired report, and sleep for the configured interval until the next poll occurs.

function Run-Dashboard()
{
  while($true)
  {
    #Write-Host "Requesting data..."
    $html_data = Get-Data;
    Refresh-Browser $html_data;
    Start-Sleep $g_interval;
  }
}

Generating the Report

Here's where all the good stuff happens.  The Get-Data function will make a few iControl calls (LocalLB.Pool.get_list(), LocalLB.PoolMember.get_all_statistics(), and LocalLB.PoolMember.get_object_status()) and from that generate a HTML report with charts generated with the Google Chart API.

The local variable $html-data is used to store the resulting HTML data that will be sent to Internet Explorer for display and we start off the function by filling in the title and start of the report table.

Then the three previously mentioned iControl calls are made and the resulting values are stored in local varables for later reference.

The main loop here goes over each of the pools in the MemberStatisticsA local array variable.  A few hash tables and counters are created and then we loop over each pool member for the current pool we are processing.

Then entries are added to the local hash tables for total connections, current connections, bytes in, and status for later reference.  Also a sum of all the values for those hash tables are stored so we can calculate percentages later on.

At this point we will use the hash tables for generating the report.  Each numeric value is calculated into a percent and chart variables are created to contain the data as well as the labels for the generated pie charts.

Once all the number crunching has been performed the actual chart images are specified in the $chart_total, $chart_current, and $chart_bytes variables and the row in the report for the given pool is added to the $html_data variable.

function Get-Data()
{
  # TODO - get connection statistics
  $now = [DateTime]::Now;
  
  $html_data = "<html>
  <head>
  <title>$g_title</title>
  </head>
  <body>
  <center><h1>$g_title</h1><br/><h2>$now</h2></center>
  <center><table border='0' bgcolor='#C0C0C0'><tr><td><table border='0' cellspacing='0' bgcolor='#FFFFFF'>";
  
  $html_data += "
    <tr bgcolor='#C0C0C0'><th>Pool</th><th>Total Connections</th><th>Current Connections</th><th>Bytes In</th></tr>";
  
  $charts_total = "";
  $charts_current = "";
  $charts_bytes = "";
  
  $PoolList = (Get-F5.iControl).LocalLBPool.get_list() | Sort-Object;

  $MemberStatisticsA = (Get-F5.iControl).LocalLBPoolMember.get_all_statistics($PoolList)
  
  $MemberObjectStatusAofA = (Get-F5.iControl).LocalLBPoolMember.get_object_status($PoolList);
  
  # loop over each pool
  $i = 0;
  foreach($MemberStatistics in $MemberStatisticsA)
  {
    $hash_total = @{};
    $hash_current = @{};
    $hash_bytes = @{};
    $hash_status = @{};
    $sum_total = 0;
    $sum_current = 0;
    $sum_bytes = 0;
  
    $PoolName = $PoolList[$i];
    # loop over each member
    $MemberStatisticEntryA = $MemberStatistics.statistics;
    foreach($MemberStatisticEntry in $MemberStatisticEntryA)
    {
      $member = $MemberStatisticEntry.member;
      $addr = $member.address;
      $port = $member.port;
      $addrport = "${addr}:${port}";
      $StatisticA = $MemberStatisticEntry.statistics;
      $total = Extract-Statistic $StatisticA "STATISTIC_SERVER_SIDE_TOTAL_CONNECTIONS"
      [long]$sum_total += $total;
      $hash_total.Add($addrport, $total);
      
      $current = Extract-Statistic $StatisticA "STATISTIC_SERVER_SIDE_CURRENT_CONNECTIONS"
      $sum_current += $current;
      $hash_current.Add($addrport, $current);
      
      $bytes = Extract-Statistic $StatisticA "STATISTIC_SERVER_SIDE_BYTES_IN"
      [long]$sum_bytes += $bytes;
      $hash_bytes.Add($addrport, $bytes);
      
      $color = Extract-Status $MemberObjectStatusAofA[$i] $member;
      $hash_status.Add($addrport, $color);
    }
    $chd_t = "";
    $chd_c = "";
    $chd_b = "";
    $chl_t = "";
    $chl_c = "";
    $chl_b = "";
    $chdl_t = "";
    $chdl_c = "";
    $chdl_b = "";
    $tbl_t = "";
    $tbl_c = "";
    $tbl_b = "";
    
    # enumerate the total connections
    foreach($k in $hash_total.Keys)
    {
      $member = $k;
      $v_t = $hash_total[$k];
      $v_c = $hash_current[$k];
      $v_b = $hash_bytes[$k];
      $color = $hash_status[$k];
      
      $div = $sum_total;
      if ($div -eq 0 ) { $div = 1; }
      $p_t = ($v_t/$div)*100;

      $div = $sum_current;
      if ($div -eq 0 ) { $div = 1; }
      $p_c = ($v_c/$div)*100;
      
      $div = $sum_bytes;
      if ($div -eq 0 ) { $div = 1; }
      $p_b = ($v_b/$div)*100;
      
      if ( $chd_t.Length -gt 0 ) { $chd_t += ","; $chd_c += ","; $chd_b += ","; }
      $chd_t += $p_t;
      $chd_c += $p_c;
      $chd_b += $p_b;
      
      if ( $chl_t.Length -gt 0 )
      {
        $chl_t += "|"; $chl_c += "|"; $chl_b += "|";
        $chdl_t += "|"; $chdl_c += "|"; $chdl_b += "|";
      }
      $chl_t += "$member";
      $chl_c += "$member";
      $chl_b += "$member";
      $chdl_t += "$member - $v_t";
      $chdl_c += "$member - $v_c";
      $chdl_b += "$member - $v_b";
      
      #$alt_t += "($member,$v_t)";
      #$alt_c += "($member,$v_c)";
      #$alt_b += "($member,$v_b)";
      
      $tbl_t += "<tr><td bgcolor='$color'>$member</td><td align='right'>$v_t</td></tr>";
      $tbl_c += "<tr><td bgcolor='$color'>$member</td><td align='right'>$v_c</td></tr>";
      $tbl_b += "<tr><td bgcolor='$color'>$member</td><td align='right'>$v_b</td></tr>";
      
    }
    
    if ( $sum_total -gt 0 )
    {
      $chart_total = "<img src='http://chart.apis.google.com/chart?
chs=$g_graphsize
&amp;chd=t:$chd_t
&amp;cht=$g_charttype
&amp;chdl=$chl_t'
alt='Total Connections for pool $PoolName' />";
    }
    else
    {
      $chart_total = "";
    }
      
    if ( $sum_current -gt 0 )
    {
      $chart_current = "<img src='http://chart.apis.google.com/chart?
chs=$g_graphsize
&amp;chd=t:$chd_c
&amp;cht=$g_charttype
&amp;chdl=$chl_c'
alt='Current Connections for pool $PoolName' />";
    }
    else
    {
      $chart_current = "";
    }
      
    if ( $sum_bytes -gt 0 )
    {
      $chart_bytes = "<img src='http://chart.apis.google.com/chart?
chs=$g_graphsize
&amp;chd=t:$chd_b
&amp;cht=$g_charttype
&amp;chdl=$chl_b'
alt='Incoming Bytes for pool $PoolName' />";
    }
    else
    {
      $chart_current = "";
    }
    
    if ( $i -gt 0 )
    {
      $html_data += "<tr><td colspan='4'><hr/></td></tr>";
    }
      
    $html_data += "
      <tr><th nowrap='nowrap'>$PoolName</th>
        <td valign='bottom'>$chart_total<br/>
          <center><table border='1'><tr><th>Member</th><th>Value</th></tr>$tbl_t</table>
        </td>
        <td valign='bottom'>$chart_current<br/>
          <center><table border='1'><tr><th>Member</th><th>Value</th></tr>$tbl_c</table>
        </td>
        <td valign='bottom'>$chart_bytes<br/>
          <center><table border='1'><tr><th>Member</th><th>Value</th></tr>$tbl_b</table>
        </td>
      </tr>";
      
    $i++;
  }
  $html_data += "</table></td></tr></table></body></html>";
  return $html_data;
}

Utility Functions

It's always useful to extract common code into utility functions and this application is no exception.  In here I've got a Convert-To64Bit function that takes the high and low 32 bits of a 64 bit number and does the math to convert them into a native 64 bit value.

The Extract-Statistic function takes as input a Common.Statsistic Array along with a type to look for in that array.  It loops over the array of Statistic values and returns the 64 bit value of the match, if one is found.

And finally the Extract-Status function is used to look through the returned value from the LocalLB.PoolMember.get_object_status iControl method for a specific pool member.  This function returns a color to display in the generated HTML table, green for good, red for bad.  The only way a green will show up will be if both it's availability_status and enabled_status values are AVAILABILITY_STATUS_GREEN and ENABLED_STATUS_ENABLED respectively.

function Convert-To64Bit()
{
  param($high, $low);  
    
  $low = [Convert]::ToString($low,2).PadLeft(32,'0')  
  if($low.length -eq "64")  
  {  
    $low = $low.substring(32,32)  
  }  
     
  return [Convert]::ToUint64([Convert]::ToString($high,2).PadLeft(32,'0')+$low,2);  
}

function Extract-Statistic()
{
  param($StatisticA, $type);
  $value = -1;
  
  foreach($Statistic in $StatisticA)
  { 
    if ( $Statistic.type -eq $type )
    {
      $value = Convert-To64Bit $Statistic.value.high $Statistic.value.low;
      break;
    }
  }
  return $value;
}

function Extract-Status()
{
  param($MemberObjectStatusA, $IPPortDefinition);
  $color = "#FF0000";
  
  foreach($MemberObjectStatus in $MemberObjectStatusA)
  {
    if ( ($MemberObjectStatus.member.address -eq $IPPortDefinition.address) -and
         ($MemberObjectStatus.member.port -eq $IPPortDefinition.port) )
    {
      $availability_status = $MemberObjectStatus.object_status.availability_status;
      $enabled_status = $MemberObjectStatus.object_status.enabled_status;
      
      if ( ($availability_status -eq "AVAILABILITY_STATUS_GREEN") -and
           ($enabled_status -eq "ENABLED_STATUS_ENABLED" ) )
      {
        $color = "#00FF00";
      }
    }
  }
  return $color;
}

Running The Application

After running the application on the console, Internet Explorer will be created in Theater Mode (Full Screen) and will look something like this.  My system is somewhat inactive so you'll see that some of the charts are missing. This was by design in that charts with no data are not very informative.  Assuming you have traffic across all your pools, charts will be created.

Extending This Application

This application merely looks at load distribution and state for members within the pools.  It would be trivial to change or extend the types of charts presented.  iControl provides you with all the data you need to build your own monitoring dashboard regardless of the types of metrics you would like to keep an eye on.

For the full application, check out the PsiControlDashboard entry in the iControl CodeShare