iControl.cgi running amok

The sample code in the SDK queries stats for all virtuals within one call. Each set of statistics for each virtual can be quite large so querying that many virtuals in one shot could cause issues with server resources.

 

 

Give this code a try. I've added chunking of the requests and I think you'll see that your resource issues go away.

 

 

!/usr/bin/perl
----------------------------------------------------------------------------
 The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5
 Software Development Kit for iControl"; you may not use this file except in
 compliance with the License. The License is included in the iControl
 Software Development Kit.
 Software distributed under the License is distributed on an "AS IS"
 basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
 the License for the specific language governing rights and limitations
 under the License.
 The Original Code is iControl Code and related documentation
 distributed by F5.
 The Initial Developer of the Original Code is F5 Networks,
 Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2004 F5 Networks,
 Inc. All Rights Reserved.  iControl (TM) is a registered trademark of F5 Networks, Inc.
 Alternatively, the contents of this file may be used under the terms
 of the GNU General Public License (the "GPL"), in which case the
 provisions of GPL are applicable instead of those above.  If you wish
 to allow use of your version of this file only under the terms of the
 GPL and not to allow others to use your version of this file under the
 License, indicate your decision by deleting the provisions above and
 replace them with the notice and other provisions required by the GPL.
 If you do not delete the provisions above, a recipient may use your
 version of this file under either the License or the GPL.
----------------------------------------------------------------------------
use SOAP::Lite + trace => qw(method debug);
use SOAP::Lite;
use MIME::Base64;
BEGIN { push (@INC, ".."); }
use iControlTypeCast;
----------------------------------------------------------------------------
 Validate Arguments
----------------------------------------------------------------------------
my $sHost = $ARGV[0];
my $sPort = $ARGV[1];
my $sUID = $ARGV[2];
my $sPWD = $ARGV[3];
my $sCommand = $ARGV[4];
my $sArg1 = $ARGV[5];
my $sArg2 = $ARGV[6];
my $sProtocol = "https";
if ( ("80" eq $sPort) or ("8080" eq $sPort) )
{
$sProtocol = "http";
}
if ( ($sHost eq "") or ($sPort eq "") or ($sUID eq "") or ($sPWD eq "") )
{
&usage();
}
sub usage()
{
my ($sCmd) = @_;
print "Usage: LocalLBVSStats.pl host port uid pwd command [options]\n";
print "    -----------------------------------------------------------\n";
if ( ($sCmd eq "") or ($sCmd eq "get") )
{
print "    get      virtual server - Query the specified virtual server\n";
}
if ( ($sCmd eq "") or ($sCmd eq "list") )
{
print "    list                     - List the entire virtual server list\n";
}
exit();
}
----------------------------------------------------------------------------
 Transport Information
----------------------------------------------------------------------------
sub SOAP::Transport::HTTP::Client::get_basic_credentials
{
return "$sUID" => "$sPWD";
}
$VirtualServer = SOAP::Lite
-> uri('urn:iControl:LocalLB/VirtualServer')
-> proxy("$sProtocol://$sHost:$sPort/iControl/iControlPortal.cgi");
eval { $VirtualServer->transport->http_request->header
(
'Authorization' => 
'Basic ' . MIME::Base64::encode("$sUID:$sPWD", '')
); };
if ( $sCommand eq "get" )
{
&validateArgs("get", $sArg1);
&handle_get($sArg1);
}
elsif ( $sCommand eq "list" )
{
&handle_list();
}
else
{
&usage();
}
----------------------------------------------------------------------------
 validateArgs
----------------------------------------------------------------------------
sub validateArgs()
{
my ($cmd, @args) = (@_);
foreach $arg (@args)
{
if ( "" eq $arg )
{
&usage($cmd);
}
}
}
----------------------------------------------------------------------------
 checkResponse
----------------------------------------------------------------------------
sub checkResponse()
{
my ($soapResponse) = (@_);
if ( $soapResponse->fault )
{
print $soapResponse->faultcode, " ", $soapResponse->faultstring, "\n";
exit();
}
}
sub handle_get()
{
(@VirtualServerList) = @_;
if ( 0 == scalar(@VirtualServerList) )
{
return;
}
 Set the default chunk size
$chunk_size = 10;
 Set the time threshold per call (in seconds)
$time_threshold = 10; 
$total_size = scalar(@VirtualServerList);
for ($i=0; $i<$total_size; $i+=$last_chunk_size)
{
if ( ($total_size - $i) < $chunk_size )
{
$chunk_size = ($total_size - $i);
}
$last_chunk_size = $chunk_size;
 Make sure chunk array is empty
@chunk_list = ();
undef @chunk_list;
 fill new array
for ($j=0; $j<$chunk_size; $j++)
{
$val = @VirtualServerList[$i+$j];
push @chunk_list, $val
}
 Get Start Time
$start_time = time();
 Make calls
print "Querying $chunk_size elements\n";
&handle_getex(@chunk_list);
 Get Stop Time
$end_time = time();
$chunk_time = $end_time - $start_time;
print "Time this section: $chunk_time s.\n";
 increment total time
$total_time += ($chunk_time);
if ( $chunk_time < $time_threshold )
{
$chunk_size *= 2;
}
}
print "Total execution time: $total_time s.\n";
}
 
----------------------------------------------------------------------------
 get
----------------------------------------------------------------------------
sub handle_getex()
{
(@VirtualServerList) = @_;
if ( 0 == scalar(@VirtualServerList) )
{
return;
}
$soapResponse = $VirtualServer->get_statistics
(
SOAP::Data->name(virtual_servers => [@VirtualServerList])
);
&checkResponse($soapResponse);
$VirtualServerStatistics = $soapResponse->result;
$time_stamp = $VirtualServerStatistics->{"time_stamp"};
@VirtualServerStatisticEntry = @{$VirtualServerStatistics->{"statistics"}};
foreach $VirtualServerStatistic (@VirtualServerStatisticEntry)
{
$virtual_server = $VirtualServerStatistic->{"virtual_server"};
$name = $virtual_server->{"name"};
$address = $virtual_server->{"address"};
$port = $virtual_server->{"port"};
$protocol = $virtual_server->{"protocol"};
print "Virtual Server: '$name' ($address:$port)\n";
@StatisticList = @{$VirtualServerStatistic->{"statistics"}};
foreach $Statistic (@StatisticList)
{
$type = $Statistic->{"type"};
$value = $Statistic->{"value"};
$low = $value->{"low"};
$high = $value->{"high"};
$value64 = ($high<<32)|$low;
$time_stamp = $Statistic->{"time_stamp"};
print "--> $type : $value64\n";
}
}
}
----------------------------------------------------------------------------
 list
----------------------------------------------------------------------------
sub handle_list()
{
$soapResponse = $VirtualServer->get_list();
&checkResponse($soapResponse);
@VirtualServerList = @{$soapResponse->result};
&handle_get(@VirtualServerList);
}

 

 

Hopefully in future SDK's we'll have the samples updated to include chunking by default.

 

 

-Joe

That runs for a bit but then ends with "500 Internal Server Error at test.pl line 212" (test.pl being what I pasted your script as). It'll list about 10 entries before dying (though it says it's retrieving 20).

 

 

 

Then the following gets printed to /var/log/httpd/httpd_errors:

 

 

Nov 1 14:53:26 bigip02 fcgi-[17233]: [warn] FastCGI: server "/usr/local/www/iControl/iControlPortal.cgi" (pid 21348) terminated due to uncaught signal '11' (Segmentation fault)

 

Nov 1 14:53:26 bigip02 fcgi-[17233]: [warn] FastCGI: server "/usr/local/www/iControl/iControlPortal.cgi" has remained running for more than 30 seconds, its restart interval has been restored to 5 seconds

 

Nov 1 14:53:26 bigip02 fcgi-[17233]: [warn] FastCGI: server "/usr/local/www/iControl/iControlPortal.cgi" restarted (pid 21511)

 

Nov 1 14:53:26 bigip02 httpd[6174]: [error] [client 10.1.1.78] FastCGI: incomplete headers (0 bytes) received from server "/usr/local/www/iControl/iControlPortal.cgi"

Which version of BIG-IP are you running? If you are getting server errors with small requests, then this has to be something related to the release you are running. I've tested this script on v9.2 with 2000+ VS's with no issues.

 

 

-Joe

Sorry about the delay in the response (I didn't forget about you!). I've just been trying to reproduce it on my end and have yet to succeed.

 

 

Are you having any problems with any of the other samples in the SDK or is it just this one?

 

 

If it's just this one, then I'd try changing the chunk_size varaible in the sample and comment out the part where the chunk_size is incremented. That will force a request size of 1 virtual per request. If this still causes a problem, then you will have to open a case with support to get a developer working on it.

 

 

-Joe