Forum Discussion

royster_128009's avatar
royster_128009
Icon for Nimbostratus rankNimbostratus
Oct 03, 2003

Using iControl with Health Monitors

Hi,

 

 

I have been using BIGIPs at our company for several of our customers for a couple of years.

 

 

I have written custom monitors using perl before testing appilication availability using simple curl commands. Whilst this has worked previously for services that were simply UP or DOWN, We now have a new customer onboard that would like a much more fine grained control over server availability and BIGIP management of them.

 

 

The requirement is for BIGIP to poll the webserver which will come back with 1 of 3 responses.

 

 

- 'OK' indicates that the service is running normally and requests can be

 

routed to it.

 

 

- 'NoNew' indicates that BigIP should continue current connections to that

 

service but not allocate any new connections to it.

 

 

- 'NotOK' indicates that the service is not functioning, and BigIP should

 

route all requests to an alternate server.

 

 

Having spoken to F5 support they tell me that iControl is the way to go since the 'Enable Connections', which is the control which will allow exisitng sessions to drain away, can only be set through manual intervention. I need this value to be set automatically from within the script depending on the outcome of the health check.

 

 

I've not used iControl at all and would interested in any feedback and how I could go about implimenting this using perl.

 

 

Thanks
dev
devops
iControl
v4
  • Joe_Pruitt's avatar
    Joe_Pruitt
    Oct 03, 2003
    Assuming you are running BIG-IP v4.5 you should have the necessary perl modules to execute iControl methods. The SDK includes sample code in for interacting with nodes including querying the states. The specific methods you are looking for are set_state() and set_availability() located in the ITCMLocalLB::Node namespace.

     

     

    The only possible issue with using iControl on the box is that the perl script will have to have the local web username and password included which could be considered a security issue. The scenarios we generally see our customers using are

     

     

    1. External Montoring

     

    Basically this is using their own monitoring applications and making node provisioning method calls from the external application.

     

     

    2. Server self monitoring

     

    In this case logic is put on the webservers so they can report their own health to the BIG-IP. This is beneficial as the server has more information about it's "health" than any outside entity could. The downside to this approach is that custom code needs to be developed and maintained on the servers.

     

     

    So, with that being said, if you (and your customers) are comfortable with putting the user credentials in the script it should be very straightforward to develop this script.

     

     

    The following methods would be used:

     

     

     
 // For draining connections to a node server (ie. 10.10.10.10:80) 
 void ITCMLocalLB::Node::set_availability( 
     in IPPortDefinition[] node_defs, 
     in AvailabilityState state 
 ); 
  
 // For stopping connectivity to a node server (ie. 10.10.10.10:80) 
 void ITCMLocalLB::Node::set_state( 
     in IPPortDefinition[] node_defs, 
     in EnabledState state 
 );

     

     

    Here's some code segments that might help in developing the code (forgive me for any typos as I'm writing this on the fly).

     

     

     
 use SOAP::Lite; 
  
 $sBIGIPAddress = "";   Enter local BIG-IP address here 
 $sUID = "";  Enter Local Username here 
 $sPWD = "";  Enter Local Password here 
  
 $sNodeIP = "";  Set to nodes ip address 
 $sNodePort = "";  Set to node's port  
  
 $node_definition = { address => $sNodeIP, port => $sNodePort }; 
  
 $AVAILABILITY_UP = 2; 
 $AVAILABILITY_DOWN = 1; 
  
 $STATE_DISABLED = 0; 
 $STATE_ENABLED = 1; 
  
 sub SOAP::Transport::HTTP::Client::get_basic_credentials 
 { 
     return "$sUID" => "$sPWD"; 
 } 
  
 $LocalLBNode = SOAP::Lite 
     -> url('urn:iControl:ITCMLocalLB/Node') 
     -> readable(1) 
     -> proxy('https://$sBIGIPAddress/iControl/iControlPortal.cgi'); 
  
  poll webserver for status code 
 $sResponse = &pollWebServer(); 
  
  perform node action based on status code 
 if ( "OK" eq $sResponse ) 
 { 
     $LocalLBNode->set_state 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $STATE_ENABLED ) 
     ); 
       Check for fault 
  
     $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_UP ) 
     ) 
       Check for fault 
 } 
 elsif ( "NoNew" eq $sResponse ) 
 { 
      "drain" connections 
     $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_DOWN ) 
     ) 
 } 
 elsif ( "NotOK" eq $sResponse ) 
 { 
      stop traffic 
     $LocalLBNode->set_state 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $STATE_DISABLED ) 
     ); 
       Check for fault 
 }

     

     

    Good luck and let us know how things go...

     

     

    -Joe
  • royster_128009's avatar
    royster_128009
    Icon for Nimbostratus rankNimbostratus
    Oct 24, 2003
    Hi,

     

     

    I'm getting the following errors whilst trying to renable a node

     

     

    the section of the code:

     

     
 elsif ( $sResponse eq "OK" ) 
 { 
 print "OK\n"; 
  
     $LocalLBNode->set_state 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $STATE_ENABLED ) 
     ); 
  
     $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_ENABLED ) 
     )

     

     

     

    Produces this error:

     

     

    [...]

     

     

     

    >

     

    >

     

    >

     

    >

     

    >

     

    >8001

     

    >10.10.11.3

     

    >8

     

     

    [....]

     

     

    ITCMCommon::InvalidArgument Exception caught on ITCMLocalLB::Node::set_availability()!
  • Joe_Pruitt's avatar
    Joe_Pruitt
    Oct 25, 2003
    I've updated the original posting with code that should work. I appologize for the incorrectness in the original post. The correct values for Node::set_availability are AVAILABILITY_UP (2) and AVAILABILITY_DOWN (1). (not AVAILABILITY_ENABLED AND AVAILABILITY_DISABLED as I previously mentioned.

     

     

    -Joe
  • royster_128009's avatar
    royster_128009
    Icon for Nimbostratus rankNimbostratus
    Oct 28, 2003
    I now have a working script, however one of the downsides is that each time the EAV is now invoked, regardless of the status, a log is made which is making for numerous duplicate entries because of the iControl

     

    $AVAILABILITY_UP node actions.

     

     

    --------------------------------------------------------------------------------

     

     

    Oct 27 17:30:47 BigIP01 last message repeated 50 times

     

    Oct 27 17:20:50 BigIP01 last message repeated 50 times

     

    Oct 27 17:10:51 BigIP01 last message repeated 10 times

     

    Oct 27 17:08:52 BigIP01 last message repeated 2 times

     

    Oct 27 17:08:28 BigIP01 bigd: Service detected UP (EAV) for 10.10.11.3:8001

     

     

     

    Is there anyway round this?
  • Joe_Pruitt's avatar
    Joe_Pruitt
    Oct 28, 2003
    You could modify the script to query the state/availability of the node and only make the iControl call if the state/availbility has changed. That will result in an additional call for conditions when the state changes, but will result in only a query when the state is the same. Queries do not result in log messages being generated so you will only get log entries for the conditions when state changed.

     

     

    -Joe
  • royster_128009's avatar
    royster_128009
    Icon for Nimbostratus rankNimbostratus
    Nov 19, 2003
    Hello.

     

     

    Below is a copy of the script that we are currently testing with.

     

     

    A few notes.

     

     

    Firstly, the 'NoNew' or 'Draining' functionality is only as good as the persistence model you use. We monitor two different types of pools, http and https. The https persistence used SSL ID and for http - we inserted a cookie with a zero value for expiration so it was more session based. It seems to work very well.

     

     

    Also for 'NoNew' we did a $STATE_DISABLED coupled with a $AVAILABILITY_UP in order to get it to work.

     

     

    Comments welcome.

     

     

     
 !/usr/bin/perl 
  
  
  BIG-IP Script to check App Status using iControl / 
  
  R.Dimbleby / Joe of F5 20/10/03 
  
  
 use strict; 
 use SOAP::Lite + trace => qw(method debug); 
 use SOAP::Lite; 
  
  
 my $programname = "web01_http_8001_checker"; 
  
 if ($programname eq '') { 
     die "Bad data in program name\n" 
 } 
  
  Process ID and file where it's to be stored.  The format 
  is significant. 
  
 my $pidfile = "/var/run/$programname.pid"; 
 my $pid = "$$"; 
  
  Maintenence.  Clean up any existing EAV. 
  
 if (-f $pidfile ) { 
     open(PID, "<$pidfile"); 
     my $pid = ; chomp $pid; $pid =~ m/^(\d+)$/; $pid = $1; 
     close(PID); 
     kill -9, $pid; 
     unlink($pidfile); 
 } 
  
  Create a new maintenence file. 
  
 open(PID, ">$pidfile"); 
 print PID $pid, "\n"; 
 close(PID); 
  
 my $sBIGIPAddress = "10.10.5.2";   Enter local BIG-IP address here 
 my $sUID = "";  Enter Local Username here 
 my $sPWD = "";  Enter Local Password here 
  
 my $sNodeIP = "10.10.11.3";  Set to nodes ip address 
 my $sNodePort = "8001";  Set to node's port 
 my $curl = "/usr/local/bin/curl"; 
 my $timeout = 10; 
 my $output = "/usr/local/lib/pingers/output_$sNodeIP\_$sNodePort"; 
 my $url = "http://$sNodeIP:$sNodePort/getstatus.jsp"; 
  
  
 my $node_definition = { address => $sNodeIP, port => $sNodePort }; 
  
 my $AVAILABILITY_ENABLED = 8; 
 my $AVAILABILITY_DISABLED = 9; 
 my $AVAILABILITY_DOWN = 1; 
 my $AVAILABILITY_UP = 2; 
  
 my $STATE_DISABLED = 0; 
 my $STATE_ENABLED = 1; 
  
 sub SOAP::Transport::HTTP::Client::get_basic_credentials 
 { 
     return "$sUID" => "$sPWD"; 
 } 
  
 my $LocalLBNode = SOAP::Lite 
     -> uri('urn:iControl:ITCMLocalLB/Node') 
     -> readable(1) 
     -> proxy("https://$sBIGIPAddress/iControl/iControlPortal.cgi"); 
  
  poll webserver for status code 
 my $sResponse = &pollWebServer(); 
  
  
  perform node action based on status code 
  
  
  If nothing at all 
 if ( -z $output ) 
 {  stop traffic 
     $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_DOWN ) 
     ); 
 } 
  
 elsif ( $sResponse eq "NotOK" ) 
  
 { 
      stop traffic 
     $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_DOWN ) 
     ); 
  
 } 
  
 elsif ( $sResponse eq "OK" ) 
  
 { 
 print "UP\n"; 
 checkOKState(); 
  
 } 
 elsif ( $sResponse eq "NoNew" ) 
 { 
      "drain" connections 
         print "No New\n"; 
         $LocalLBNode->set_state 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $STATE_DISABLED ) 
     ); 
  
   $LocalLBNode->set_availability 
     ( 
         SOAP::Data->name ( node_defs => [$node_definition] ), 
         SOAP::Data->name ( state => $AVAILABILITY_UP ) 
     ) 
  
 } 
  
  
  
  pollWebServer 
  Get status from Webserver 
  
 sub pollWebServer { 
         my($returnval)=""; 
         my $line=""; 
  
          For SSL use system("$curl -m $timeout -E BigIPMonitor.pem $url > $output"); 
         system("$curl -m $timeout $url > $output"); 
         open(fd,"/usr/local/lib/pingers/output_$sNodeIP\_$sNodePort"); 
         my @lines=; 
         foreach $line (@lines) { 
                 if ($line =~ /\bNotOK\b/) { $returnval="NotOK"; } 
                 if ($line =~ /\bOK\b/) { $returnval="OK"; } 
                 if ($line =~ /NoNew/) { $returnval="NoNew"; } 
         } 
         close fd; 
         return($returnval); 
 } 
  
  
  checkOKState 
  Is the node already UP 
  
 sub checkOKState { 
  
         my $soap_response =               
                 $LocalLBNode->get_availability 
                   ( 
                     SOAP::Data->name ( node_def => $node_definition ) 
                   ); 
  
                         my $result = $soap_response->result; 
                         print "Node $sNodeIP:$sNodePort availability: "; 
                         if ( 0 == $result ) 
                         { 
                                 print "UNCHECKED\n"; 
                         } 
                         elsif ( 1 == $result ) 
                         { 
                                 print "DOWN\n"; 
                          Bring back UP after failure 
                         $LocalLBNode->set_state 
                         (  
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $STATE_ENABLED ) 
                         ); 
  
                         $LocalLBNode->set_availability 
                         ( 
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $AVAILABILITY_UP ) 
                         ) 
  
                         } 
                         elsif ( 2 == $result ) 
                         { 
                                 print "UP\n"; 
                         } 
                         elsif ( 4 == $result ) 
                         { 
                                 print "FORCED DOWN\n"; 
                          Bring back UP after failure 
                         $LocalLBNode->set_state 
                         ( 
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $STATE_ENABLED ) 
                         ); 
  
                         $LocalLBNode->set_availability 
                         ( 
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $AVAILABILITY_UP ) 
                         ) 
  
                         } 
                         elsif ( 9 == $result ) 
                         { 
                         print "DISABLED\n"; 
                          Bring back UP after NoNew 
                         $LocalLBNode->set_state 
                         ( 
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $STATE_ENABLED ) 
                         ); 
  
                         $LocalLBNode->set_availability 
                         ( 
                         SOAP::Data->name ( node_defs => [$node_definition] ), 
                         SOAP::Data->name ( state => $AVAILABILITY_UP ) 
                         ) 
                         }