Best Practice for Sync Between Primary and Failover

For CONFIGSYNC_BASIC, that maps to the CLI command "bigpipe config sync min" which will issue a save to /config/bigip.conf and then a copy of that bigip.conf file to the secondary system.

 

 

I'm not sure what you have setup for your client socket timeout but if it's at the default 30 seconds, then if you have a large configuration or the copy takes a long time, then you might hit that 30 seconds in which your client connection is timing out.

 

 

-Joe

I'm leaning away from it being a timeout issue with the sync process between the 2 devices. Running it from the command line results in times between 21 to 26 seconds.

 

 

time b config sync min

 

Saving active configuration...

 

Creating UCS for config sync request...

 

Configsync Mode: Push

 

Transferring UCS to peer...

 

Installing UCS on peer...

 

Obtaining results of remote configuration installation...

 

Saving active configuration...

 

Current configuration backed up to /var/local/ucs/cs_backup.ucs.

 

Hostname of UCS file is nc-sils-f5c-ict02-mgt.infra.company.net, local hostname is nc-sils-f5c-ict01-mgt.infra.company.net

 

Installing shared configuration on host nc-sils-f5c-ict01-mgt.infra.company.net

 

Installing...

 

Reloading configuration... It may take a few minutes...

 

Reading configuration from /defaults/config_base.conf.

 

Reading configuration from /config/bigip_base.conf.

 

Reading configuration from /config/bigip_sys.conf.

 

Reading configuration from /usr/bin/monitors/builtins/base_monitors.conf.

 

Reading configuration from /config/profile_base.conf.

 

Reading configuration from /config/daemon.conf.

 

Reading configuration from /config/bigip.conf.

 

Reading configuration from /config/bigip_local.conf.

 

Loading the configuration ...

 

 

real 0m25.138s

 

user 0m1.270s

 

sys 0m0.140s

 

 

 

To further simplify, I've tried just making the call via soapUI:

 

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "POST /iControl/iControlPortal.cgi HTTP/1.1[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "Content-Type: text/xml;charset=UTF-8[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "SOAPAction: "urn:iControl:System/ConfigSync"[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "Authorization: Basic ==[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "User-Agent: Jakarta Commons-HttpClient/3.1[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "Host: nc-sils-f5c-ict02-mgt.infra.company.net[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "Content-Length: 557[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "[\r][\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "[\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " [\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " [\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " [\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " CONFIGSYNC_BASIC[\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " [\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> " [\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "[\n]"

 

Thu Jan 08 15:29:30 EST 2009:DEBUG:>> "[\n]"

 

 

If I set the timeout less than 5 minutes, I get a socket timeout. If timeout > 5 minutes, I get HTTP 500:

 

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "HTTP/1.1 500 Internal Server Error[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Date: Thu, 08 Jan 2009 20:29:31 GMT[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Server: Apache[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Set-Cookie: BIGIPAuthCookie=33544E35D354461B56F4BB3FEF6DDA9B6CD1A30C; path=/; [\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Content-Length: 1[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Connection: close[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "Content-Type: text/html; charset=iso-8859-1[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< "[\r][\n]"

 

Thu Jan 08 15:34:31 EST 2009:DEBUG:<< " "

 

 

 

These 2 lines in /var/log/ltm are the only evidence I can find of *something* happening on the BigIP.

 

 

Jan 8 15:29:34 nc-sils-f5c-ict02-mgt bigpipe: Begin config sync operation

 

Jan 8 15:34:34 nc-sils-f5c-ict02-mgt bigpipe[15477]: 01110001:3: Error running config sync

 

 

 

 

Is there anyway to enable more verbose logging of iControl/configsync?

FastCGI: comm with server "/usr/local/www/iControl/iControlPortal.cgi" aborted: idle timeout (300 sec)

 

FastCGI: incomplete headers (0 bytes) received from server "/usr/local/www/iControl/iControlPortal.cgi"

 

 

 

Found those 2 lines in /var/log/httpd/httpd_errors as well.

 

 

I have the same exact problem using the perl soaplite code. if i set the timeout to 500 i get a 500 error from the bigip, else I get a SSL Read timeout.

 

 

I also have all the errors above.. I can configsync via the webgui or the bigpipe command just fine.

 

 

sub syncConfiguration()
{
print "Syncronizing Config to Backup\n";
my ($syncFlag, $quiet) = (@_);
my $success = 0;
if ( "" eq $syncFlag )
{
&usage("sync");
}
$soap_response = $cfgsync->synchronize_configuration
(
SOAP::Data->name(sync_flag => $syncFlag)
);
if ( $soap_response->fault )
{
if ( 1 != $quiet )
{
print $soap_response->faultcode, " ", $soap_response->faultstring, "\n";
}
}
else
{
$success = 1;
if ( 1 != $quiet )
{
print "Configuration synchronized successfully!\n";
}
}
return $success;
}

no solution?

This looks like it's going to have to be a question from Product Support. The iControl Portal is running under FastCGI and it looks like the idle timeout that is configured for it is too small. FastCGI will monitor it's processes and kill them off if the run past the idle_timeout. I cannot recommend making system changes officially as they will likely get overwritten next time you upgrade. It's best to get this into the support queue to get it checked out by PD.

 

 

With that being said, you can try to tweak the idle timeout to test it yourself in the meantime by looking in the /etc/httpd/conf/httpd.conf file and look for the iControlPortal.cgi entry under the factcgi-settings. The -idle-timeout is currently set to 300. You could try increment it and see what happens.

 

 

Let us know how it goes.

 

 

-Joe

I tweaked the value, and it didn't seem to make any difference. I'm still getting SSL timeouts. When I run it from the commandline or through the web gui it takes 20-30 seconds to configsync. Also, after I attempt to run it via script i run a ps on the bigip and see the configsync still running.

I'll put in a ticket I guess, but I wasn't sure if icontrol stuff was covered under support.

 /usr/sbin/fcgi- HOSTNAME=bigip1.evri.com SHELL=/bin/bash TERM=xterm HISTSIZE=1000 SSH_CLIENT=10.75.4.2 59899 22 REMOTEROLE=0 SSH_TTY=/dev/pts/0 USER=root LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=
 9092 ?        S      0:00  |   \_ /usr/local/www/iControl/iControlPortal.cgi
 9108 ?        S      0:00  |       \_ /bin/bigpipe config sync min FCGI_ROLE=RESPONDER HTTPS=on SSL_VERSION_INTERFACE=mod_ssl/2.0.58 SSL_VERSION_LIBRARY=OpenSSL/0.9.7l SSL_PROTOCOL=TLSv1 SSL_CIPHER=DHE-RSA-AES256-SHA SSL_CIPHER_EXPORT=f
 9138 ?        S      0:00  |           \_ /usr/local/bin/SOAPCSClient --source /var/tmp/__sync_local__.ucs --destination __sync_remote__.ucs SSL_SERVER_A_KEY=rsaEncryption SSL_CLIENT_VERIFY=NONE SSL_SESSION_ID=47EE2A3A198BDDBD8185740E59

I seem to be able to reproduce the same behavior on at least 4 separate devices. Please post back to the thread if support comes up with anything, or if there is any info I can provide.

 

 

Thanks

Hi all, I meet with exactly the same problem with you in BigIP6900 OS 9.4. Waiting for solution from F5!!!

We also met this problem. The script we are using was working well in old platform: BIGIP6400, 9.3.1. Now we met this error when we tried to run the script on BIGIP6900, 9.4.6.

 

F5 told me 9.4.6 is the only OS that BIGIP6900 can use so far. So I don't know the problem was caused by new BIGIP hardware or the new 9.4.6 OS.