Forum Discussion
Can I use iControl to force disconnect of all tcp sessions on a pool member?
We are planning some simple HA for two databases, Node A and Node B (active/standby). When we take one DB node offline we'd like to be able to forcibly reset any active tcp connections that are connected to that node. This should at a minimum clear the connections off the bigIP. It would be nice to reset the connections on both the client and server side so we can be sure that neither side has any lingering connections.
A simple setup would be as follows:
Node A has PGA priority of 5
Node B has PGA priority of 1
No traffic should go to Node B while Node A is active. For example, Node B is either kept 'forced offline' state or PGA turned on with no monitors attached to the node or pool.
The steps for a manual connection failure to Node B would be as follows:
1) shut down DB service on Node A
2) put Node A into forced down mode (which only allows active connections)
3) force disconnect of any active connections to node A
4) bring up node B
I'd like to trigger this via iControl. Most steps seem pretty trivial except for step 3 - how to force the disconnect of any active tcp sessions that are connected to node A after node A has been disabled.
Our first go round is a manual failover soln. Once we've determined potential monitor points we may re-visit to automate failure via irule.
thanks!
andy
11 Replies
Hamish_Marson_3Nimbostratus
I believe if you set the member monitor state ($poolMember->set_monitor_state() ) to 'STATE_DISABLED' that it should achieve what you're looking for.
Nicolas_MenantEmployee
i don't think it will work.
To use STATE_DISABLED means you maintain ACTIVE and PERSISTED connections. So if you use a long TTL for persistency you will always have connection coming on this pool member.
We may need something like SESSION_STATUS_FORCED_DISABLED [UPDATE]- Thanks- I think even Forced Down allows Active connections. Ideally I'd like to drop all active connections as well. I guess there isn't a node/pool member state that does this. perhaps, if possible, i'd have to iterate through the active connections and send a disconnect of some sort. for example, a tcp reset.
it may be a moot point, but i'm working on a current assumption that we may want to drop all connections to a Databaase node without actually having to stop the database service on the node first. - You might try the get_monitor_association() to get the monitor, and use the set_instance_state() method of the Monitor Interface to set the pool to set the state to disabled. IIRC, that will completely clear connections.
Sorry I'm not digging through my samples to make certain this will work - I'm on vacation and have stuff going on all around me, but believe this is the way to clear all connections.
Hope that helps.
Don. - I just got a request to work on this same problem myself.... I referenced
http://devcentral.f5.com/wiki/default.aspx/iControl/PoolToggle.html..
I just tested this on version 9.4.6. Setting the instance monitor state to STATE_DISABLED will trigger whatever the provisioned 'Action on Service Down' is set to on the member's pool. If it is set to 'None' connections will remain. If it is set to 'Reject', toggling the monitor state will send resets to the clients.
Hope this helps,
John - That's a step in the right direction, but I don't want to force the whole pool offline- just a single pool member. If that's not possible at least this is something to work with.
- I just tested a solutions... You can indeed:
1) Check to see what the current action on service down for a given pool is set to
2) Set it to 'SERVICE_DOWN_ACTION_RESET'
3) Set the pool member state to 'STATE_DISABLED'
4) Set it's instance monitor to 'STATE_DISABLED'
5) Pause a few seconds.... (allowing for the reset reaping to run through it paces before you call it off!)
6) Reset the pool to its original action on service down value
via iControl, and you will reset the client side connections. I had 4 long lived TCP socket services working through a virtual and they all got TCP RSTs when the steps above were applied.
Again, if the pool action on service down is simply the default (None) setting the status of the member and the monitor will not cause established connections to reset.
Thanks,
John - CharlesCS
Cirrus
Posted By jgruber on 12/26/2008 1:11 PM
I just tested a solutions... You can indeed:
1) Check to see what the current action on service down for a given pool is set to
2) Set it to 'SERVICE_DOWN_ACTION_RESET'
3) Set the pool member state to 'STATE_DISABLED'
4) Set it's instance monitor to 'STATE_DISABLED'
5) Pause a few seconds.... (allowing for the reset reaping to run through it paces before you call it off!)
6) Reset the pool to its original action on service down value
via iControl, and you will reset the client side connections. I had 4 long lived TCP socket services working through a virtual and they all got TCP RSTs when the steps above were applied.
Again, if the pool action on service down is simply the default (None) setting the status of the member and the monitor will not cause established connections to reset.
Thanks,
John
Do you have a script for this process that you would be willing to share, either here or on the codeshare? Thank you. 
MR_RJIm looking for a solution for this as well. Isnt it a bit strange that it isnt available in the web gui?
On the pool there is a lot of options to reject / reselect etc but not for the node.
I'm having problems with a application that takes forever to drain out. It seems that the application is keeping the connection / using them frequently so they wont time out.
Any suggestion how I solve this or manage to localize the problem?
It's communicating with MS IIS over a specific port, not 80.
Best regards
Robert- If you use the calls on this page: http://devcentral.f5.com/wiki/default.aspx/iControl/LocalLB__PoolMember.html set_session_enabled_state will allow you to stop new incoming connections, while set_monitor_status will allow you to "force down" the node by setting the monitor to report that it is currently down. This should end all connections to it.
Hope that helps,
Don.
