HTTP Health Monitor Failure Behavior
I have a question regarding the connection status of a pool member when a health check fails and the service is marked down. I have two database servers in a pool where their active status is determined by an http health monitor that is looking for a response. One server is up and one is down and during maintenance events their status is toggled active-to-standby or standby-to-active by manipulating the status text that the health check is reading.
My question involves the expected behavior for connections when the status text is changed so that the expected receive string is not received and the service is marked down by the ltm. What does “down” mean? Because we are observing that connections don’t clear for hours after the service has failed its health check. I understand the impact to connections when a server is disabled or forced offline manually but I don’t understand why connections and database activity doesn’t go away when it is marked down due to health check failure.
If anyone can help, that would be super.
Try checking the Action on Service Down setting on the database server pool. By default, when a monitor marks a pool member down, no new or persistent connections are allowed to the pool member, however existing open connections are allowed to complete normally, assuming they can. For performance reasons, database systems often keep connections open (e.g. connection pooling). You can change the BIG-IP system's behavior when a monitor marks a pool member down by setting Action on Service Down to Reject or Drop. In brief, Reject resets the existing connection (TCP RST). Drop provides no notification to the pool member. See K15095: Overview of the Action On Service Down feature for more information.