persistence cookie at BIG IP level too persistent?

When a member is detected down, persistence records to it should be invalidated. Since I can't see the traffic flow to your servers, my best guess is that since service checking occurs out-of-band, the next request may be hitting the server before service checks detect it as down. You can work around this in a combination of 2 ways:

 

 

a) Invalidate the persistence record from an LB_FAILED event, and reselect or redirect back again.

 

 

and

 

 

b) Use bl0ndie's request-reissuing rule to resend the same request to a different server if the error occurs after the request has already been sent to the server. (e.g. a 503 error.)