Forum Discussion
Roman_80473
Nimbostratus
NimbostratusHow to detect if pool member is down?
Hi folks,
I need to detect if pool member is down (crashed, hung, out of memory), take it out of the mix, write details to the log (node ip, name, port), and resend traffic back to the original pool before health monitor detects a failure.
So, inside my iRule, I have two events to capture server failure:
1. HTTP_RESPONSE checks if server's alive, but not responding to requests:
when HTTP_RESPONSE {
if { [HTTP::status] >= 500 } {
log local0.debug "Node [LB::server addr]:[LB::server port] is not responding, mark down, resend traffic ..."
LB::down
resend cached request
HTTP::retry $LB_request
}
}
2. LB_FAILED checks if server's completely dead:
when LB_FAILED {
log local0.debug "node [LB::server addr]:[LB::server port] is dead, mark down, resend traffic ..."
LB::down
resend cached request
HTTP::retry $LB_request
}
When I shutdown one of the members, the logic seems to be somewhat working (no blank page in the browser), but I see this in the logs:
01220001:3: TCL error: My_LB_Rule - Error: HTTP::retry may not be used in client-side event LB_FAILED (line 1) invoked from within "HTTP::retry $LB_request"
Is there a better way to accomplish this?
Any help is greatly appreciated
Roman
Michael_YatesHi Roman,
The LB_FAILED Event should occur prior to the HTTP_RESPONSE Event, so the HTTP::retry in your response event should suffice for what you are trying to do.
See this Tech Tip: http://devcentral.f5.com/Tutorials/TechTips/tabid/63/articleType/ArticleView/articleId/344/iRules-Event-Order.aspx
Hope this helps.- Roman_80473Hi Michael,
If node is shutdown, http_response is never executed. Same goes for suspended/out of memory situation (server responds with 500 errors) when the lb_failed is not executed. I think I need both events present in iRule. I just need to figure out how to resend traffic back in each event.
Thanks Roman - Michael_YatesHi Roman,
No way to execute the HTTP::retry in the LB_FAILED Event. There are ways around it though.
You can set the HTTP::request in a variable in the HTTP_REQUEST Event and if the LB_FAILED Event occurs you can perform an LB::down, persist none, and LB::reselect to restart the process instead of dying. Setting a retry flag to know if you need to resend the previous request could then be processed in the HTTP_RESPONSE.
Something like this:when HTTP_REQUEST { log local0. "HTTP Request Event." set httprequest [HTTP::request] set retry 0 } when LB_SELECTED { log local0. "LB Selected Event [LB::server addr] : [LB::server port]" } when LB_FAILED { log local0. "LB Failed Event [LB::server addr] : [LB::server port]" LB::down persist none LB::reselect set retry 1 } when HTTP_RESPONSE { log local0. "HTTP Response Event." if { $retry == 1 } { set retry 0 HTTP::retry $httprequest } }
Hope this helps. - Roman_80473Hi Michael,
I think I got it now. Thank you very much for your help
Roman 
Gopal_Prakriya_I would like to page myself by sending a text message on LB_FAILED, appreciate any help on how to do that.
What_Lies_Bene1Cirrostratus
Gopal, please start a new topic rather than posting in unrelated existing (and unresolved) ones.