Forum Discussion

ben_stokes_8749's avatar
ben_stokes_8749
Icon for Nimbostratus rankNimbostratus
Oct 12, 2009

http health monitor help

Hi,

 

 

I'm trying to configure an http health monitor. My app using port 8080 and the app behind the F5 is Sun Glassfish server 2.1 (java based webservices). I've tried quite a few things already and I guess have come to a brick wall.

 

 

If I create a new http health monitor, simply specifying "GET /" in the send string and nothing in the receive string, my nodes are marked as down even though I am able to browse to http://mynode:8080/ and see content on the page.

 

 

If I open an ssh session to the F5, and use 'telnet mynode 8080', it connects but as soon as I start typing "GET", the telnet session is terminated. I think this is the reason the health monitor is failing. The app server is ending the http connection as it is not behaving 'normally'.

 

 

For the moment I have configured a tcp monitor to check if port 8080 is open, but in our testing environment I have found a node in the pool can crash and it is not marked as down, because port 8080 is still open.

 

 

Is there something I can do to 'trick' my app servers into thinking the F5 is a real browsing session? Or is there a better way to do this?

 

 

Thanks
management
monitoring
  • hoolio's avatar
    hoolio
    Icon for Cirrostratus rankCirrostratus
    Oct 12, 2009
    Hi,

     

     

    I'd guess the node closes the connection quickly when you're testing with telnet due to a short idle timeout. Can you try this for a send string:

     

     

    GET / HTTP/1.1\r\nHost: \r\nConnection: Close\r\n

     

     

    Set the receive string to HTTP. If that works, then change the receive string to HTTP 200 and see if that works.

     

     

    Aaron
  • ben_stokes_8749's avatar
    ben_stokes_8749
    Icon for Nimbostratus rankNimbostratus
    Oct 12, 2009
    Thanks for the advice ... I put in your send string, now a receive string set to HTTP works, but the nodes are marked down if I use HTTP 200.
  • hoolio's avatar
    hoolio
    Icon for Cirrostratus rankCirrostratus
    Oct 12, 2009
    That's progress. Can you check what response the server is sending by looking at the web server access logs; enabling debug on the monitoring daemon, bigd, by running 'b db bigd.debug enable' (output is written to /var/log/bigdlog) or by running a tcpdump like this:

     

     

    tcpdump -ni 0.0 -Xs0 host SERVER_IP and host LTM_SELF_IP

     

     

    Aaron
  • ben_stokes_8749's avatar
    ben_stokes_8749
    Icon for Nimbostratus rankNimbostratus
    Oct 13, 2009
    With the debug enabled, I changed the receive string to HTTP 200 and observed the log, it doesn't make a gret deal of sense to me but I think this would be the relevant part: 

    10:15:18.679176: (_main_loop): wfd selected for addr=::ffff:10.20.15.32:8080 pending=1  
 10:15:18.679198: (_send_active_service_ping): addr=::ffff:10.20.15.32:8080  
 10:15:18.679230: send_active_service_ping: addr=::ffff:10.20.15.32:8080 writing GET / HTTP/1.1  
 Host:  
 Connection: Close  
 ...

    Can I just clarify what is currently happening? Am I understanding correctly that currently the only thing coming back is part of the http header containing the text "HTTP" before the connection is closed by the app server?
  • hoolio's avatar
    hoolio
    Icon for Cirrostratus rankCirrostratus
    Oct 13, 2009
    That's just showing the request to the server. Is there anything after that in the bigdlog for the response from 10.20.15.32:8080?

     

     

    Aaron
  • ben_stokes_8749's avatar
    ben_stokes_8749
    Icon for Nimbostratus rankNimbostratus
    Oct 13, 2009
    Ahhhh found it .... 

    10:43:04.090645: (_recv_active_service_ping): addr=::ffff:10.20.15.32:8080 
 10:43:04.090691: (_recv_active_service_ping): addr=::ffff:10.20.15.32:8080 send=GET / HTTP/1.1 
 Host:  
 Connection: Close 
  
  
  regexp=HTTP recv=HTTP/1.1 200 OK 
 X-Powered-By: Servlet/2.5 
 Server: Sun GlassFish Enterprise Server v2.1 
 Content-Type: text/xml;charset="utf-8" 
 Date: Tue, 13 Oct 2009 09:43:04 GMT 
 Connection: close

    Also under this is the XML from the webservice, I've tried adding some of the test from the XML as the receive string and it works!

    e.g. this is in the log:

    I've set "BoxSearchService" as the receive string and the nodes are marked as up.

    I've switched off debugging using: b db bigd.debug disable

    Thanks for your help!
  • Nit_67494's avatar
    Nit_67494
    Icon for Nimbostratus rankNimbostratus
    Dec 10, 2010
    Hi Aaoron,

     

    I have an F5 in the DMZ zone, if I put only GET / the pool is still mark as down, but if I put: GET / HTTP/1.1\r\nHost: \r\nConnection: Close\r\n the pool is mark as up?

     

    But in the internal F5-LTM, it will work even if I put only the standard GET /

     

    The pool member is in internal server farm.

     

    Any idea?
  • hoolio's avatar
    hoolio
    Icon for Cirrostratus rankCirrostratus
    Dec 10, 2010
    Hi Nit,

     

     

    GET / is an HTTP/0.9 formatted request. Is it possible that there is a proxy between the LTM and pool member which is blocking the request? Regardless, it's better to use an HTTP 1.1 request and tell the server that LTM isn't planning on reusing the connection with the Connection Close header.

     

     

    Aaron
  • Nit_67494's avatar
    Nit_67494
    Icon for Nimbostratus rankNimbostratus
    Dec 11, 2010
    Hi Aaron,

     

    I don't think there is a proxy blocking, if it's blocking, the GET / HTTP/1.1 will not work also, btw..is the health monitoring using the self-IP to check the health? because in my server farm firewall, I can see the log that the self IP is trying to connect to the server