Forum Discussion

Narendra's avatar
Narendra
Icon for Nimbostratus rankNimbostratus
Nov 06, 2019

BIG DNS failing health checks to application services

I ran into a scenario where we need to have TCP health monitor at LTM level and HTTPS at GTM. when we have them applied GTM's were failing to listen to the application services. So, we tweaked the same settings at the LTM level and they were succeeded. Both LTM and GTM are on v12.1.4. Any differences that needs a tweak...

  • Is that GTM and LTM placed in the same LAN if not please check the https reachability to application is avaialble from GTM.

  •  

    Yes. GTM and LTM are in the same VLAN. the send and receive strins are working at the LTM level when we bring down the application services LTM is able to listen to the path and marking down based on the response.

     

    here is the information that is working at the LTM and GTM fails...

     

    send:

    GET /refdata-st1 HTTP/1.1\r\nHost:middleware.iadd1-stg-ace-a.ihgext.global\r\nConnection: close\r\n\r\n

     

    receive:

    OK

     

     

  • Any reason why you are not using Iquery to get the ltm object status.

    Simply put the https monitor for the ltm pool. Iquery mesh between the GTM & LTM. Add the LTM as bigip server. The LTM will update the GTM about the VS status.

     

    Unless there's some real good reason.

     

    For the troubleshooting. From the GTM, perform these basic functions,

     

    1. ping <server-ip>
    2. telnet <server-ip> <port>
    3. curl -kv http://<server-ip>:<port>/refdata-st1

     

    See what response you get for them. Keep us posted.

  •  

    Iquery is in place. GTM wideIP is using its default big3d daemon to get the LTM VIP status. However, in this case we are trying to have TCP defined at LTM pool level and HTTPS at GTM pool. GTM send and receive strings were not being validated to listen the application services healtch check where we tried the HTTPS health check at LTM pool and we see it is working fine. But we need to have that level of health check at GTM level for easy failover across datacenters.

     

    I tried the troubleshooting steps - ping, telnet and curl and all works good from GTM.

     

    Can you elaborate the Iquery checks you are talking about...

    • jaikumar_f5's avatar
      jaikumar_f5
      Icon for Noctilucent rankNoctilucent

      Any reason why you are trying to have TCP at ltm pool level and https on GTM pool. Why not let the LTM have the https monitor at the pool level. And let the iquery do its job to tell the GTM if that LTM VIP is healthy or not. Are you seeing any issue there. Maybe that is what you have to troubleshoot 1st.

      If the https monitor at the ltm pool is giving you right results, but the GTM in the event of VS being down still sending the traffic to that VIP is something to be looked at.

      I see you are asking for easy failover across datacenters. The best method is the GA at the GTM pool level.

      WIDEIP --> GTM Pool --> [VS1 (DC1-LTM-VS - GA Order 0 ) and VS2 (DC2-LTM-VS - GA Order 1)

      When the DC1-LTM-VS goes down because of https monitor, the GTM Iquery will get the VS from the big3d daemon and markdown the GTM pool member.

      Automatically the traffic will failover to VS2, which from DC2.

      From troubleshooting standpoint, share us your curl output, masking the fields wherever needed. If curl is working fine like you say, returning 200 OK. But the GTM pool still marking it down,,, this could mean that GTM's big3d is stuck, the GTM probing to that server is stuck with some old session ID.

      On the GTM, take a pcap for the destination.

      Also share us the big3d version from both LTM & GTM,

      /shared/bin/big3d -v 
      • Narendra's avatar
        Narendra
        Icon for Nimbostratus rankNimbostratus

        When we have the HTTPS health check at LTM the GTM is working fine with no defined health monitor since it is using big3d agent itself. However, the scenario is different.. all applications are using only one LTM to route it to pool members with a pass through and we want to have one GTM per application with its own custom health check in that going further new applications LTM doesn't needs any modification since it just uses TCP.

         

        This is little out of the skilled one.. the LTM's are tested with custom HTTPS monitor when we have no health check at GTM. GTM's are doing good in marking the LTM VS down in GTM pool whenever the application services were brought down. when we switch the LTM to TCP health check and put the ssam ecustrom HTTPS health check we used for LTM in the GTM, they are marked down when application services are up and running.

  • GTM HTTPS and HTTP health checks are working perfectly fine. A change has been made in the response code where F5 is listening for JASON payload message "status:UP".

     

    Thank you   and  for getting together.

     

    Basically the application services are running in open shift containers and not a 3 tier server architecture model.