Forum Discussion

Thomas_Schocka1's avatar
Thomas_Schocka1
Icon for Altocumulus rankAltocumulus
Sep 23, 2012

GTM synchronisation issue?

Hi guys,

 

 

I'm experiencing a very strange issue and I want to make sure that it is not due to my lack of understanding.

 

I have 2 GTM's, certificates are ok, bigip_add was run, all iQuery stuff seems to be running fine, there are no errors in logs nor GUI.

 

When I add a virtual server A to a certain server A on GTM A (this server is an LTM), I enter its IP address, I enter port 3389 and I select the TCP health monitor. Now if my understanding is correct this means that either GTM itself, or a member of the prober pool assigned to this server A will execute this monitor and that the results will be communicated to the GTM A.

 

After the sync has occurred between both GTM's, I check the settings of GTM B for this virtual server A on GTM server A. What I see here is that the tcp monitor is not in the selected list, whereas on GTM A it is there.

 

To make things worse, the virtual server is being marked 'down' (red status) on both GTM's, and the message is "(Monitor /Common/tcp from gtmd : no reply from big3d: timed out)".

 

I find two things odd here:

 

1) Why is it saying 'no reply from big3d' here? It's a simple tcp monitor, and all devices capable of executing this probe have full iQuery communication capabilities with both GTM's, there is no firewall or anything that might block this.

 

2) Why is the tcp monitor not syncing completely? When I look in the config (using tmsh), the tcp monitor is mentioned on both GTMs after sync has occurred, but the GUI is only showing me the monitor on the GTM on which I have created the virtual server.

 

Could I be running into a bug here?

 

All devices are running v11.2 HF1. The release notes for v11.2 HF2 don't mention anything related to this (or at least not as far as I can tell - I did read it entirely)

 

Can anybody shine a light on this?

 

 

Thanks in advance,

 

Thomas

 

config
design
  • Hamish's avatar
    Hamish
    Icon for Cirrocumulus rankCirrocumulus
    Sep 23, 2012
    1. Sounds like you run multiple traffic-groups on the LTM 11.x? If so, there's a known bug where big3d doesn't report monitor and vip status to the upstream (includes both GTM and EM).

     

     

    2. Not sure about that. If big3d was working properly you wouldn't need the tcp monitor. You obviously don't run VS discovery as you're saying you created the VS in the GTM Server (Or if you do, you're setting the translation addresses which disable the auto discovery).

     

     

    What can you see in the logs if you bump gtmd to debug?

     

     

    H
  • Thomas_Schocka1's avatar
    Thomas_Schocka1
    Icon for Altocumulus rankAltocumulus
    Sep 23, 2012
    Hi Hamish,

     

     

     

    I have resolved the issue in a different manner.

     

    It was apparently something about the prober pools that prohibited connectivity. I'm pretty sure I've run into an thing that should normally either simply work or at least give some more descriptive error.

     

     

    What I needed was the detection of a physical IP address (address of the node) being online in one datacenter or in the other. So I needed to add both the virtual IPon the LTM as well as the physical address of the node to the GTM. Then cross-reference these with pools and use topology records. If I added the physical IP of the server to the GTM server for the LTM in datacenter A as well as the actual virtual IP for this server to the same LTM in datacenter A, the VIP address monitor was returning correctly, the physical was giving me big3d timeouts. Considering I also created the GTMs themselves as GTM servers (because you have to), what I did was limit the prober pool for each GTM server to the GTM itself. This way I am certain that I will always have only one object for the physical IP that is online.

     

     

    I proceeded to add the GTM virtual server object for the VIP from datacenter A and the GTM virtual server object for the physical from datacenter B to a pool. I then created topology records to tell the GTM that when a request comes in from datacenter A, it should use the pool 'from_datacenter_A' which contains those two objects: the GTM virtual server object with the physical IP from the datacenter A GTM server for the GTM in that datacenter. and the GTM virtual server object with the VIP from the datacenter B GTM server for the LTM in that datacenter.

     

     

    Since the goal was to make the GTM return physical IP's when the requesting server is in the same datacenter as the IP it requested, and return the virtual IP for the same server in the other datacenter.

     

    All of this because our customer doesn't want to change physical IP addresses when moving servers between datacenters.

     

     

    Long story short: it's the prober pool logic that was throwing me off I think. It has been resolved and tested now.