More on GTM + LTM design across 2 DC

awfully quiet in here..

Your setup sounds very similar to ours with the exception that you have redundant GTMs. One note.. when setting this up.. auto-link discovery is BAD per F5 professional services and F5 back-line support. Don't turn it on.. ensure it's off.'

 

 

First make sure you've got NTP setup on all servers. Also, make sure your GTMs are all at the same level and are at the same or higher revision of software than the LTMs. ( ideally every device is running the same version )

 

Make sure you've done the self-certificates, provisioning and licensing on all of the GTMs as well.

 

 

Installing LTM & GTM certificates on your GTMs

 

 

Your GTMs need to communicate to each other and to your LTMs. This is done through using ssh and port 4353. On every GTM you will need to run "bigip_add [ipaddr]" against a reachable IP on every LTM and GTM it will communicate with. These IP addresses on the target GTM and LTMs must have port 22 and 4353 allowed on the self-ips that will communicate. These IP addresses must be self-IPs that are NON- cluster addresses ( no failover IPs ).

 

 

Example: on GTM00DC run, "bigip_add [gtm00drcl ip]" enter in gtm00drc's root password when prompted. Then repeat for each LTM ( both the active and standby ) in both datacenters.

 

 

Only one self-ip is required for communication.

 

 

Setting up GTM-GTM syncrhronization

 

 

So far.... You've already traded installed the certificates between your GTMs and added all of your LTMs to both GTMs as well. ( see above and the "bigip_add" command ). You've also already created identical "Data Centers" on each of the GTMs.

 

 

Now you need to setup your GTMs in a synchronization group ( they will still be standalones even though they are synchronizing.. yeah, odd.. a redundant pair is only two in the same location).

 

 

On the first GTM ( either one ):

 

 

"System">"Device">"Global Traffic". Select the Synchronization box and create a name for the "Synchronization Group Name" field. ( You'll use this EXACT same name for all GTMs that will be synchronizing DNS entries.

 

 

On the First GTM: ( You've already created identical datacenters and done the certificate swap between all LTMs and GTMs right? ... check to be sure )

 

 

"Server" then click "Create". Type the name of the other GTM, enter in the IP address and select the appropriate "Data Center". Add the "bigip" monitor, and enable "Virtual Server Discovery" but DO NOT enable "Link Discovery". Your second GTM will pull all server definitions from the other GTM and will stay synced should you add a Server or DNS entry to either of the GTMs in the future.... AFTER we do the next step!

 

 

On the second server from the CLI do a "gtm_add [ipaddr]". This will initiate the first pull and will keep you in sync from now on. Adding a DNS record to either side, or a server to either side will be synced to the other GTM(s). This is a bi-directional sync, add information to either side and the GTMs will handle the syncing. Synchronization is based off of file timestamps.

 

 

 

Then add the LTMs to either GTM definition.

 

Adding a "Server" is not creating a "WideIP" nor is it creating a "Virtual IP".. a "Server" by GTM definitions is another load balancer ( LTM for us ) or GTM.

 

 

* From the GUI select "Global Traffic"->"Servers" and click "Create"

 

* Give the server a name ( doesn't have to be a resolvable fqhn or anything like that... this is just descriptive. )

 

* Select the type of server ( If you select a "Big-IP System (Redundant)" it adds fields for the second system ). For the address do NOT use a relocatable IP. Static Self-IP only.

 

* Select the data center and the status. ( enabled )

 

* For health monitors, select "bigip" equipment. ( We're not doing anything else currently

 

* Under "Resources" set Virtual Server Discovery set to "Enabled".

 

* Leave "Link Discovery" set to "DISABLED"!! Rule is: server discovery good.... link discovery bad

 

* Then click "Create".

 

 

If all is correct you'll see information start to populate. What? You don't see it populating.. Aha! That's because you'll need to add device certificates between devices for effective communication. To do this, log into your GTM and run "bigip_add [ipaddr]" Where ipaddr is the IP address of the LTM or GTM you are adding to this device.

 

 

 

 

I am not certain of your end goal design, but it sounds like you'll want to setup the six VIPs ( one for each ISP in each DC ... three on each LTM pair ). If your LTMs are using the same pool for each VIP then when the pool becomes unavailable all three VIPs in that DC will become unavailable. With the LTMs reporting up to the GTMs, they'll note the VIP as down and not give out it's IP address. Your GTM in each datacenter should be authoritative for whichever domain your DNS entries will be located in. Each GTM should have a 'listener" that is registered as the authoritative IP for that domain.. the way DNS works is that the first one of those IPs to reply to a query will be the one that is used.. so there's no need to have a GTM in one datacenter re-point people to DNS in another datacenter. ( You could do this... but it introduces complexity, points of failure and is simply not the cleanest solution ) ..

 

 

With this setup, your GTMs will see all the VIPs and all the LTMS and will handle failover. If a GTM or even a redundant pair of GTMs disappears, the remaining ones will still have a listener up and available that will be hit by any DNS resolution queries.

 

 

 

Jason

Superb! Jason, thank you very much!..

 

we are in implementation phase right now and your response do help us abundantly, especially with the certificates..

 

I will run through your hints one by one, and hopefully it works, with only minor troubleshooting..

 

Will update back soon..

 

 

Thanks

 

Teguh

 

Hi Jason, what is needed to make sure that the LTM and GTM in DC and DRC can communicate correctly over the Internet?

 

The DC and DRC in my topology is geographically separated.

 

 

And btw, are your steps is what is meant by activating iQuery?

 

 

Thanks before

 

Teguh

 

To get the nodes to communicate you'll have to ensure the following:

 

"These IP addresses on the target GTM and LTMs must have port 22 and 4353 allowed on the self-ips that will communicate."

 

So.. your GTMs will need to be able to communicate to a self-IP on each of the LTMs that have "custom" ports allowed of port 22 and 4353.

 

Your GTMs will also need to be able to communicate to self-IPs on each other ( the pairs will be doing this via the crossover cable ) using these same ports.

 

To set these up you'll need to go to the "Network" section.. then "Self-IPs". Then select the Self-IP that will be used for communication. The communication is secure using ssh, but I highly recommend that all of the communication occur on an internal subnet. For IPV4 use an RFC1918 subnet. For IPV6... you'll have to find someone that knows more about IPV6 than me to get a recommendation.

 

 

iQuery is the API that the bigips use to communicate with each other.

 

 

Hope this helps.

 

 

Hi Jason,

 

 

We are trying to implement similar design except GTM and LTM are separated from firewall and LTM are configured with private IP address space. GTM. and LTM can. Communicate on iQwuery fine in the same data center once the firewall is open for iquery ports but not from other data enter, ie GTM from DC1 can not communicate to LTM in DC2 and vicevesa. I think it because GTM from remote DC is not able to communicate to LTM over private network.

 

 

I'm thinking of casting VIP on LTM public address and add that VIP as a server in GTM . Will that work.

 

Can. You suggest some option to address this issue. Also, NAT on firewall is not an option.

 

 

Thanks