Forum Discussion
How can I automatically mark pool addresses down when one ISP fails with GTM acting as Link Controller with multiple ISP's?
We have LTM monitoring individual pool members and GTM finds out if the pool is available only by asking LTM - this is fine. GTM has multiple public IP's to give out for each ISP providing service. So here's the problem - if an ISP link goes down (and I've played with things like transparent monitors and bigip-link monitors to watch this) how can the GTM know NOT to give out the addresses for the failed ISP - LTM will still be telling it the pool is fine and available. ?? This is a basic design question I guess and would appreciate just an understanding of the approach I should take, I can study the details and ask more questions if it's still a mystery. -Thanks
8 Replies
- dhsmith_116072
Nimbostratus
OK, maybe I worded this badly - I really can't believe it's that difficult a question - Maybe I'll repost if this comment doesn't help. Multiple ISP's - multiple public address ranges to give out via GTM/DNS - how do you handle it when one ISP "fails" (their local router may still be answering pings but for whatever reason upstream you just don't want to be giving out that address range) How do you do that? - Kevin_Stewart
Employee
I guess it just boils down to monitoring. Is there anything the LTM or GTM can query that will indicate the status of the link?
- dhsmith_116072
Nimbostratus
I see an interesting option you can add to the "virtual servers" the GTM learns about from the LTM - the box is titled "Dependency List" -- From the onscreen help: "Specifies the virtual servers on which the current virtual server depends. If any of the specified servers are unavailable, the current virtual server is also listed as unavailable." This would seem to be the key. I'm guessing creating some kind of server/pool/monitoring combination on the LTM will have just one entity doing the upstream monitoring for an ISP and if that goes down the corresponding server/IP goes down on the GTM (hopefully stopping it from giving out that particular IP address) If anyone has an example of this kind of design configuration I'd love to see it. - Tom_G__134358
Nimbostratus
Hi,
I have a similar issue. I kind of managed to get it working using :
- links with custom monitors (based on gateway ICMP) pinging google DNS servers via the link.
- explicit-link-name to explicitly attach the link to each GTM virtual server associated with the ISP link. (see Sol13827)
It works fine with two remaining issues :
- in my case, some virtual servers are automatically attached to the link (even though I did not specify an "explicit-link-name" value) because they belong to the same subnet. So I'll have to create another link and attach them to it.
- I still have one virtual server that does not follow the status of the link although it is configured in a similar way. I.e the link is marked as down, most virtual servers associated with the link are brought down, but this particular one stays up. Looks like a bug to me, and I have an open case with F5.
Also what bugs me is that all service provisioning can be done from the WebUI, but the "explicit-link-name" needs to be done in TMSH... not optimal.
Your last post on this topic was 3 weeks ago. Did you manage to get it working ? Is there anything you can share ?
Thanks
Tom
- dhsmith_116072
Nimbostratus
Hi Tom - thanks a LOT for your information. I'll have to look into the explicit-link-name command, I'm unfamiliar with it. I am still slightly amazed that this isn't a very common scenario out there and that there isn't a simple webUI configuration option (or at least examples) for dealing with multiple ISP's I have not been able to work on this with the transparent monitoring configuration because I'm still working though another problem trying to get a happy HA pair of LTM's on 11.3 to be just as happy on 11.4.1 When I get back to it I will post whatever I learn here since this should be the best resource we have for sharing. To recap my objectives: 1. minimum monitoring (ie. only one device should be doing upstream polling, pinging, whatever, not 500 virtuals) 2. LTM AND GTM must BOTH modify their behavior as a result (ie. LTM must not use the "down" ISP for outbound traffic initiated from inside AND GTM must stop giving out IP's of the "down" ISP. Return traffic from outside will behave nicely automatically thanks to auto-last-hop magic) Thanks again - wish there were more people dealing with this. More later.... --Dave
- dhsmith_116072
Nimbostratus
This is the solution provided by F5 support. I have configured it and it is running but I haven't actually "tested" it yet. The only shortcoming from my original ideal is that both the GTM and LTM are doing upstream monitoring through the gateways, but I'm thinking that might not be a bad thing if one of them fails and the other is still working.
Excerpt from support response:As we discussed, I have opened Service Request 1-347018108 to track your request to monitor an ISP Link; to ensure that the LTM and GTM mark the appropriate configuration items as unavailable at the time of an ISP failure.
On the GTM, apply the following Solution:
SOL6848: Monitoring link objects on BI-IP GTM or BI-IP Link Controller system http://support.f5.com/kb/en-us/solutions/public/6000/800/sol6848.htmlFor the LTM:
SOL7215: Configuring multiple default routes on the BIG-IP system http://support.f5.com/kb/en-us/solutions/public/7000/200/sol7215.html SOL8971: Creating transparent ICMP health monitors https://support.f5.com/kb/en-us/solutions/public/8000/900/sol8971.htmlThese solutions provide the following:
-
Minimal monitoring.
- You will only need to apply monitoring to a single gateway pool on the GTM and a single gateway pool on the LTM.
-
You can then apply this gateway pool to all of your internal Virtual Servers.
-
The GTM will mark the links as unavailable when it detects the link has become unavailable.
-
- dhsmith_116072
Nimbostratus
OK, I just had to try hitting all the objectives so I did a little test. There are reasons you might not want it this way, but here it is.
First, I created several "transparent monitors" to verify communication through gateways. Then I created a pool on the LTM called LinkTestISP1. I assigned the transparent monitors to the pool and added a single member, the ISP1 router. This gives me a pool that is "UP" if any one of the monitors is ok. Then I created a virtual server (it seems unnecessarily indirect, but the communication I need to the GTM is only by virtual server) Call the server ISP1isOK, give it a completely bogus IP address (I used a non-existent and non-routeable address and bogus service port) and give it the default pool of LinkTestISP1. Now on the GTM I find a pool with one member using an address provided by ISP1 and I edit that member and at the bottom of the screen in the "Dependency List" box I can now find and add my virtual server ISP1isOK.
This sounds very cludgey probably because it is, but it gets me the objective of having only one device monitor the links. The other big advantage in this production environment where I do not have a "lab" to play with is that I was able to test this without disrupting any product services. I changed the health monitors to one designed to fail, watched the pool go down, watched the VS go down, and watched the GTM remove only that member address from it's pool (product still up on the primary address)
The big downside to this solution is that you must now edit ALL of the servers the GTM learns from the LTM and add the appropriate dependency (after building the same kind of testing for the other ISP's of course) That's a lot of config work, and I still haven't even addressed the config changes for the LTM so that the same tests failing will remove options from it's gateway pool for outbound traffic. So - conclusion is - the F5 suggested solution whereby both devices do upstream monitoring for the Links is the way to go. Now if I can only find a way to test it without disrupting actual product I'll be completely happy with it.
Cheers
- Tom_G__134358
Nimbostratus
Thanks for sharing, Dave.
It is indeed a weird way to achieve what link monitors seemed to be have been designed for.
I like the idea of adding the VS as a dependency in the GUI though, instead of using an "explicit link name" that needs to be set in TMSH...Thanks again.
Thomas
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com