Forum Discussion
nik_65678
Nimbostratus
Nimbostratusactive/active sites.
we've always been a one datacenter shop but now we'll be running active active from different locations. i do have a scenario that i've been unable to solve without a pretty gross hack.
here's the old 1-datacenter setup:
servers running a web app die. load balancer says "hey, no nodes are up" and reroutes to a "downtime" page. pretty simple, right? this however becomes silly when you have two datacenters.. here's what happens with no changes:
servers running the webapp at datacenter 1 die and downtime page is returned. app at datacenter 2 is running so the app is returned. some users get the downtime page, others get a working page. problem!
the two f5 pair are able to talk to eachother and eachother's members privately. the only solution i've come up with so far is making two pools for every single app, one for the app servers in each datacenter. here's the rule i'd create on the f5 in datacenter 1:
if (dc1 pool == up) --> working app
if (dc1 pool != up) & (dc2 pool == up) --> reject connection
if (dc1 pool != up) & (dc2 pool != up) --> downtime page
the opposite would exist on datacenter 2. it seems like a semi-reasonable solution but is obviously a hack.
does anyone have experience with multiple site solutions along with downtime/failover apps?
11 Replies
- The_BhattmanHi Nik,
What you are talking about can easily be solved by introducing a GTM system or similiar system in the mix. It would essentially monitor the virtual services on datacenter and send traffic in an active/active and active/passive scenerio.
I hope this helps
Bhattman 
nik_65678i'm not particularly familiar with the gtm device but from what i understand it's just a glorified dns server with application monitors. the one issue i have with using dns as a fail-over solution for this is regardless of the ttl updating records can take time.. often more time than i want to spend down.
hoolioCirrostratus
Hi Nik,
Ken Salchow from F5 discussed some of the purported downsides to DNS-based load balancing on vegan.net and here:
http://devcentral.f5.com/weblogs/ksalchow/archive/2009/06/03/shame-on-gslb-shame-on-me.aspx
With that said...
If you are happy with the client-to-each-datacenter connectivity/routing but want to take advantage of servers in both datacenters, you could configure a single pool for each VIP that contains the servers in both datacenters.
If you select a load balancing algorithm that takes into account the latency and prefers the lower latency servers, you could automatically select the local servers if they're available and the remote servers if no local ones are available.
Or you could manually configure this logic using priority group activation. You'd configure a pool with the local servers set with a higher priority than the remote servers. You could also specify a minimum number of active servers so that LTM would start using the lower priority pool members.
Does this sound closer to what you were thinking of? If not, can you clarify?
Thanks,
Aaron- The_BhattmanAnother possible solution, if you are running v10 is to use iSessions. Here is an article about it http://devcentral.f5.com/weblogs/dmacvittie/archive/2009/04/29/v.10---introduction-to-isessions.aspx
I hope this helps
Bhattman - nik_65678i haven't really read much on isessions but i want these two datacenters to be symmetric, and if i understand isessions correctly it's more along the lines of master/slave.
regarding the other solution - it looks very much like what i want to test, having nodes from BOTH datacenters in each pool. the only problem i'm not sure how to work around looks like this.
let's say www.fu.bar has two round-robin entries, one for dc1 and one for dc2. this would mean that most of the time a single user would hit one of those but sometimes request data from the other
1) dc1 up, wan up, dc2 up -> success in both locations
2) dc1 up, wan up, dc2 dn -> success in both locations
3) dc1 up, wan dn, dc2 up -> success in both locations
4) dc1 up, wan dn, dc2 dn -> failure in dc2 & no way to talk to dc1
kinda what i was looking to do was something along the lines of
if (i am up) and (remote is up) { return content }
if (i am up) and (remote is down) { return content }
if (i am down) and (remote is up) { break connection, round-robin goes to next ip }
if (i am down) and (remote is down) { return downtime page}
this would be easy if my company didn't want downtime pages :] - hoolioGTM would handle that logic nicely in the DNS resolution without worrying about disconnecting clients at the TCP layer.
Is there reasonably low latency between the datacenters? At the LTM level, if you had both sets of pool members enabled in one pool as described above, LTM would automatically use the remote servers if the local ones were unavailable so no disconnection would be necessary.
Else, you could take your original approach and define one pool of local servers and one of remote servers. The iRule logic you've described above would be simple using checks on the local pool status. And if the local pool was down you could check the status of the remote pool.
Aaron - The_Bhattman
Posted By Nik Ambrosch on 02/10/2010 10:15 AM
i haven't really read much on isessions but i want these two datacenters to be symmetric, and if i understand isessions correctly it's more along the lines of master/slave.
Not exactly. iSessions allows you to send traffic to another server in another datacenter as if it where hosted locally. So imagine 2 separate nodes in a single pool but in this case it's encrypted and using WAN optimization. This communication be initiated in both datacenters.
my 2 cents
Bhattman - hoolioiSessions could be a very efficient solution for this scenario. Nice idea Bhattman.
Aaron - nik_65678i don't entirely understand the documentation that's kicking around for isessions - is there a good howto that you know of?
- The_BhattmanI haven't see one out there explicitly but I will ask around and see if there is one upcomming in a configuration/solutions guide.
Bhattman
