One issue I see is that Ruby on Rails is a single threaded application. Meaning that if I want to scale my traffic/connection handling capabilities, I need to run multiple Mongrel webservers running our Rails app. (Fronted by a reverse proxy of some sort. In our case it is a BigIP 6400).
There are two major issues we have found with this.
1) Rails is a single threaded application, so that if one client request takes 5 seconds to respond to, it will queue up all other client HTTP requests until it has served the first). Most of the existing load balancing algorithms will queue up requests. Ideally we would like a load balancing algorithm that will make only one connection to each Webserver. (With extra traffic being handled as an exception case, that either goes to a specific pool, is redirected, or some other action.
2) The least connections load balancing algorithm is the closest suitable algorithm for Rails apps. However the problem is that in our case we have dozens of servers, with scores of webservers running on each server. When we are at a low load level, we are seeing all the traffic going to one webserver. e.g. - The first ten connections are going to ports xxx1-10 on webserver one. Only when traffic has been sent to all the webserver instances on webserver one, are we seeing any traffic being sent to webserver two.
Long term, 1 would be nice to fix, but for now, properly striping my traffic across my servers would be best (2).
Are you using least connections (member) or least connections (node)? If using (node), the traffic should distribute equally across the servers with no reference to pool counts.
My only concern is that if we were to use node it wouldn't also try to do least connections for the individual members on each node. IE: Once we have sent one connection to each node, the next connection should go to a member that doesn't have a connection. IE: If there are two connections being sent to a node, they should be on different members.
I would expect this to be the behaviour, but it is hard to know for certain without testing.
I don't think the algorithms can be nested like that, but that's an intriguing idea. I suppose you could keep the node balancing and use an array to keep track of each server's count within a pool and select the pool member in the array with the least number of connections. Not terribly efficient, though.
