Intro to Load Balancing for Developers – How they work

So last time we were talking about when and why you might implement load balancing – either with a dedicated load balancer, or with a full-blown Application Delivery Controller (ADC). This time we’ll briefly run over what a load balancer does and how it does it.

Note that this is a very high level overview, but the end of the article includes links to other articles that offer you more detail if you need it. The terminology I use in this document is a mix of F5 terminology, industry terminology, and terms that make more sense in an appdev context than the normal networking terms applied to these products.

From a developers’ perspective, basic load balancing is pretty straight-forward. One machine doesn’t handle your needs, so a load balancer can offer you the ability to spread the work amongst several machines.

To accomplish this simple task, you need a few things. First off, you need the publicly broadcast IP address and DNS name. This obviously isn’t the actual address or DNS name of the machines running your application – since there will be several of them, and the network would have a cow if you put several machines on it with the same IP.

So we’ll call this address your Virtual IP – it really is nothing more than the address you’re advertising your application at.

Next up, the load balancer needs to know what servers it is balancing load across. This can be as simple as a list in low-end load balancing solutions, or a list of objects that can have properties applied to them as a group in an ADC. In the case where they’re a list that can’t really be manipulated other than additions/subtractions, we really don’t need a name for them, it’s just “the servers”. In the case of an ADC, there is some notion of the group as a whole. We’ll call this the pool of available servers. When you have a pool, you can add functionality at the pool level that we won’t get into right now, but suffice it to say there are circumstances where you might want to manipulate or redirect traffic after it has entered the VIP and before it hits a server.

Which brings us to our next level, the actual machines that service requests. Again, in the case of a list it is just that, a list of IP address/port combinations that your application can be found on. In the case of an ADC, they’re entries in the Pool that include IP Address and Name, but might contain a ton of other information, like weighting for load balancing, etc.

Finally, you need a load balancing algorithm. This tells the load balancer how you want to distribute requests coming in. The most commonly known load balancing algorithm is round robin – where the list is iterated, one connection sent to each server, then starting back at the beginning of the list. There are a ton other algorithms including weighted round robin (called ratio in F5 products), and predictive weighting. There are a ton more, but since each load balancer names and handles them differently, we’ll stick to the basics – round robin distributes loads evenly without thought to load or response time, ratio or weighted round robin, where there is a pre-configured weight assigned to each server, and adaptive, where the load on the server and other dynamic factors help determine where to send a new connection.

When we talk about ADC's, there is a lot more to it – there are many things you can do with advanced functionality – but we’re focused upon load balancing at this point in the series. Later in the series, we’ll talk more about the power of ADCs.

In the simplest of load balancers, and with most configurations of ADC, you cannot count on any user coming back to the same server connection after connection. This is where the rubber starts to meet the road for application developers. Relying on state in between trips to the server can be dangerous. That means you have to make certain you send all necessary state information back to the client – but most web apps are written that way anyway, and with an ADC you can get around this restriction.

The next thing to consider is how your load balancing solution is deployed. The two basic ways are as a proxy – sitting between your app servers and the world and accepting all traffic for them. The other configuration is to make the load balancer a gateway, assigning a user to a server on first connection and then leaving the client and server to carry on their conversation without the load balancer being involved. IN most cases, the network staff will have configured the load balancer, and it won’t likely effect you unless your application has a lot of return trips and long-running queries, then the gateway configuration (commonly called “sidearm”) can cause one server to bog down if the load balancing method doesn’t take feedback from the server.

That’s all there really is to it. Well, the basics of it anyway. There are those programming gotchas I mentioned in the first installment of this series and alluded to above, and we’ll get into those full-bore in the next installment.

And this is definitely a very high-level overview. There’s a lot more detail available in the following resources.

• Server Load Balancing Architectures – JavaWorld articles on programming and load balancing. Focused on Java of course, but the architectures apply to any language, just some of the solutions don’t.
• Load Balancing Web Applications – O’Reilly article on load balancing architectures. Again, the site is Java, but the article is not. Focused on DNS round robin, it helps with basic understanding.
• Network Load Balancing (NLB) in Windows 2008 – Windows only, programmer irrelevant, it provides another view of simple load balancing with software.
• The Linux Virtual Server Project – from a user to a code monkey, you’ll find bits about load balancing here to learn about.
• BIG-IP Datasheets – yeah, this is partially because I’m an F5 employee, but it’s also because there’s some good info about the functionality (load balancing and more) available in ADCs.

No doubt our competitors have some good info too – but I’m not linking to them here, please accept this one bit of reality, I am an F5 employee after all.

That’s it for this week, until next time!

Don.