Achieving Command and Control through Requests and Responses

I’ve been talking about layer 7 load balancing (L7 LB) since, well, for a long time. From the first inception of it back in the day when someone decided that routing requests using URIs and host headers was a pretty innovative thing to do. If you must know, that was back in 2001.

And it was innovative then. Because at the time, load balancing and routing was something addressed at layers 3 and 4 – that’s TCP/IP – in the network using routers and switches and load balancers and network architecture.  You didn’t commonly see network devices operating at L7. You just didn’t, except in the app infrastructure.  

Today you see it all the time in the network. That virtual server definition in httpd.conf that relies on HTTP host header? That’s part of L7 LB. Rewriting URLs? Part of L7 LB. Persistent (sticky) sessions? You got this, right? Right. L7 LB.

So basically I’ve spent most of this century preaching about L7 LB.

One Monday morning in May I was reading the Internet (cause I do that on Mondays) and came across a lengthy discussion of microservices and L7 LB.

Guys, I must tell you I was totally excited by this blog. I was excited by the content, by the focus on the role of L7 LB in microservices and emerging app architectures (he mentions canary deployments), and by the words the author used to seamlessly move what has been a traditionally network-focused technology into an ops-focused technology. This is, without a doubt, one of the best (and most concise) descriptions of L7 LB I’ve read on the Internet:

It’s this experience that motivated linkerd (pronounced “linker dee”), a proxy designed to give service operators command & control over traffic between services. This encompasses a variety of features including transport security, load balancing, multiplexing, timeouts, retries, and routing.

In this post, I’ll discuss linkerd’s approach to routing. Classically, routing is one of the problems that is addressed at Layers 3 and 4—TCP/IP—with hardware load balancers, BGP, DNS, iptables, etc. While these tools still have a place in the world, they’re difficult to extend to modern multi-service software systems. Instead of operating on connections and packets, we want to operate on requests and responses. Instead of IP addressees and ports, we want to operate on services and instances.

I highlighted that one part because man, there’s just so much wrapped up in that single statement I can’t even. Literally.

The concept of operating on requests and responses is the foundation of entire solution sets across security, scale, and performance. A proxy capable of inspecting requests and responses is able to note only deal with transport security (TLS/SSL offload) and load balancing, but app security, as well. Request and response inspection is a critical component of app security, scanning and scrubbing of content deep down in the payload (the JSON, the HTML, the XML) to find exploits and malicious content is the premise of a web application firewall.

And then there’s access control, which increasingly cannot simply rely on IP addresses and user names. The proliferation of cloud and roaming, mobile employees and users alike means a greater focus on controlling access to applications based on context. Which means operating on requests and being able to extract a variety of information from it that will provide richer access policies able to cross the chasm from users to things (devices).

And of course there’s scale. Scale today is not about load balancing algorithms, it’s about architecture. Application and operational architecture alike. The use of DevOps-driven deployment patterns like canary and blue-green deployments as well as sharding and partitioning architectures are critical to achieving not just the seamless scale required today but the efficacy of those architectures. L7 LB is key to these endeavors, enabling fine-grained control over the routing of requests and handling of responses between apps (micro or monolith) and users (thing and human). 

And that’s really what the aforementioned (did I mention it was awesome, already?) is talking about: L7 LB. Whether it’s hardware or software, in the cloud or on-premises, isn’t really all that important. That’s an operational detail that is (or should be) irrelevant when we’re talking about architecting a scalable application composed of “services and instances.”

I cannot reiterate often enough the importance of L7 LB as part of modern application architectures. And it’s exciting to see the dev and ops side of the world starting to shout the same thing as they encounter the operational challenges of scale and routing amidst a highly interconnected and interdependent set of services that are the foundation for apps (and business) today.