What is HTTP Part II - Underlying Protocols

Last week in part one of this series, we took the 50,000 foot view of the HTTP protocol. HTTP defines the structure of message transfer for web resources, but doesn’t have anything to say about or do with the underlying infrastructure (through version 1.1, version 2 is a slight departure to my claim and we’ll cover that in part 12.) So before we move on to HTTP terminology next week, we’ll spend time today discussing internet plumbing.

Photo: Bob Duran on Flickr

The first two layers in the OSI model, physical and data-link, are consolidated in the network interface layer of the TCP/IP model. Consider the network interface layer to be a river and ship, a road and a car, a rail and a train, or a plane and smooth skies. The lowest level means for which messages are transferred doesn’t really matter to HTTP. It doesn’t care or have visibility into the network interface layer, so whether it’s frame-relay, ATM, sonet, dwdm, ethernet, etc…it doesn’t care.

IP Layer

The network layer (OSI model ) or the internet protocol (IP) layer (TCP/IP model,) is the next layer. Think of this as a shipping container your letters or boxes are contained in. That container left its source, was shipped via many potential shipping paths, and arrived at a destination. At that destination, the container is then opened and your letter or box is delivered to you (the next layer up, which would be TCP.) Now, HTTP still doesn’t care about this purely from a messaging standpoint, but you as an administrator, or the way the server is configured, might care about some of the information regarding that container, like the source and destination information.

It’s really amazing the internet functions as well as it does. It’s not one centrally-controlled organism, it’s built by thousands of companies with thousands of devices that just play hot potato all day long. Receive here, send there. Routing infrastructures are complex beasts, but aside from some security controls in place, they really just pass packets as fast as possible.

The transport control protocol (TCP) layer, or transport layer (OSI,) (as should be obvious by its name,) is the control freak of message transportation. The letter pulled from your container has overhead (the envelope) and payload (your actual letter.) The mailman doesn’t read your letter, he just reads the IP information (src/dst,) and validates postage, and any particular services you might have on that envelope. Say you sent 10 letters and they arrived in 4 different containers, the mailman would collect them as they arrived, and re-order them for you before handing up the chain. Ok, maybe that’s a lot to expect of a mailman, but hopefully the analogy is clear.

TCP Layer

At this layer, typically only the client and server machines, and intermediary devices like firewalls and proxies get involved. TCP is session-oriented and as such, there are expectations of behavior. Think of a phone call. Jason dials a phone number and waits for John to answer. Once John says “hey,” Jason received that acknowledgement that John sent, and can respond as well. So Jason and John have that “handshake” to start the call (the 3-way handshake: phone dial, acknowledge, acknowledge) followed by a meaningful conversation (payload) and finally a closing salutation (the 4-way close: I’m finished, I’m finished, good-bye, good-bye.) So it is with TCP. And what we do naturally with our speech, slow down my speech if John isn’t able to receive that many words at once, repeat myself if there was static in the line, TCP handles as well with it’s protocol definitions. So HTTP does not need to manage transport reliability, TCP does this on HTTP’s behalf. This is important to understand for highly available technologies like session mirroring, where it’s largely unnecessary overhead for protocols like HTTP.

As with IP information, there is some TCP information you might care about with HTTP, like src/dst ports, or other header information like Akamai does with TCP options to insert original client IPs hiding behind proxies, but very little of that information is useful or helpful at the HTTP layer.

DNS

DNS is not a layer, but a critical application that combined with HTTP makes the web work. Now back in my network engineer days (and my younger days when I had a better memory) I used to call DNS a crutch for people without the fortitude to remember IP addresses. DNS, or the domain name system, exists to map names to numbers. Largely, people remember names better than numbers. DNS, like the IP infrastructure it relies on, is a patchwork of individually controlled and managed systems in an hierarchical architecture of domains. For more details on how DNS works and why it’s important, check out Peter’s DevCentral Basics What is DNS? article.

Putting It All Together

When you type https://devcentral.f5.com/s into the locator bar of your browser, the first task is not to send an IP packet to our server. First, it does a DNS lookup for that name, checking your system’s cache, then to the local DNS server configured on your server. At that point, the LDNS server will do the lookups on your behalf and retrieve that answer, then return it to your system, where the answer will be cached for some period of time (hopefully at or shorter than the specified time to live.)

At that point, your system will attempt to establish a session with our server (dial the phone and greet each other.) If they reach an agreement, only THEN will the HTTP message be sent from the client to the server. A simple flow of the HTTP traffic can be seen in the image below

Conclusion

HTTP messages aren’t exchanged in a vacuum. A robust infrastructure of routers, firewalls, proxies, switches, and assisting protocols like DNS exist to help make the web work. Come back next week when we begin the journey into the specifics of the HTTP protocol.