Like a Matrushka, WAN Optimization is Nested

WAN Optimization seeks to reduce the cost of transferring data over the WAN from one data center to another. This group of technologies has been around for a while, but with the advent of cloud computing, it is getting more attention than it has in the past. Whether you are transferring data to your backup/regional datacenter or out to a cloud provider, they can serve the same purpose – to speed both the general connection and specific protocols. As more and more enterprises consider moving some functionality out to the cloud, these devices have seen increased interest simply because the pool of organizations interested in them grows from those who keep multiple active data centers to all enterprises making use of the cloud.

The best way to describe WAN Optimization is to compare it to a Russian Matrushka doll, with layers that contain each other and each adds something to the hole. You can skip a layer if you have the need (akin to  leaving a doll out of the matrushka), but each layer offers independent benefits and going without them should be considered carefully.

---

---

According to Wikipedia, WAN Optimization consists of dedup, compression, caching, protocol spoofing, traffic shaping, equalizing, connection limits, and simple rate limits. I would argue that the last four are all variants of traffic shaping – you are trying to control how much traffic is flowing through the pipe and what kind, which is the heart of rate shaping. I would add encryption to this list. The difference of opinion here is one of which pigeon-hole does a technology belong in. I feel that since the product is sending your data out on a public line, it is reasonable to expect that it handle encryption. It is also far more beneficial for the device to process data and then encrypt it, send it, decrypt it, rehydrate and uncompress it, and only then pass it on than it is for things like dedup and compression to be done on an encrypted stream. But Wikipedia isn’t wrong, because encryption isn’t strictly necessary in the scientific sense and reduces the efficacy of the optimizations performed, I just feel that for the device to be useful, encryption must be an option, that way if you encrypt by another method, you can turn it off, but if you don’t have an alternate method you can turn it on.

---

---

So coming in, you accelerate the protocol – what Wikipedia called protocol spoofing. Doing this first makes sense as the unencrypted stream will have the full set of back-n-forth commands that the protocol needs “spoofed”. Let us face it, CIFS sounds a lot like The Toddler with his “push this button” Optimus Prime toy. He says “I am Optimus Prime” about 20 times a minute, just as CIFS says “I am a server” over and over. Often this repetition is not necessary. Eventually I take the toy away from The Toddler and convince him to play with something less annoying. WAN Optimization is roughly the same, it takes the unnecessary bits out of protocol communication, leaving the important stuff. Implementing protocol acceleration correctly is definitely the geeky bits, but this has been going on for so long that it’s no longer rocket science. Most vendors support a variety of application protocols, check with yours to see what they cover and what’s on the roadmap.

 

 

 

---

---

Next it’s a good idea to do deduplication, simply because it must be done before compression – or deduplication will not be very fruitful. Dedup is another area that has been around a good long while, and the deduplication algorithms are utilized in a fairly wide selection of IT technologies, so they get a lot of exercise. At its simplest, this is a simple case of pulling out strings of matching bytes and replacing them with a placeholder, then telling the remote box what the placeholder is for “rehydration”. Of course there are a variety of ways to define “strings of bits” and “placeholder”, but  that goes way beyond the scope of this simple blog.

 

 

---

---

Once you’ve optimized the protocol and removed as much  as your flavor of deduplication will allow, we’re at a good point to apply compression. The remaining areas – caching and other TCP-based optimizations – can be handled on the compressed stream because only the compressed stream needs to be sent across the WAN. Compression is also a time-tested technology that applies standard compression techniques to the data being sent out – much the same as it can be applied to the data being saved in a file (though most file “compression” includes deduplication as a matter of course).

 

 

---

---

At this point, you’ve got three layers of optimization applied, and we take a bit of a step back to encrypt, which does take some overhead. How much overhead depends upon how the data is encrypted,  but in general it is not much. You are probably thinking “this data is useless in this form, it’s been modified, deduped, and compressed”, and there’s some validity in that argument, but let’s think of a determined Man-in-the-Middle attacker. Once they capture your stream, there are a finite number of compression algorithms for them to try to uncompress your data. Once it is uncompressed, a small amount of data analysis should show the pattern of deduplication because that technology is not a security function, and all of the dedup schemes I know of are pretty straight-forward in terms of examining the data and seeing patterns. That’s all the attacker needs to do – not something your average Joe would get through, but a resourceful hacker would not find it difficult at all. So encryption it is. This is a good place to apply it too, because the TCP optimizations should be working on the finished product – the sum of the other functions. As you know, encryption too is well-known and stable.

 

---

---

Next come an array of TCP optimizations that can be applied to limit the amount of overhead in your stream. What those optimizations are varies from product to product and even from one option within the product  to another, but the important bit is that it lowers the overhead of TCP. Less overhead is your friend.

And finally, rate shaping in its various forms. Rate shaping a connection – at this point let us call it a tunnel since we’re talking point-to-point – out of your datacenter has a two-sided benefit. It allows your tunnel to use as much bandwidth as you  require to maintain performance, and allows you to set the maximum amount of bandwidth your tunnel can use so that you are not hogging the entire connection and stifling other valid users. TCP rate shaping is another area that has been around a very long time, and is both useful and viable on a connection that is not dedicated to your tunnel’s traffic. Of course if you have a separate connection for just communications between the two data centers it is less useful, but still leaves room for other traffic that is not run  through the Optimization process.

---

---

As you can see, there is nothing terribly new or earth shattering in any of these items, though of course as time goes on developers have gotten better at writing them, new ideas have improved them, and they get more efficient, they are in general stable and effective. Taken as a whole, these techniques can have a huge impact on symmetric datacenter communications. Symmetric because some of the important bits – dedupe, compression, and encryption – require a box on the other end to reverse them. But check with your vendor for numbers, they definitely carry their weight, particularly if your connection is busy… Like it might be if it was pointing at the cloud.

Our matrushka – bought in Red Square – goes from six inches in diameter down to less than an inch. Not as impressive as WAN Optimization’s layers, but still impressive.

---

---

Why yes, F5 does have a solution set to address this issue. F5’s solution to this complex problem is twofold – BIG-IP LTM version 10 and higher (essentially any version with iSessions support) for Compression, Encryption, Rate Shaping, and TCP Optimizations, BIG-IP WAN Optimization Module  (WOM) for Protocol Acceleration and data De-duplication. I’ve seen actual results and test-based results for this and other vendor’s solutions, and if you’re looking at upgrading bandwidth to support dc-to-dc communication or dc-to-cloud communication, I would recommend including an eval of these products in your plan.