It’s been a while - a long while - since I last (officially) blogged. Too many distractions with a new role, new travels, and a new family member - sucks away the creative juices. Alas, sitting at an airport lounge after a 3 city Security Roadshow in Bombay/Delhi/Bangalore brings out the adrenaline rush of blogging once more.
I’ve been evangelizing the need for DoS protection for quite a while. Events of last year (2013), and even the beginning of 2014, have made my job easier (Happy New Year!). I used to equate DoS (rather Distributed DoS) attacks with Bot Armies - kinda like the Orcs that we saw in the LOTR (Lord of the Rings**) saga. The Bots (Orcs) still exist - but seems like the Bot Herder (I’ll call him “Sauron” **) is summoning a new type of Army these days - a new generation of orcs used to carry out his mission to wipe a particular service/server from the face of the Internet world - You and Me!
Yes - US!
2014 began with an interesting new type of attack against a couple of gaming sites like League of Legends, Steam and Battle.net using yet another type of “botnet”, except that these weren’t bots but actual Network Time Protocol (NTP) servers. The NTP protocol is used by many servers and endpoints to sync system clocks - for example my mac uses time.asia.apple.com. Leveraging the inherent “trust” that the UDP protocol exhibits (aka stateless), and a relatively forgotten command called “monlist”, this is how it works. From my laptop, a small query of 234bytes returns a response of 100 packets each of 482bytes, producing an amplification factor of approximately 205x (you get the hint). So I need to be armed with a list of open and unpatched NTP servers (which isn’t difficult to find - there are automated programs out there that can “assist"), and spoof the IP address of someone whom I don’t like, and shazam.
An example of the now infamous “monlist” command for an unpatched NTP server looks like:
This reminds me of the DNS Reflective Amplification attacks during the Cyberbunker-Spamhaus saga in March of last year (2013).
Again, the trust model of the DNS protocol was exploited, and what that didn’t help with the problem were those 20-something million open DNS resolvers which neither perform any kind of filtering nor any request verification or traffic management of DNS responses. And as a result, 300Gbps of DNS responses towards an unsuspecting victim, which even caused collateral damage to International Peering exchanges.
Now, how do we deal with these?
For the DNS and NTP reflective amplification attacks, apart from deciding to not be an open DNS resolver, or disabling the “monlist” command, if you are a service or hosting provider to one of these potential “Orcs”, one simple way is to monitor the outbound BPS/PPS/Number of connections originating from the server. When it gets anomalously high, move to the next step of monitoring perhaps the Top 10 destinations these guys are sending towards. It doesn’t make sense that a DNS server out there is sending 1M DNS “Any” responses per second to www.f5.com. Apply an appropriate rate shaping policy as needed and that will help protect your own network infrastructure, save some peering/transit costs, and help the poor victim.
The HTTP reflection attack is a bit more challenging, given that as mentioned previously - it is REAL HTTP traffic from REAL browsers out there. You can’t filter based on malformed HTTP packets. You can’t filter based on URL since it’s typically a legitimate URL.
This calls for Anti-Bot Intelligence, like:
With the ability to step-up countermeasures dynamically and when needed. Of course, the latter verification methods are higher in the OSI stack (Layer 7), more advanced and computationally more expensive to perform.
Lastly, work with your upstream service provider, or engage a cloud-based anti-ddos service when the attacks become too large to handle. Sound the SOS! It is common-sense the only possible way to mitigate 300Gbps floods is in the service provider cloud / cloud-based anti-ddos service, likely using a distributed scrubbing “anycast” model, where each distributed scrubbing center sucks in the traffic destined for the victim like a big washing machine and starts cleaning.
That said, there are many ways to skin a cat - likewise, there are many ways to solve a DoS problem. Pay particular attention to attacks as they move up the OSI stack toward Layer 7. These are typically harder to mitigate, but fortunately not for F5.
So if you’re thinking that botnet armies are the Sole source of menace in the DoS world out there, they’re not. Everyday Internet users like you and me could be an unsuspecting participant in the world’s largest DoS to be.
** Source: The Lord of the Rings, J.R.R Tolkien, 1954