1. SYN Cookie: Intro

Introduction

As for today one of the most common and easiest DDoS attacks to carry out is TCP SYN flood attack. Due to this, many efforts have been dedicated to implement the best solution to mitigate it. This is why you can find different countermeasures with the same goal, like reducing SYN-RECEIVED timer TCP state, increasing backlog, recycling old TCBs, using TCPCT (RFC6013), implementing SYN cache, etc. but the widest spread choice is TCP SYN Cookie.

In this article series I explain several aspects of SYN Cookie like how this countermeasure works and how is implemented in BIG-IP, how must be configured, differences between hardware and software SYN Cookie, things to take into account when dealing with SYN Cookie and troubleshooting. All examples shown in these articles are based on TMOS v16, although there could be some slight difference with your specific version it will help you to understand what it is happening in your own system.

Note that, SYN Cookie term refers to the countermeasure itself, but sometimes you will read SYN Cookie meaning in fact SYN Cookie challenge. I will talk about the difference throughout the articles.

As a final comment, please be aware that I have tried as much as possible to avoid talking about DB keys or internal commands throughout these article series. In any case, as a rule of thumb, please do not change any DB key unless this change is recommended by an F5 engineer.

Why SYN Cookie?

If you review the well-known TCP state diagram you can probably notice a weak point. Below I have extracted the section of TCP diagram that describes a TCP passive open and I have filled in red the specific TCP state that support the TCP SYN flood attack. Reason is that in SYN_RCVD state the system already reserves memory for the incoming connection information, which is saved in the so called TCB (Transmission Control Block). This means that the system starts to consume memory even before the connection is already ESTABLISHED.

Fig1. TCP state diagram section

 

This can be dangerous due to a simple reason, if a device running a server and waiting for connections receives a huge enough amount of TCP SYN packets then the service could be unavailable for legitimate users. This is because, according to TCP standard, the server must answer with a SYN/ACK packet to all these TCP SYN packets, create a TCB entry for each connection and also wait for last ACK from client. Therefore, the server will need to allocate memory, even without knowing if these connections will be finished successfully or not. 

This is the base of a TCP SYN flood attack. An attacker just needs to generate enough amount of TCP SYN packets to overwhelm a server. Any ISP will allow TCP SYN packets inside their network infrastructure since they cannot know if these packets are legitimate or not, and hence, all these packets will reach the victim.

TCP SYN flood attack can cause two different symptoms depending on reachability of attacker’s source IPs. Attending to which one you face maybe you could have clues about the origin of the attack:

1. Source IPs from where the TCP SYN flood attack is being run are reachable from the attacked device.

In this case when the target victim replies with expected responses (SYN/ACK) to source IPs the source IPs will response with a RST packet to the server since they are not aware of any TCP connection started from them.

 

Fig2. DDoS TCP SYN Flood attack (bots)

 

This behaviour will cause that the server most probably detects a TCP RST flood attack together with TCP SYN flood attack:

 

Feb 22 10:12:54 slot1/AFM1 err tmm2[14782]: 01010252:3: A Enforced Device DOS attack start was detected for vector TCP RST flood, Attack ID 31508769

 

The good news in this case is that server will be able to free up space for more TCBs since upon receiving the RST packet TCP state changes to CLOSED and then to LISTEN and therefore reserved space used for TCP connection will be freed up.

This is not the common behaviour since what an attacker wants is waste all server resources. Typically, source IPs are random and not reachable from server.

Attending to this explanation, if you face a RST flood together with a SYN flood attack maybe attacker is just sending an hping3 from internal subnets, or near to your Big-IP device.

2. Source IPs are not reachable from the attacked device.

This is the typical behaviour since, as commented, the idea for TCP SYN flood DDoS attack is consuming all server resources dedicated to TCP connections in order to avoid providing the service to legitimate users.

Note that when I say ‘reachable’ I mean that TCP SYN/ACK packets sent by server never has a response. Source IP could be reachable by ICMP or by starting a new TCP connection from server, but unrelated SYN/ACK packets from server are dropped at any point in the path, or even in the client itself. So the result is the same as if they were not reachable at all. In this case TCP SYN/ACK packets sent by the server will be lost and hence space for TCB will be reserved until timeout.

 

Fig3. TCP SYN Flood attack (generic)

 

Note that the word ‘attack’ in this article series does not mandatorily refers to intentional attacker, but also can refer to legitimate devices doing something wrong by error or ignorance. Also a wrong SYN Cookie configuration can warn us about a non real TCP SYN flood attack.

Conclusion

At this point you know basic theory under TCP SYN flood attack, so the question now is, how can you avoid that a simple TCP SYN packet reserves space for TCB entry?

You cannot. Since you have to keep with TCP standard only choice left is rejecting the TCP SYN packet and close the connection. Then the question is how to guarantee that legitimate clients will be able to connect to the server if you close their connections, and solution is, using TCP specification to your benefit.

This is what TCP SYN Cookie does and what I will review in the next article, showing how it is implemented in BIG-IP.