F5 BIG-IP ASM – L7 DDoS attack Protection (TPS, Behavioral & Stress Based detection)
I would like to extend a big thank you to Mr.Mohamed_Ahmed_Kansoh for his invaluable help in creating this article on the F5 BIG-IP ASM - L7 DDoS protection You can also check this article on: https://nishalrai.com.np/2023/07/14/f5-big-ip-asm-l7-ddos-attack-protection/ F5 BIG-IP ASM DoS profile provided two distinct types of detection of DoS protection – TPS-based detection and Behavioral & Stress-based detection. TPS Based detection On threshold mode, if it is configured automatic then the TPS-based DoS protection is triggered, “If the ratio of the transaction rate during the detection interval to the transaction rate during the history interval is greater than the percentage indicated in the TPS increased by setting, the system considers the URL to be under attack, or the IP address to be suspicious.” Reference:Preventing DoS Attacks for Layer 7 Traffic On this post, We will discuss about the DoS protection profile especially the Behavioral and Stress Based detection features available on the F5 BIG-IP ASM module. One can find the DoS security profile protection profile on F5 Main Dashboard > Security > DOS Protection Behavioral & Stress based detection The server-based detection only implements the mitigation when there is significant latency from the backend server responses. In case of the occasional high spike of CPU usage on the backend server, it does trigger the stress-based DoS but when you view the output of the event timestamp. This sums up the significant latency on the server response, that ends up the stress-based DoS. To view the server response latency: Statistics > Dashboard > Behavioral DoS Regarding the behavior DoS protection of F5 BIG-IP ASM, the behavioral method is an intelligent way to identify and mitigate DoS attacks by creating a good baseline from normal traffic – users from normal or legitimate traffic. One needs to allocate sufficient time for F5 BIG-IP to create its baseline of learning good traffic. One can monitor the learned baseline of traffic on F5 BIG-IP: [root@bigipo01:Active:Standalone] config # admd -s vs./Common/vs_hackazon_http+/Common/hackazon_bados.info.learning - /Common/vs_hackazon_http – is the name of the virtual server - /Common/hackazon_bados – is the name of the DoS profile. **It may take several minutes for baseline numbers to be generated** Ref: Base Configuration and Traffic Baseline Behavioral Detection and mitigation stage When the F5 BIG-IP discovers any abnormal pattern of traffic that violates its learned baseline and observes the latency increased in Sever. It starts to create signatures for these attack patterns and starts to slow down / rate limits the request on the basis of the implemented mitigation technique – standard protection, aggressive protection, and others. F5 BIG-IP takes some headers from the violating/attack requests “abnormal requests ones” and stores it in a signature form, then these headers are bundled with each other within a single signature to identify the attacked request. So these are the parameters that F5 BIG-IP ASM DoS protection uses to create the signatures. In the ASM DoS profile, if “Use approved signature only” has been enabled then the administrator needs to open the signature and review its headers and information in order to approve it. In case you feel that it’s an attack then you need to disable it. If “Use approved signature only” is enabled in F5 BIG-IP ASM DoS profile, it will use the created signatures immediately without the administrator’s approval and will start to block any request that does not match the approved signature. Reference: Behavioral and stress-based detection settings Stress-Based detection The stress-based detection can be configured for detection criteria for server latency and transactions-per-second (TPS). F5 BIG-IP ASM marks the IP address as suspicious if any attacks trigger the set latency thresholds by the F5 BIG-IP. Once it has been triggered, the DoS engine starts collecting TPS information for the suspicious IP then analyzes TPS history rate to determine if it is a real DoS attack or a false positive. When the suspicious criteria TPS is reached, the BIG-IP ASM drops connections based on the policy - either by source IP or per URL. If the TPS (per IP address) is reached, the prevention policy switches to Source IP-Based Rate Limiting mode. If the TPS (per URL) is reached, URL-Based Rate Limiting is applied, and connections are reset. If the request is still considered an attack, the second and third policy are attempted with a two-minute interval between them. The BIG-IP ASM allows you to set a Prevention Duration to limit connections during a DoS attack. This defines the F5 BIG-IP ASM to considered the configured time period to either escalate or deescalate the configured mitigation methods – client-side integrity, CAPTCHA challenger or even block the request. Rate-Limit Prevention The rate-limiting is a mechanism used by the F5 BIG-IP ASM to prevent DDoS attacks on the protected applications or services. The F5 BIG-IP ASM calculates the average Transactions per seconds (TPS) for both the Detection Interval that lasts 10 seconds and the Historical Interval (last hour). Then, it updates them every 10 seconds. To rate-limit incoming traffic, F5 BIG-IP ASM uses a simple equation as: (History Interval TPS + Configured Threshold) / 2 For instance, if the absolute threshold is set to 200 TPS and the Historical Interval TPS average is 100 TPS, then the BIG-IP ASM will rate-limit incoming traffic to 150 TPS. The F5 BIG-IP ASM will rate-limit incoming traffic if the TPS exceeds the absolute threshold or if the sum of the relative threshold and the current TPS exceeds the absolute threshold. Note: You can review it from logs when attack was initiated, you can observe the limit which F5 BIG-IP was set and how it violated by crossing those configured threshold when the attack started. You have the option to record the DoS traffic on the DoS protection profile. During a DDoS attack, the recorded packet capture file shows that some traffic receives RST packets from the BIG-IP ASM, while other traffic receives normal responses (200 OK). This indicates that, while rate-limiting can prevent some traffic from passing through the BIG-IP ASM, it is not a foolproof method. Compared to blocking all traffic, rate-limiting allows some traffic to pass through the BIG-IP ASM, while blocking all traffic resets all traffic from a specific source. Bad Actors behavior detection On the bad actor’s behavior detection, F5 BIG-IP monitors and tracks the server response time. Once significant server latency is observed then starts to keep the track of all the ingress traffic request ip address, monitors their request and calculates the value (kind of score either to quarantine or discard the further request). To display the IP greylist: ipidr -l /Common/<vs-profile>+/Common/<dos-profile> To view the current maximum TPS of the DoS profile. tmsh show security dos profile my_dos_profile In-Sight of F5 BIG-IP ASM DoS Profile Sometimes the spike in CPU usage on the backend server triggers the attached ASM DoS profile. So, what would be the F5 BIG-IP approach to learn and detect those spike patterns as a DoS attack like for certain intervals of time - will it ignore those spikes and only after a certain interval of time with a predefined % of CPU usage of the backend server like +90% for 5 minutes or less that triggers the DoS profile. and if so, can we customize those defined parameters on the DoS profile. Such behavior has resulted into a lot of false positive. Stress-based mitigation will not be triggered if only configured TPS was exceeded and high latencies coming from servers are in AND Boolean condition. With such configuration, even if servers encountered spike due to high number of transactions TPS, F5 BIG-IP will not start mitigation if only there is high latency in server side. In short, even if the dos profile is triggered when the server-side latency is automatically spiked without any violation of the configured baseline TPS, the implemented mitigation will not be enforced on the new inbound traffic. Moreover, we cannot manually configure the baseline of those CPU usage and server latency percentage threshold. Even though F5 has released the white paper addressing those changes can be possible in the F5 BIG-IP, those changes are not possible on the major version or may not be possible on the preceding version too. Reference: Intelligent Layer 7 DoS and Brute Force Protection for Web Applications | F5 White Paper dosl7d dosl7d daemon runs in the background which will continuously monitor the two major variables – Memory Usage and TMM count. The services and scripts attached to the dosl7d process All those logics and algorithms to detect those DoS detection based on the behavioral and stress-based detection has been compiled and stored on the dosl7 dosl7d daemon responsible for detection of the DoS attacks (when triggered) and the log are stored on the /var/log/dosl7d In the directory, /etc/bigstart/startup Scripts of dosl7d - /etc/bigstart/scripts Sample of dosl7d logs (CLI & GUI) F5 BIG-IP GUI Why can't we configure SNMP alert for L7 dos attack on F5 BIG-IP, right now? In F5 Big-IP v 11.3.0 and later, messages generated by the dosl7d daemon cannot trigger commands or custom scripts because they are not processed by the alertd SNMP process. This is because the dosl7d daemon writes directly to its log file (/var/log/dosl7d/dosl7d.log) instread of using syslog facilities. As a result, the messages the dosl7d daemon issues do not pass through the syslog pipe. So, the alertd daemon cannot see the dosl7d messages and unable to trigger SNMP traps. The only current options for detecting a DoS attack is through an external logging device. The other workaround which will implicitly solve the existing issue by creating an email alert notification when the configured threshold (manual TPS of the DoS profile or server latency or others) has been triggered on the selected virtual instances, pools or application. However, the limitation of this configuration can be the inability to select the specific virtual instances and pools to configure separate threshold values to trigger. But it can somehow mitigate the issues of implementing dos protection mitigation without being noticed by the F5 administrator on high critical business oriented applications or services.
