Mitigating Slow HTTP Post DDoS Attacks With iRules

Hi Maddox,

 

 

I'm guessing you're hitting a case where there isn't a Content-Length header in a POST request. There should probably be more validation added to the example iRule to ensure more than 0 and less than 1Mb of data is collected. Here's an updated version you could try based on George's example. I haven't tested the default collection amount or timeouts, so you give this an extra review.

 

 

If you try this, could you let us know how it goes? You can use the R U Dead Yet tool to try this:

 

 

http://code.google.com/p/r-u-dead-yet/downloads/detail?name=R-U-Dead-Yet-v2.0.tar.gz&can=2&q=

 

 

Thanks, Aaron

 

 

 Based on 'Mitigating Slow HTTP Post DDoS Attacks With iRules' from George Watkins
 http://devcentral.f5.com/Tutorials/TechTips/tabid/63/articleType/ArticleView/articleId/1086402/Mitigating-Slow-HTTP-Post-DDoS-Attacks-With-iRules.aspx
 Requires LTM v10.0+ for the after command
rule block_slow_post_requests {
when RULE_INIT {
    Default amount of request payload to collect (in bytes)
   set static::collect_length 2048
    Default timeout for POST requests to send $collect_length bytes (in seconds)
   set static::timeout 2
}
when HTTP_REQUEST {
    Only check POOST requests
   if { [HTTP::method] equals "POST"} {
       Create a local variable copy of the static timeout
      set timeout $static::timeout
       Check for a non-existent Content-Length header
      if {[HTTP::header Content-Length] eq ""}{
          Use default collect length of 1k for POSTs without a Content-Length header
         set collect_length $static::collect_length
      } elseif {[HTTP::header Content-Length] == 0}{
          Don't try collect a payload if there isn't one
         unset collect_length
      } elseif {[HTTP::header Content-Length] > $static::collect_length}{
          Use default collect length
     set collect_length $static::collect_length
      } else {
      Collect the actual payload length
         set collect_length [HTTP::header Content-Length]
          Calculate a custom timeout based on the same ratio we use for the default collect length and default timeout
         set timeout [expr {[HTTP::header Content-Length] / $static::collect_length * $static::timeout}]
      }
       If the POST Content-Length isn't 0, collect (a portion of) the payload
      if {[info exists collect_length]}{
          If the entire request hasn't been received within X seconds, send a 408, and close the connection
         set id [after $timeout {
            HTTP::respond 408 content "Your POST request is not being received quickly enough. Please retry." 
            TCP::close 
         }]
          Trigger collection of the request payload
         HTTP::collect $collect_length
      }
   }
}
when HTTP_REQUEST_DATA {
    Check if the 'after' ID exists
   if {[info exists id]} {
       If all the POST data has been received, cancel the connection closure
      after cancel $id
   }
}

Thanks for your reply!

 

I will try more after lunchtime.

 

 

I got this messages:

 

 

Nov 30 08:06:08 local/tmm1 err tmm1[22762]: 01220001:3: TCL error: block_slow_post_requests - can't unset "collect_length": no such variable while executing "unset collect_length"

 

 

Regards Mattias

Is this config appropriate for handling slow post dilemma?

 

 

when RULE_INIT {

 

set static::timeout 20

 

}

 

 

when HTTP_REQUEST {

 

if { [HTTP::method] equals "POST"} {

 

if {[HTTP::header Content-Length] > 1}{

 

set collect_length [HTTP::header Content-Length]

 

set timeout $static::timeout } }

 

 

if {[info exists collect_length]}{

 

set id [after $timeout {

 

HTTP::respond 500 content "Your POST is to slow"

 

TCP::close

 

}]

 

 

HTTP::collect $collect_length

 

}

 

}

 

 

when HTTP_REQUEST_DATA {

 

if {[info exists id]} {

 

after cancel $id

 

}

 

}

 

 

TCL error: slow_post_prod - Illegal argument. Can't execute in the current context. (line 1) invoked from within "HTTP::respond 500 content "Your POST is to slow" "

Hi Mattias,

 

 

That would still be susceptible to crashing TMM if a client sent more than 4Mb of request payload. I'll try to rework my example and post an updated rule.

 

 

Aaron

Here's an updated version. I've fixed a couple of issues including the unset collect_length error and a maths rounding problem. Can you try testing this on a non-production virtual server first and reply back with the results?

For requests with a content length header value greater than $static::collect_length bytes, the iRule collects $static::collect_length bytes and gives the client up to the $static::timeout in seconds to send the data.

For requests with a content length header value greater than 0 and less than the default collect length (2048 bytes), the client is given the same ratio of bytes per second as the defaults ($static::collect_length / $static::timeout).

This method of collecting a small amount of data even for large requests is more efficient for LTM, but leaves open the possibility of a client sending the first part of the payload fast enough but then slowing the transmission down to keep the server busy. If you're concerned about security more than performance, you could increase the collection size. You'd probably want to increase the timeout as well to ensure slow clients aren't unintentionally blocked.

Thanks, Aaron

 Based on 'Mitigating Slow HTTP Post DDoS Attacks With iRules' from George Watkins

 Requires LTM v10.0+ for the after command

when RULE_INIT {

    This iRule enforces a minimum length of time ($static::timeout) for a client to send POST request data.
    The initial values are 2Kb / 2 sec = 1 Kb/s for the first 2Kb.  These values should be tailored for the client base.

    Default amount of request payload to collect (in bytes).
    This is the maximum amount of content we will collect.
    Clients will still be able to send unlimited payload sizes.
   set static::collect_length 2048

    Default timeout, for POST requests, to send $collect_length bytes (in milliseconds)
   set static::timeout 2000

    HTML response to send for blocked requests
   set static::block_html {Your POST request is not being received quickly enough. Please retry.}

    Log debug messages to /var/log/ltm? 1=yes, 0=no.
   set static::post_debug 1
}
when HTTP_REQUEST {

    Only check POST requests. If the application supports other request methods with payloads, add them in a switch statement here.
   if { [HTTP::method] equals "POST"} {

      if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: POST to [HTTP::host][HTTP::uri]" }

       Create a local variable copy of the collection amount
      set collect_length $static::collect_length

       Create a local variable copy of the static timeout
      set timeout $static::timeout

       Check for a non-existent Content-Length header
      if {[HTTP::header Content-Length] eq ""}{

          Use default collect length for POSTs without a Content-Length header
         set collect_length $static::collect_length
         if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: No Content-Length value" }

      } elseif {[HTTP::header Content-Length] <= 0}{

          Don't try to collect a payload if there isn't one
         unset collect_length
         if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: 0." }

      } elseif {[HTTP::header Content-Length] > $static::collect_length}{

          Use the default collect length
         set collect_length $static::collect_length
         if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: [HTTP::header Content-Length], collecting $collect_length" }

      } else {

          Collect the actual payload length
         set collect_length [HTTP::header Content-Length]

          Calculate a custom timeout based on the same ratio we use for the default collect length and default timeout
         set timeout [expr {[HTTP::header Content-Length] * $static::timeout / $static::collect_length }]
         if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: [HTTP::header Content-Length], collecting $collect_length bytes with timeout $timeout ms" }
      }

       If the POST Content-Length isn't 0, collect (a portion of) the payload
      if {[info exists collect_length]}{

          If the entire request hasn't been received within X seconds, send a 408, and close the connection
         set id [after $timeout {
            if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: $timeout ms reached. Closing connection" }
            HTTP::respond 408 content $static::block_html Connection Close
            TCP::close 
         }]

          Trigger collection of the request payload
         HTTP::collect $collect_length
         if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Collecting $collect_length" }
      }
   }
}
when HTTP_REQUEST_DATA {

   if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes." }

    Check if the 'after' ID exists
   if {[info exists id]} {

       If all the POST data has been received, cancel the connection closure
      if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Canceling \$id: $id" }
      after cancel $id
   }
}

One way to test this is to use netcat from the LTM command line. Type: 'nc . Then type the POST request line, a Content-Length header, two enters and wait.

$ nc 10.1.0.15 80
POST / HTTP/1.0
Content-Length: 5000

HTTP/1.0 408 Request Time-out
Server: BigIP
Connection: close
Content-Length: 69

Your POST request is not being received quickly enough. Please retry.

The /var/log/ltm will show the debug logging:

< HTTP_REQUEST>: 10.1.0.1:56236: POST to /

< HTTP_REQUEST>: 10.1.0.1:56236: Content-Length: 5000, collecting 2048

< HTTP_REQUEST>: 10.1.0.1:56236: Collecting 2048

< HTTP_REQUEST>: 10.1.0.1:56236: 2000 ms reached. Closing connection

Sorry for bringing up an old topic. But thought would be useful to add my own edited copy of the above irule.

The addition is rejecting any verb except for those specified. Helpful when a server on the back end is accepting any verb.

 Based on 'Mitigating Slow HTTP Post DDoS Attacks With iRules' from George Watkins

 Requires LTM v10.0+ for the after command

when RULE_INIT {

     This iRule enforces a minimum length of time ($static::timeout) for a client to send POST request data.
     The initial values are 2Kb / 2 sec = 1 Kb/s for the first 2Kb.  These values should be tailored for the client base.

     Default amount of request payload to collect (in bytes).
     This is the maximum amount of content we will collect.
     Clients will still be able to send unlimited payload sizes.
    set static::collect_length 2048

     Default timeout, for POST requests, to send $collect_length bytes (in milliseconds)
    set static::timeout 2000

     HTML response to send for blocked requests
    set static::block_html {Your POST request is not being received quickly enough. Please retry.}

     Log debug messages to /var/log/ltm? 1=yes, 0=no.
    set static::post_debug 1

     List of http methods supported by webserver
    set sec_http_methods [list "GET" "PUT" "HEAD"]

}
when HTTP_REQUEST {

    if { [matchclass [HTTP::method] equals $::sec_http_methods] } {

        if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: POST to [HTTP::host][HTTP::uri]" }

         Create a local variable copy of the collection amount
        set collect_length $static::collect_length

         Create a local variable copy of the static timeout
        set timeout $static::timeout

         Check for a non-existent Content-Length header
        if {[HTTP::header Content-Length] eq ""}{

         Use default collect length for POSTs without a Content-Length header
        set collect_length $static::collect_length
        if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: No Content-Length value" }

        } elseif {[HTTP::header Content-Length] <= 0}{

             Don't try to collect a payload if there isn't one
            unset collect_length
            if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: 0." }

            } elseif {[HTTP::header Content-Length] > $static::collect_length}{

                 Use the default collect length
                set collect_length $static::collect_length
                if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: [HTTP::header Content-Length], collecting $collect_length" }

            } else {

             Collect the actual payload length
            set collect_length [HTTP::header Content-Length]

             Calculate a custom timeout based on the same ratio we use for the default collect length and default timeout
            set timeout [expr {[HTTP::header Content-Length] * $static::timeout / $static::collect_length }]
            if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Content-Length: [HTTP::header Content-Length], collecting $collect_length bytes with timeout $timeout ms" }
      }

             If the POST Content-Length isn't 0, collect (a portion of) the payload
            if {[info exists collect_length]}{

             If the entire request hasn't been received within X seconds, send a 408, and close the connection
            set id [after $timeout {
            if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: $timeout ms reached. Closing connection" }
            HTTP::respond 408 content $static::block_html Connection Close
            TCP::close 
            }]

             Trigger collection of the request payload
            HTTP::collect $collect_length
            if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Collecting $collect_length" }
      }
    } else {
    reject   
    }
}
when HTTP_REQUEST_DATA {

   if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Collected [HTTP::payload length] bytes." }

    Check if the 'after' ID exists
   if {[info exists id]} {

       If all the POST data has been received, cancel the connection closure
      if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Canceling \$id: $id" }
      after cancel $id
   }
}

The "after cancel" command in this iRule can trigger Bug ID454692, resulting in a connflow / tcl memory leak.

To work around this bug, you must unset the variable referencing the after object.

---snip---

       If all the POST data has been received, cancel the connection closure
      if { $static::post_debug } { log local0. "[IP::client_addr]:[TCP::client_port]: Canceling \$id: $id" }
      after cancel $id
      unset id
   }
}