how to block spam on forums application

using firebug. Here is the source of post. (Sorry am not able to post complete headers due to security reasons)

 

 

bodytest3

 

doPostPost Message

 

forumID779

 

linkText

 

messageID3626500

 

postedFromGUIEditorfalse

 

replytrue

 

subjectRe: test

 

tags

 

threadID791911

 

url

 

Source

 

Content-Type: application/x-www-form-urlencoded Content-Length: 153 forumID=779&threadID=791911&reply=true&messageID=3626500&postedFromGUIEditor=false&subject=Re%3A+test&url=&linkText=&body=test3&tags=&doPost=Post+Message

What version are you running so I know how to "optimize" this guy?

Thanks. We use Big-IP 9.4.6 Build 401.0.

 

Appreciate your reply. I'll do some load test on it and see if there's any impact to performance.

 

 

thanks again.

 

Puli.

 

Posted By Puli on 01/11/2011 10:10 AM

Thanks. We use Big-IP 9.4.6 Build 401.0.

Appreciate your reply. I'll do some load test on it and see if there's any impact to performance.

thanks again.

Puli.

Need to use "matchclass" instead of "class match" then.

when RULE_INIT {

    Default amount of request payload to collect (in bytes)
   set static::collect_length 2048
}

when HTTP_REQUEST {

    Only check POST requests
   if { [HTTP::method] eq "POST" } {

       Check for a non-existent Content-Length header
      if {[HTTP::header Content-Length] eq ""}{

          Use default collect length of 2k for POSTs without a Content-Length header
         set collect_length $static::collect_length

      } elseif {[HTTP::header Content-Length] == 0}{

          Don't try collect a payload if there isn't one
         unset collect_length

      } elseif {[HTTP::header Content-Length] > $static::collect_length}{

          Use default collect length
         set collect_length $static::collect_length

      } else {

          Collect the actual payload length
         set collect_length [HTTP::header Content-Length]

      }

       If the POST Content-Length isn't 0, collect (a portion of) the payload
      if {[info exists collect_length]}{

          Trigger collection of the request payload
         HTTP::collect $collect_length
      }
   }
}

when HTTP_REQUEST_DATA {
   if { [matchclass [HTTP::payload] contains dg_blocked] }{
      HTTP::respond 403 "Blocked"
   }
}

Nice work Chris! One small note: the static namespace didn't exist in 9.x, so you could replace that with a local variable set in HTTP_REQUEST:

when HTTP_REQUEST {

    Only check POST requests
   if { [HTTP::method] eq "POST" } {

       Default amount of request payload to collect (in bytes)
      set collect_length 2048

       Check for a non-existent Content-Length header
      if {[HTTP::header Content-Length] eq ""}{

          Use default collect length of 2k for POSTs without a Content-Length header
         set collect_length $collect_length

      } elseif {[HTTP::header Content-Length] == 0}{

          Don't try collect a payload if there isn't one
         unset collect_length

      } elseif {[HTTP::header Content-Length] > $collect_length}{

          Use default collect length
         set collect_length $collect_length

      } else {

          Collect the actual payload length
         set collect_length [HTTP::header Content-Length]

      }

       If the POST Content-Length isn't 0, collect (a portion of) the payload
      if {[info exists collect_length]}{

          Trigger collection of the request payload
         HTTP::collect $collect_length
      }
   }
}

when HTTP_REQUEST_DATA {
   if { [matchclass [HTTP::payload] contains dg_blocked] }{
      HTTP::respond 403 "Blocked"
   }
}

Aaron

Much appreciated Aaron!

 

 

How did you decide to use 2048 as the amount of payload to collect if the Content-Length header didn't exist?

2048 bytes of payload was what I was suggesting as a start point to protect against the slow HTTP payload delivery attack that was popularized a few weeks ago. I believe TMM will timeout the collection if it only receives a portion of the data. So it seemed like a small enough amount of data to wait for before deciding whether the request was a slow payload attack.

 

 

For this scenario, I'd try recording a few POST requests that have suspect payloads (ie spam) and see how many bytes into the payload they are.

 

 

Aaron

Another question. I've started looking into TCP::collect as well and found an interesting part.

http://devcentral.f5.com/wiki/default.aspx/iRules/TCP__collect.html

It is important to note that, when an explicit length is not specified, the semantics of TCP::collect and TCP::release are different than those of the HTTP::collect and HTTP::release commands. With TCP::collect, the event for processing the data (CLIENT_DATA or SERVER_DATA ) will fire without TCP::release being called, whereas with HTTP::collect, the event (HTTP_REQUEST_DATA or HTTP_RESPONSE_DATA) will not fire without HTTP::release being called.

From HTTP::release, it gets even more confusing:

Releases the data collected via HTTP::collect. Unless a subsequent HTTP::collect command was issued, there is no need to use the HTTP::release command inside of the HTTP_REQUEST_DATA and HTTP_RESPONSE_DATA events, since (in these cases) the data is implicitly released.

It is important to note that these semantics are different than those of the TCP::collect and TCP::release commands. With TCP::collect, the event for processing the data (CLIENT_DATA) will fire without TCP::release being called, whereas with HTTP::collect, the event (HTTP_REQUEST_DATA or HTTP_RESPONSE_DATA) will not fire without HTTP::release being called (at least implicitly).

So, when do I need to release and when don't I?

Thanks very much guys.

 

I just implemented the code in our stage environment and its works great.

 

 

Actions i need to confirm before i move to production.

 

1) Performance impact with 7000 hits per minute. Currently our BigIP CPU usage is around 10-15%.

 

2) As chris mentioned, do i need to call a HTTP release, or is it implicitly called.

 

 

I'll post my performance results to this forums once i complete it.

 

 

appreciate your time.

 

Puli.

 

Chris, I think the only time you need to call HTTP::release is if you've called HTTP::collect with no amount of bytes to collect specified. The only time I've needed to do this is to halt a connection until a sideband request is made to either DNS or an auth server.

 

 

Puli, as suggested above, HTTP::release shouldn't be necessary if you've specified the collection amount in HTTP::collect. Please do report back with what you find on resource utilization with the iRule. If you can also confirm the version of code and platform you're using it would help.

 

 

Aaron