For more information regarding the security incident at F5, the actions we are taking to address it, and our ongoing efforts to protect our customers, click here.

Forum Discussion

Burak_ISIKSOY_1's avatar
Burak_ISIKSOY_1
Icon for Nimbostratus rankNimbostratus
Jul 18, 2014

uri query limiting

hi,

I want to limit the queries to certain uri unfortunately I couldnt manage so far. Can someone point me what I am missing.Thanks

----------data group--------

query_limiter_uri; Search.aspx

query_limiter_whitelist; doesnt have anything defined

-------------irule-------------

when RULE_INIT { set static::maxrate 2 set static::timespan 60 set static::respond_page {

Too many requests. Try again later
            }

} when HTTP_REQUEST { if { ([HTTP::method] eq "POST") and ([class match [string tolower [HTTP::uri]] contains query_limiter_uri]) } {

log local0. "method = [HTTP::method], uri = [HTTP::uri]"
    if { [class match [IP::client_addr] eq query_limiter_whitelist] }{
       return
    }
    set cl_q [string tolower [HTTP::uri]]
    set clid [IP::client_addr]:[TCP::client_port]:[IP::remote_addr]:[TCP::remote_port]
    set clid_q $clid:$cl_q
    set get_count [table key -count -subtable $clid_q]
    if { $get_count < $static::maxrate } {
        incr get_count 1
        table set -subtable $clid_q $get_count $clid_q indefinite $static::timespan
    } else {
        log local0. "-- $clid -- violation detected -- $cl_q -- exceeded number of requests --"
                                           HTTP::redirect "http://google.com"
                                           TCP::close
HTTP::respond 200 content $static::respond_page HTTP::close HTTP::respond 301 Location "http://google.com" return
    }
}

}

5 Replies

  • Try this:

    when RULE_INIT { 
        set static::maxrate 2 
        set static::timespan 60 
        set static::respond_page { Too many requests. Try again later }
    } 
    when HTTP_REQUEST { 
        if { ( [HTTP::method] eq "POST" ) and ( [class match [string tolower [HTTP::uri]] contains query_limiter_uri] ) } {
            if { [class match [IP::client_addr] eq query_limiter_whitelist] } {
                return
            }
            set cl_q [string tolower [HTTP::uri]]
            set clid [IP::client_addr]:[IP::remote_addr]:[TCP::local_port]
            set clid_q "${clid}:${cl_q}"
    
            if { [table lookup -subtable REQCOUNT $clid_q] eq "" } {
                table set -subtable REQCOUNT $clid_q 1 $static::timespan
            } elseif { [table lookup -subtable REQCOUNT $clid_q] > $static::maxrate } {
                log local0. "-- $clid -- violation detected -- $cl_q -- exceeded number of requests --"
                HTTP::respond 200 content $static::respond_page
            } else {
                table incr -subtable REQCOUNT $clid_q
            }
        }
    }
    

    There were a few things that might have given you a problem in the first version of the iRule:

    1. You're table entries include the client side port, which would likely change often enough to bypass your catch.

    2. You're attempting to respond and redirect in the same condition.

    3. Your class match is doing a [string tolower ] evaluation, but you have a string in your data group with an uppercase letter.

    The above basically says:

    1. If the table entry doesn't exist, create it.
    2. If it exists and is greater than maxrate, respond with static content.
    3. Else increment the table entry.
    • Burak_ISIKSOY_1's avatar
      Burak_ISIKSOY_1
      Icon for Nimbostratus rankNimbostratus
      Hi Kevin, I have configured the rule but no luck again. Still I can query as many times as I want
  • Try with some additional logging:

    when RULE_INIT { 
        set static::maxrate 2 
        set static::timespan 60 
        set static::respond_page { Too many requests. Try again later }
    } 
    when HTTP_REQUEST { 
        if { ( [HTTP::method] eq "POST" ) and ( [class match [string tolower [HTTP::uri]] contains query_limiter_uri] ) } {
            log local0. "POST to [HTTP::uri] and matches data group"
            if { [class match [IP::client_addr] eq query_limiter_whitelist] } {
                log local0. "Matches whitelist"
                return
            }
            set cl_q [string tolower [HTTP::uri]]
            set clid [IP::client_addr]:[IP::remote_addr]:[TCP::local_port]
            set clid_q "${clid}:${cl_q}"
    
            log local0. "clid_q = $clid_q"
    
            if { [table lookup -subtable REQCOUNT $clid_q] eq "" } {
                log local0. "new entry"
                table set -subtable REQCOUNT $clid_q 1 $static::timespan
            } elseif { [table lookup -subtable REQCOUNT $clid_q] > $static::maxrate } {
                log local0. "-- $clid -- violation detected -- $cl_q -- exceeded number of requests --"
                HTTP::respond 200 content $static::respond_page "Connection" "close"
            } else {
                log local0. "incrementing table entry"
                table incr -subtable REQCOUNT $clid_q
            }
        }
    }
    
  • So instead of trying to figure whats wrong I am trying an other code i have found. Issue this time is when try to respond with a message stating that too many request webpage complaining me that "An HTTP Content-Type header is required for SOAP messaging and none was found."

     

    Well since I suck at coding I would be glad if someone can show me what I am doing wrong

     

    when RULE_INIT {
       set static::maxRate 5
       set static::windowSecs 20
       set static::timeout 30
    }
    
    when HTTP_REQUEST {
    
        This I-Rule limits "POST" requests, if you want to limit GETs instead, replace
        "POST" with "GET" in if statement below.
        If you want to limit all types or requests, remove this "if" statement below as well as its 
        Corresponding curly bracket '\}' on or around line 67 clearly maked with a comment.
    
       if { [HTTP::method] eq "POST" } {
    
          set myUserID "user"
               Uncomment the block below if you want to throttle requests for each user individually.
               Also remove the line directly above this comment which sets the value of "myUserID"
          
                     if {[HTTP::header exists FooUser]} {
                       set myUserID [HTTP::header FooUser]
                     } else {
                       HTTP::respond 401
                       return
                     }
    
          set currentTime [clock seconds]
          set windowStart [expr {$currentTime - $static::windowSecs}]
          set postCount 0
    
           PH stands for posthistory, a term from the original irule.
    
          log -noname local0. "Table Keys [table keys -subtable 'PH:${myUserID}.${AES_key}']"
    
          foreach { requestTime  } [table keys -subtable "PH:${myUserID}"] {
             count POSTs with start time > $windowStart, delete the rest
             if { $requestTime > $windowStart } {
                incr postCount 1
             } else {
                table delete -subtable "PH:${myUserID}" $requestTime
             }
          }
    
          if { $postCount < $static::maxRate } {
             add new record to array w/myUserID.rand + currentTime
             set requestID "PH:${myUserID}"
             table set -subtable $requestID $currentTime "ignored" $static::timeout 
             } else {
                log -noname local0. "POST Rejected: current postCount for ${myUserID}: $postCount"
       HTTP::respond 200 content {
          
             
                Too many requests
             
             
                Too many requests, please try again later.
             
          
       }
             }
    
           Remove the curly bracket "\}" on the line below if you removed the 'if' statement above.
       }    
    }