Problem this snippet solves:

This iRule will block ALL further site access to source IP addresses that exceed a certain number of HTTP requests to server resources that results in a 404 not found error. This script is useful for scenarioes where you have a system that contains fixed paths with files and want to keep people from brute-forcing those filenames.

By default, this will apply when a source IP gets 100 errors in a 10 minute period - and will block site access for another 10 minutes. This will slow down a brute force attack significantly and provide you with some log details you can use to troubleshoot.

This rule bases its logic on the incoming source IP, so if you have a large number of users coming through proxies you may want to consider setting a relatively high threshold. If you don't like using this with a source IP, it's possible to rewrite this logic to use a cookie instead, but most brute-force tools don't honor cookies -- source IP is still the most logical choice, even if not the most precise.

Code :

when RULE_INIT {

# excessive_404_blacklist
# v1.0 - Joel Moses
#
# Block all site access to source IP addresses that exceed a
# certain number of HTTP requests to non-existant objects within
# a fixed period of time. Useful for blocking harvesting 
# activities (or at least slowing them down significantly).
#
# Use this iRule at your own risk. If your site has a propensity
# to throw off lots of 404 errors in normal operation, you may
# inadvertently find valid users blocked.

# Set the maximum number of 404 errors for a client IP in a
# certain period of time. The default setings are one hundred
# 404 errors in ten minutes. Following this, the client IP
# address will be blacklisted and an error page displayed for
# the next ten minutes.

        set static::maxerrors 100
set static::maxperiod 600
set static::holdtime 600

# If for any reason you need to rid yourself of a 
# blacklist, you can send the VIP running this iRule
# a query string parameter in the format:
#   http(s)://sitename.com/?DeleteKey=
# where the case-sensitive key is defined below. Consider
# this a "get out of jail free" card in case something 
# goes totally wrong.

set static::deletekey "DeleteAllBlacklistData"

# If set to true, log all blacklisted source IP addresses
# to the LTM log.

set static::log_offender 1

# If set to true, this will create unique tables per VS
# for blacklisting activities.
# This is good if you wish to apply the iRule across multiple
# virtuals but want blocking tables to be created for each
# individual virtual and not shared across multiple ones.

set static::unique_tables_per_virtual 1

# Display this block page for all attempts that hit the blacklist.

set static::blockpage {



403 - Forbidden
Too many requests for invalid objects. Your access is denied.


}

}

when CLIENT_ACCEPTED {

# Collect the remote IP address.

set srcip [IP::remote_addr]

if { ($static::unique_tables_per_virtual ) } {
set blacklist_name "blacklist_[virtual]"
set countlist_name "[IP::remote_addr]_[virtual]"
} else {
set blacklist_name "blacklist"
set countlist_name "[IP::remote_addr]"
} 
}

when HTTP_REQUEST {

# If the request has a DeleteKey parameter that matches
# the setting supplied in RULE_INIT for static::deletekey,
# delete the blacklist subtable (values and keys).

if { [URI::query [HTTP::uri] DeleteKey] equals "$static::deletekey" } {
log local0. "BLACKLIST: Table manually cleared by [IP::remote_addr]."
table delete -subtable $blacklist_name -all
}
}

when HTTP_RESPONSE {

# If the source IP is already in the blacklist table, 
# respond with the block page.

if { [table lookup -subtable $blacklist_name $srcip] != "" } {
HTTP::respond 403 content "[subst $static::blockpage]" 
return

# If the source IP has gotten a 404 but does not currently
# have a unique subtable entry, create a new offender.

} elseif { ([table lookup -subtable $countlist_name "count"] == "") && ([HTTP::status] equals "404") } {
set count 1
table add -subtable $countlist_name "count" $count indef $static::maxperiod

# If the source IP has a 404 and is a repeat offender,
# increment the count within the table.

} elseif { ([table lookup -notouch -subtable $countlist_name "count"] != "") && ([HTTP::status] equals "404") } {
        set count [table incr -notouch -subtable $countlist_name "count"]

# If the maximum number of errors has been reached, 
# add the source IP to the blacklist and get rid of the
# offender's unique table (he'll get a new one after his holdtime 
# is over). Respond with the error page.

        if { $count >= $static::maxerrors } {
table add -subtable $blacklist_name $srcip "blocked" indef $static::holdtime
table delete -subtable $countlist_name -all
HTTP::respond 403 content "[subst $static::blockpage]" 
if { ($static::log_offender) } {
log local0. "BLACKLIST: [IP::remote_addr] is blacklisted on [virtual] for $static::holdtime seconds."
}
return
}
}
}