Forum Discussion
adalan_1411
Nimbostratus
Aug 15, 2008Bot Redirections
Here's the iRule:
class bots {
"archiver"
"argus"
"arks"
"bot"
"core"
"crawl"
"cursor"
"custo"
"exalaed"
"fast-"
"favorg"
"googlebot"
"jakarta"
"java"
"larbin"
"libwww"
"msnbot"
"muncher"
"nutch"
"pioneer"
"powermarks"
"python"
"redcarpet"
"sbider"
"scanalert"
"scooter"
"shopwiki"
"slurp"
"spider"
"sunrise"
"susie"
"t-h-u-n-d-e-r-s-t-o-n-e"
"teoma"
"traveler"
"trivial"
"walhello"
"webcollage"
"zyborg"
}
when RULE_INIT {
set ::vip_https_ports [list \
443 \
]
set ::vip_http_ports [list \
80 \
]
set ::special_bots [list \
"googlebot" \
]
}
when HTTP_REQUEST {
set ::is_bot 0
if { [matchclass [string tolower [HTTP::header User-Agent]] contains $::special_bots] } {
HTTP::header insert Is_Bot "Yes"
persist none
set ::is_special 1
set ::is_bot 1
} elseif { [matchclass [string tolower [HTTP::header User-Agent]] contains $::bots] } {
HTTP::header insert Is_Bot "Yes"
persist none
set ::is_special 0
set ::is_bot 1
} elseif { [matchclass [HTTP::method] equals $::sec_http_methods] } {
reject
}
if {$::is_bot} {
if { ([matchclass [TCP::local_port] equals $::vip_https_ports]) }{
if {$::is_special}{pool pool_bot_1a_https}
if {!$::is_special}{pool pool_bot_2a_https}
} elseif {([matchclass [TCP::local_port] equals $::vip_http_ports]) }{
if {$::is_special}{pool pool_bot_1a_http}
if {!$::is_special}{pool pool_bot_2a_http}
}
}
}
I am trying move Googlebot into it's own 2 servers and all other bots onto their own 2 servers. It seems though, that bots are still managing to get by this rule. Any ideas?
- deltapez_21676
Nimbostratus
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects