Forum Discussion
adalan_1411
Nimbostratus
Aug 15, 2008Bot Redirections
Here's the iRule:
class bots {
"archiver"
"argus"
"arks"
"bot"
"core"
"crawl"
"cursor"
"custo"
"exalaed"
"fast-"
"favorg"
"googlebot"
"jakarta"
"java"
"larbin"
"libwww"
"msnbot"
"muncher"
"nutch"
"pioneer"
"powermarks"
"python"
"redcarpet"
"sbider"
"scanalert"
"scooter"
"shopwiki"
"slurp"
"spider"
"sunrise"
"susie"
"t-h-u-n-d-e-r-s-t-o-n-e"
"teoma"
"traveler"
"trivial"
"walhello"
"webcollage"
"zyborg"
}
when RULE_INIT {
set ::vip_https_ports [list \
443 \
]
set ::vip_http_ports [list \
80 \
]
set ::special_bots [list \
"googlebot" \
]
}
when HTTP_REQUEST {
set ::is_bot 0
if { [matchclass [string tolower [HTTP::header User-Agent]] contains $::special_bots] } {
HTTP::header insert Is_Bot "Yes"
persist none
set ::is_special 1
set ::is_bot 1
} elseif { [matchclass [string tolower [HTTP::header User-Agent]] contains $::bots] } {
HTTP::header insert Is_Bot "Yes"
persist none
set ::is_special 0
set ::is_bot 1
} elseif { [matchclass [HTTP::method] equals $::sec_http_methods] } {
reject
}
if {$::is_bot} {
if { ([matchclass [TCP::local_port] equals $::vip_https_ports]) }{
if {$::is_special}{pool pool_bot_1a_https}
if {!$::is_special}{pool pool_bot_2a_https}
} elseif {([matchclass [TCP::local_port] equals $::vip_http_ports]) }{
if {$::is_special}{pool pool_bot_1a_http}
if {!$::is_special}{pool pool_bot_2a_http}
}
}
}
I am trying move Googlebot into it's own 2 servers and all other bots onto their own 2 servers. It seems though, that bots are still managing to get by this rule. Any ideas?
1 Reply
- deltapez_21676
Nimbostratus
Do you need the "googlebot" entry in both lists? Seems like it would be easier to code without the duplicate entry.
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects
