Forum Discussion
Bob_10976
Nimbostratus
NimbostratusiRule to Block Google and other Search Engines
Hello all,
We would like to use an iRule to block google and other search engines from crawling our sites and was hopeing someone could point me in the right direction. In the past we woul...
I agree with Aaron that a datagroup is another good way to do this and it would allow you to keep a much larger txt list that you simply upload and apply.
Something like this would work.
when HTTP_REQUEST {
if { class match [string tolower [HTTP::header User-Agent]] contains "bots-list" ] } {
drop
}
else {
return
}
}