Forum Discussion

sfracass_83515's avatar
sfracass_83515
Icon for Nimbostratus rankNimbostratus
Apr 11, 2008

iRule to block networks works, sometimes.....

I just want to block googlebot requests to certain virtual servers at the LTM but this rule doesn't seem to work. Oddly enough, when I change the IP values to block my own internal network subnets at the same virtual servers, it works. I had to block at the router where I could see in the logs the rejection of the IP specified, so I know the IP is coming into my network as expected (not NAT'd).

 

 

when CLIENT_ACCEPTED {

 

if { [IP::addr [IP::remote_addr] equals 60.249.0.0/255.255.0.0] } {

 

reject

 

}

 

}

 

 

Any help is appreciated.
application delivery
dev
devops
iRules
  • Joe_Pruitt's avatar
    Joe_Pruitt
    Apr 11, 2008
    Did you log the value of IP::remote_addr to verify it's correct when googlebot is trying to access it?

    Wouldn't it be simpler to use a user-agent test?

    when HTTP_REQUEST {
  if { [string tolower [HTTP::header "User-Agent"]] contains "googlebot" } {
    reject
  }
}

    This would block googlebot from any source network and is very easy to test with the user-agent switcher in FireFox.

    -Joe
  • sfracass_83515's avatar
    sfracass_83515
    Icon for Nimbostratus rankNimbostratus
    Apr 15, 2008
    BUT, Googlebot is a legitimate useragent for organizations with google appliances. We needed to specifically block Google's googlebot.