Forum Discussion
mtobkes_64700
Nimbostratus
May 12, 2010Rate-Limiting Crawlers
Hi I found this iRule here that will limit requests to 1 request per n seconds. I would like to know how I'd be able to allow n requests per 1 second, e.g. allow 5 requests per 1 second.
wh...
mtobkes_64700
Nimbostratus
May 13, 2010Thanks Aaron. I changed the line however I now get this TCL error in my logs:
TCL error: googlebot_rate-limit_vb5 HTTP_REQUEST - invalid command name 1273758282 while executing $::active_crawlers$user_agent $curr_time
Help guide the future of your DevCentral Community!
What tools do you use to collaborate? (1min - anonymous)Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects