Forum Discussion
Johnny_Schmidt_
Dec 29, 2011Historic F5 Account
You ought to be able to see if those crawlers are hitting your site at all by user agent string. That's where I'd start looking first for an answer. Create a request-logging profile (for v11) or enable logging in WAM (v10) to start collecting that data.
Cheers,
-dude