Learn how to use the F5 Advanced Web Application Firewall to easily protect your applications against bots. Bots can be used as tools for a variety of attacks such as DoS, credential stuffing and brute force, or web scraping.
See the entire AppSec Made Easy series.
I've been playing with Proactive Bot Defense and want to know how to deal with a particular situation. I am needing to allow certain bots that has a valid signature that is categorized as a crawler. Is there a way to get specific bots to work without configuring IP exceptions?
I think you have two good options here:
iRule - https://devcentral.f5.com/s/articles/proactive-bot-defense-bypass-by-bot-signature-1150
If you set the category to report and the signature has a reverse DNS lookup, then it will bypass PBD (note that you need to setup a DNS resolver). If the current one doesn't have reverse DNS on the signature, you could disable the signature, create a custom signature that does have reverse DNS, create a custom category to put it in, and set that custom category to report.
Great job Graham!