Forum Discussion
academictame
Feb 13, 2023Nimbostratus
How to handle data aggregators hitting my site?
I'm looking for a more intelligent way to handle to prevent data aggregators from crushing my site. The way I do this today is with a policy in the LTM section of the BigIP (it's the only license we ...
Amine_Kadimi
Feb 14, 2023MVP
First thing which is not directly related to the question: while it is possible using a local traffic policy to redirect to another VS internally, you can instead use that same policy to directly select the intended pool, and getting rid of the second VS.
Regarding your question, I see a few ways you can use to rate limit these source IPs:
- Having a second VS identical to your primary VS except for the Source Address field where you specify an address list (shared object) containing the source IPs to limit. Then in the Advanced configuration, use Connection Limit, Connection Rate Limit and/or a Bandwidth Controller (or Acceleration Rate Class) to rate limit by number of opened/new connections, or by allocated bandwidth.
- Use one VS for all the traffic, with a policy that selects a pool (e.g AGG_pool) if traffic is coming from the aggregators' source IPs and with a default rule that points to your default pool. In AGG_pool use the Connection Limit and Connection Rate Limit attributes of the members, which will allow you to control how many open/new connections the F5 can have with the pool member.
I think using a Bandwidth Controller or a Rate Class is less aggressive than specifying connection limits but all depends on the specific needs.
Recent Discussions
Related Content
DevCentral Quicklinks
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com
Discover DevCentral Connects