Forum Discussion
Bot Defense causing a lot of false positives
Hello DevCentral Community,
While configuring a Bot Defense profile for our websites, we noticed a lot of false positives, where legitimate browsers are flagged as Malicious Bots to a point where we cannot safely enable Malicious Bot blocking.
The detected anomalies are mostly :
- Device ID Deletion (can be worked around by raising the threshold from 3 to ~10)
- Resource request without browser verification cookie
- Session Opening
- Browser Verification Timed out (more rarely)
We have tried various configuration, none of which worked properly.
Currently, our test bot defense profile is as follows :
- DoS Attack Mitigation Mode : Enabled
- API Access for Browsers and Mobile Applications : Enabled
- Exceptions:
- Device ID Deletions : Block for 600s Detect after 10 (instead of 3) access attemps in 600s
- No microservice
- Browser Access : Allow
- Browser Verification : Verify After Access (Blocking) / 300s grace perdiod (we also tried verify before, but the white challenge page isn't acceptable for our users)
- Device ID mode : Generate After Access (we also tried Generate Before access)
- Single page application : Enabled (we also tried to disable it)
- Cross Domain Requests : Allow configured domains; validate upon request (with all of our websites added in related site domains)
We also tried with allow all requests
After a bit of digging around, we noticed the following :
- The false positives often happen after visiting a website that loads various resources from other domains, and we believe the issue might be linked to cross domain requests
- Google Chrome (and derivatives) are dropping the TS* cookies for cross domain requests, even with the domains added in the related domain list
- After creating an iRule that updates TS* cookies with SameSite=None; Secure, some previously blocked requests were now allowed but not all
Disabling the check for the detected anomalies feel like it would severely affect the bot defense effectiveness.
We have opened a support ticket related to this is issue over a year ago and haven't found any solution yet.
Has anyone faced a similar problem before, and has managed to solve it ?
If so, how ?
Thank you for any help.
Regards
1 Reply
- Jeff_Granieri
Employee
Hi renaud-gaspard ,
As you mentioned the TS Cookies which are domain scoped pose the issue when one site loads resource from another domain. I think the "SameSite" iRule can help this with a modification on where the "Set-Cookie" is placed, if we can adjust to cover all TS cookies on all responses this should help. Bot Defense injects its cookies after HTTP_RESPONSE so changing the event handler should cover this. I used F5's F5 AI Assistant - iRules to help validate the iRule code that can help with this scenario. I have not tested this so please do so in a non-prod environment.
when HTTP_RESPONSE_RELEASE { foreach cookie [HTTP::header values "Set-Cookie"] { if { [string match "TS*" $cookie] || [string match "BIGipServer*" $cookie] } { if { !([string match "*SameSite=*" $cookie]) } { HTTP::header replace "Set-Cookie" $cookie "$cookie; SameSite=None; Secure" } } } }Here is what F5's AI Assistant for iRules, analyzed and explained for the iRule:
- Objective
• This iRule runs at the HTTP_RESPONSE_RELEASE event to inspect all outgoing Set-Cookie headers.
• It targets cookies whose names start with “TS” or “BIGipServer” and, if they lack any SameSite attribute, appends “; SameSite=None; Secure” to enforce cross-site usage and require secure transport. - Execution Flow
- Event HTTP_RESPONSE_RELEASE fires just before the BIG-IP releases the HTTP response to the client (see docs).
- The code retrieves every Set-Cookie header value via
foreach cookie [HTTP::header values "Set-Cookie"] - It checks if the cookie string begins with “TS” or “BIGipServer” ([string match "TS*" $cookie] || [string match "BIGipServer*" $cookie]).
- For each matching cookie, it tests absence of a SameSite attribute (!([string match "*SameSite=*" $cookie])).
- If no SameSite is found, it replaces that specific Set-Cookie header with a version that appends “; SameSite=None; Secure” using
HTTP::header replace "Set-Cookie" $cookie "$cookie; SameSite=None; Secure".
- Potential Logic Concerns
• If a matching cookie already has a SameSite attribute but lacks the Secure flag, this rule will skip it entirely and not add Secure. (high confidence)
Summary
The iRule correctly identifies and modifies specified persistence cookies to include SameSite=None and Secure in most cases, but it will not add Secure to cookies that already declare a SameSite attribute.Under Cross-Domain-Request config -make sure this is set: Allow configured domains; validate upon redirect.
All FQDN's that server cross-domain should be in the site domain list. For example where you see <script src> and other reference tags that contain domains.
- Objective
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com