Bot Defense causing a lot of false positives

Hi renaud-gaspard​ ,

As you mentioned the TS Cookies which are domain scoped pose the issue when one site loads resource from another domain.  I think the "SameSite" iRule can help this with a modification on where the "Set-Cookie" is placed, if we can adjust to cover all TS cookies on all responses this should help. Bot Defense injects its cookies after HTTP_RESPONSE so changing the event handler should cover this.   I used F5's F5 AI Assistant - iRules   to help validate the iRule code that can help with this scenario.  I have not tested this so please do so in a non-prod environment.  

when HTTP_RESPONSE_RELEASE {
    foreach cookie [HTTP::header values "Set-Cookie"] {
        if { [string match "TS*" $cookie] || [string match "BIGipServer*" $cookie] } {
            if { !([string match "*SameSite=*" $cookie]) } {
                HTTP::header replace "Set-Cookie" $cookie "$cookie; SameSite=None; Secure"
            }
        }
    }
}

Here is what F5's AI Assistant for iRules, analyzed and explained for the iRule:

1. Objective
   • This iRule runs at the HTTP_RESPONSE_RELEASE event to inspect all outgoing Set-Cookie headers.
   • It targets cookies whose names start with “TS” or “BIGipServer” and, if they lack any SameSite attribute, appends “; SameSite=None; Secure” to enforce cross-site usage and require secure transport.
2. Execution Flow

• Event HTTP_RESPONSE_RELEASE fires just before the BIG-IP releases the HTTP response to the client (see docs).
• The code retrieves every Set-Cookie header value via
  foreach cookie [HTTP::header values "Set-Cookie"]
• It checks if the cookie string begins with “TS” or “BIGipServer” ([string match "TS*" $cookie] || [string match "BIGipServer*" $cookie]).
• For each matching cookie, it tests absence of a SameSite attribute (!([string match "*SameSite=*" $cookie])).
• If no SameSite is found, it replaces that specific Set-Cookie header with a version that appends “; SameSite=None; Secure” using
  HTTP::header replace "Set-Cookie" $cookie "$cookie; SameSite=None; Secure".

1. Potential Logic Concerns
   • If a matching cookie already has a SameSite attribute but lacks the Secure flag, this rule will skip it entirely and not add Secure. (high confidence)

Summary
The iRule correctly identifies and modifies specified persistence cookies to include SameSite=None and Secure in most cases, but it will not add Secure to cookies that already declare a SameSite attribute.

Under Cross-Domain-Request config -make sure this is set:  Allow configured domains; validate upon redirect.  

All FQDN's that server cross-domain should be in the site domain list.  For example where you see <script src> and other reference tags that contain domains.  

Hello,

Thank you for your response.

We already had a similar iRule :

when HTTP_RESPONSE_RELEASE {
    foreach cookie_name [HTTP::cookie names] {
        if {$cookie_name starts_with "TS"} {
            HTTP::cookie attribute $cookie_name remove SameSite
            HTTP::cookie attribute $cookie_name insert SameSite None
            HTTP::cookie secure $cookie_name enable
        }
    }
}

While it helped reduce the occurence of the anomalies, we still see a lot of "Resource request without browser verification cookie".

We also noticed the anomaly "High Number of HTML transactions since JavaScript verification". But this one happened only on one single website.

This happens even after adding the full list of all the domain names liked to cross domain requests.

I’ve run into something similar; Bot Defense can get pretty noisy if it hasn’t “learned” your traffic patterns properly yet. A lot of the false positives tend to come from legit users who don’t behave like a typical browser (mobile apps, certain extensions, corporate proxies, etc.). Instead of loosening everything at once, it helps to review the triggered violations and identify patterns; like specific URLs, user agents, or geos, and then apply targeted relaxations there. You can also temporarily switch some actions to log/alarm mode to observe behavior before enforcing blocks again. It’s a bit of an iterative process, but once you fine-tune it around real traffic, the noise usually drops off काफी a bit without compromising protection.

Thank you for your answers,

Using the iRule that update the cookie with "SameSite=None; Secure; Partitioned" didn't help solving most of our false positives issues.
However, enabling it slightly changes how browsers handle the third party cookies in Incognito mode. This means the protected websites are now usable in Incognito mode.

We managed to isolate the false positives to specific cases :

• Browser Verification Timed out :
  • Happened on a website that has a malformed HTML page : From what I understand, because of the malformed HTML, the challenge wasn't correctly injected and was never solved by the browser, causing this anomaly.
  • Also happened after a specific series of action : A new browser tab was opened by mistake after clicking on a link. Because it was opened by mistake, the tab was immediately closed (before the injected challenge had the time to be solved). Later browsing to the site was blocked.
    One possible workaround would be to show a CAPTCHA challenge instead of blocking the user entirely.
• Resource request without browser verification cookie
  • This happens more frequently, we probably can just disable that specific check, or add the list of static file to the whitelist as Jeff_Granieri​ suggested.
• Session Opening
  
  • This was reported by someone else and wasn't reproduced during our testing.
    If it happens again, we probably can slightly raise the detection threshold.
• High number of HTML transactions since verification challenge:
  • This only happens on a specific website/domain.
    This website uses various iframes embedding the website into itself, and also has an API endpoint where the returned Content-Type header is "text/html" even if the returned data isn't HTML.
    For this specific site we could create a microservice, with challenge-free browser verification.
• Device ID Deletion
  • We raised the detection value slightly and it didn't cause any issue since.

As you said, carlbidwell268​ , we'll need to tweak this over time, but at least we know have a mostly working profile.

Thank you both for your help