Bot Defense Deployment/Troubleshooting

[lnxgeek]

Hi all

I have started to push Botd more and more as a solution for fending off ddos attacks.

However, I often end up with a lot of false positives or blocked or challenged requests which are normal clients.

For now I circumvent it by turning off botd for parts of the application with ltm policies.

Any good ideas or a better and more systematic approach would be very welcome 🙂

[Juergen_Mang]

How do the false positive requests look like?

[lnxgeek]

Hi Juergen

Here are a couple of examples.

This one is caught as a suspcious browser:

POST /identity/connect/token HTTP/1.1
Host: xxx.domingo.dk
Connection: keep-alive
Content-Length: 121
sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106"
device-type: 8
Bitwarden-Client-Version: 2023.2.0
sec-ch-ua-mobile: ?0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36
content-type: application/x-www-form-urlencoded; charset=utf-8
accept: application/json
Bitwarden-Client-Name: desktop
sec-ch-ua-platform: "Linux"
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US

 This is the same:

GET /api/accounts/revision-date HTTP/1.1
Host: xxx.domingo.dk
Connection: keep-alive
sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106"
Pragma: no-cache
device-type: 8
Bitwarden-Client-Version: 2023.2.0
sec-ch-ua-mobile: ?0
authorization: Bearer xxxx
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36
accept: application/json
Cache-Control: no-store
Bitwarden-Client-Name: desktop
sec-ch-ua-platform: "Linux"
Sec-Fetch-Site: cross-sit

This one is called an Android browser but is being challenged:

POST /api/ciphers HTTP/1.1
Content-Type: application/json; charset=utf-8
Authorization: Bearer xxxxx
Accept: application/json
Device-Type: 0
Bitwarden-Client-Name: mobile
Bitwarden-Client-Version: 2023.2.0
User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-G998B)
Accept-Encoding: identity
Cookie: TS9f79eae6029=0858c33216ab280069fed2335f832686f40bee553f364277a813f24ec7c741052c6dfd58c2e7e1949548e17d06aec041; TS7f78903d027=0858c33216ab20007d1ce8a65ebef8e54dd2bcec14433de6d8981b0e52e56dce619fda5b3bcb086

POST /api/ciphers HTTP/1.1
Content-Type: application/json; charset=utf-8
Authorization: Bearer xxxx
Accept: application/json
Device-Type: 0
Bitwarden-Client-Name: mobile
Bitwarden-Client-Version: 2023.2.0
User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-F721B)
Accept-Encoding: identity
Cookie: TS9f79eae6029=0858c33216ab2800f9a3ea0a88ee8d54455848012a8855571e4a03813af486c0dfc70243c46674171fc6c489f05e3781; TS9f79eae6078=0858c33216ab20007dd688a601902ab986ede4a8603a3cd1541458a19896ced1f2a3d26c8b431ea

All requests are coming from the Bitwarden app on either a desktop pc or an Android App.

[Nikoolayy1]

Have you tried enabling "api access for browsers and mobile applications" https://my.f5.com/manage/s/article/K42323285 ? Single Page Protection (SPA) needs to enabled for this as your Application could be also needed this if it is AJAX.

Also do the POST requests have a body as maybe the F5 bot signatures don't like the lack of a Body in a POST request?

My final thought is again if your app is SPA maybe the Javarscript generates some HTTP requests that have incorect HTTP header order or something like that and F5 Bot signatures don't like this.

Also as you say " Android browser" could be a mobile application as this case F5 Bot SDK feature needs to be licenced ?

[Leslie_Hubertus]

Following up to make sure @lnxgeek saw Juergen's question. 🙂

[Nikoolayy1]

After some tests switching the Bot protection profiles with Local Traffic policies (for the web form login url the bot profile uses the javascript based browser verification ) it seems the feature browser verification does not work even when from the local traffic policy and the logs the URL is matching the Bot protection profile that has browser verification enabled. I tested switching with verify before and after access but it does not work on 15.1.8.1  😀

Do you also see the same issue @lnxgeek  ?

[lnxgeek]

@Nikoolayy1 I have bellow LTM policy on my VS:

for bw_botd I have these settings to accomodate the app:

and this to go more aggressively on the browser:

Btw I think enabling SPA does make it work more smoothly - thanks 🙂

I'm using 16.1.3.3.

[Nikoolayy1]

And you have tested that something like curl, browser where cookies are  or apache benchmark are blocked when accessing urls that are protected with bot profile that has "Verify After Access" and it is the javascript checks not the signatures that block them?

This can be seen in the traffic logs for bot or just opening browser and blocking it to accept cookies and you will see if you are truly blocked and you will also see the javascript message.

[Nikoolayy1]

I am asking as I think on 16.1.3.2 I think that had seen the same limitation and this means that the identification and fingerprinting are little bit more limited and it could be related also to your issue as I think the fingerprinting is less accurate.

https://cdn.f5.com/product/bugtracker/ID703129.html

You can also update the Bot signatures just in case as many such issues are solved with signature upgrade.

Also the workarounds like modify sys db botdefense.suspicious_js_score value 60 or dosl7.browser_legit_min_score can be done till F5 maybe fixes the signatures.

https://cdn.f5.com/product/bugtracker/ID1029373.html

https://cdn.f5.com/product/bugtracker/ID699772.html

[lnxgeek]

Definitely some settings I will dig into.

Thanks yet again.

[lnxgeek]

I found this db key:

sys db botdefense.suspicious_jsfree_score {
value "49"
}

I have changed this to 60 as well.

On 16.x the key dosl7.browser* doesn't exist, don't know if it has been replaced by something else or if it all is placed under botd now.

[Nikoolayy1]

Nice!

By the way  I tested on a VM on 16.1.3.3 with updated bot and browser signatures and again I am never blocked by the active javascript bot protection when using Local Traffic policies to switch between Bot profiles, so the Local Traffic policies seem to break this feature even with SPA enabled as even with bot debug https://my.f5.com/manage/s/article/K10478323 I see that F5 things that javascript verification is not configured.

With Javascript verification under for example the login pages the the Impersonating/suspcious Browser category could be more accurate than just checking the HTTP header order etc.

You can ask F5 TAC about this as this makes pointless to switch bot profiles as just with an irule you can change the blocking setting for a bot category for a particular URL like the web form login page to block "Suspicious Browser" based on the Challenge-Free Verification.