Forum Discussion
lnxgeek
MVP
MVPBot Defense Deployment/Troubleshooting
Hi all
I have started to push Botd more and more as a solution for fending off ddos attacks.
However, I often end up with a lot of false positives or blocked or challenged requests which are normal clients.
For now I circumvent it by turning off botd for parts of the application with ltm policies.
Any good ideas or a better and more systematic approach would be very welcome 🙂
Juergen_MangHow do the false positive requests look like?
- lnxgeek
Hi Juergen
Here are a couple of examples.
This one is caught as a suspcious browser:
POST /identity/connect/token HTTP/1.1 Host: xxx.domingo.dk Connection: keep-alive Content-Length: 121 sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106" device-type: 8 Bitwarden-Client-Version: 2023.2.0 sec-ch-ua-mobile: ?0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36 content-type: application/x-www-form-urlencoded; charset=utf-8 accept: application/json Bitwarden-Client-Name: desktop sec-ch-ua-platform: "Linux" Sec-Fetch-Site: cross-site Sec-Fetch-Mode: cors Sec-Fetch-Dest: empty Accept-Encoding: gzip, deflate, br Accept-Language: en-USThis is the same:
GET /api/accounts/revision-date HTTP/1.1 Host: xxx.domingo.dk Connection: keep-alive sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106" Pragma: no-cache device-type: 8 Bitwarden-Client-Version: 2023.2.0 sec-ch-ua-mobile: ?0 authorization: Bearer xxxx User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36 accept: application/json Cache-Control: no-store Bitwarden-Client-Name: desktop sec-ch-ua-platform: "Linux" Sec-Fetch-Site: cross-sitThis one is called an Android browser but is being challenged:
POST /api/ciphers HTTP/1.1 Content-Type: application/json; charset=utf-8 Authorization: Bearer xxxxx Accept: application/json Device-Type: 0 Bitwarden-Client-Name: mobile Bitwarden-Client-Version: 2023.2.0 User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-G998B) Accept-Encoding: identity Cookie: TS9f79eae6029=0858c33216ab280069fed2335f832686f40bee553f364277a813f24ec7c741052c6dfd58c2e7e1949548e17d06aec041; TS7f78903d027=0858c33216ab20007d1ce8a65ebef8e54dd2bcec14433de6d8981b0e52e56dce619fda5b3bcb086POST /api/ciphers HTTP/1.1 Content-Type: application/json; charset=utf-8 Authorization: Bearer xxxx Accept: application/json Device-Type: 0 Bitwarden-Client-Name: mobile Bitwarden-Client-Version: 2023.2.0 User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-F721B) Accept-Encoding: identity Cookie: TS9f79eae6029=0858c33216ab2800f9a3ea0a88ee8d54455848012a8855571e4a03813af486c0dfc70243c46674171fc6c489f05e3781; TS9f79eae6078=0858c33216ab20007dd688a601902ab986ede4a8603a3cd1541458a19896ced1f2a3d26c8b431eaAll requests are coming from the Bitwarden app on either a desktop pc or an Android App.
- Nikoolayy1
Have you tried enabling "api access for browsers and mobile applications" https://my.f5.com/manage/s/article/K42323285 ? Single Page Protection (SPA) needs to enabled for this as your Application could be also needed this if it is AJAX.
Also do the POST requests have a body as maybe the F5 bot signatures don't like the lack of a Body in a POST request?
My final thought is again if your app is SPA maybe the Javarscript generates some HTTP requests that have incorect HTTP header order or something like that and F5 Bot signatures don't like this.
Also as you say " Android browser" could be a mobile application as this case F5 Bot SDK feature needs to be licenced ?
Following up to make sure lnxgeek saw Juergen's question. 🙂
- Nikoolayy1
After some tests switching the Bot protection profiles with Local Traffic policies (for the web form login url the bot profile uses the javascript based browser verification ) it seems the feature browser verification does not work even when from the local traffic policy and the logs the URL is matching the Bot protection profile that has browser verification enabled. I tested switching with verify before and after access but it does not work on 15.1.8.1 😀
Do you also see the same issue lnxgeek ?
- lnxgeek
Nikoolayy1 I have bellow LTM policy on my VS:
for bw_botd I have these settings to accomodate the app:
and this to go more aggressively on the browser:
Btw I think enabling SPA does make it work more smoothly - thanks 🙂
I'm using 16.1.3.3.
- Nikoolayy1
And you have tested that something like curl, browser where cookies are or apache benchmark are blocked when accessing urls that are protected with bot profile that has "Verify After Access" and it is the javascript checks not the signatures that block them?
This can be seen in the traffic logs for bot or just opening browser and blocking it to accept cookies and you will see if you are truly blocked and you will also see the javascript message.
- lnxgeek
I found this db key:
sys db botdefense.suspicious_jsfree_score { value "49" }I have changed this to 60 as well.
On 16.x the key dosl7.browser* doesn't exist, don't know if it has been replaced by something else or if it all is placed under botd now.
- Nikoolayy1
Nice!
By the way I tested on a VM on 16.1.3.3 with updated bot and browser signatures and again I am never blocked by the active javascript bot protection when using Local Traffic policies to switch between Bot profiles, so the Local Traffic policies seem to break this feature even with SPA enabled as even with bot debug https://my.f5.com/manage/s/article/K10478323 I see that F5 things that javascript verification is not configured.
With Javascript verification under for example the login pages the the Impersonating/suspcious Browser category could be more accurate than just checking the HTTP header order etc.
You can ask F5 TAC about this as this makes pointless to switch bot profiles as just with an irule you can change the blocking setting for a bot category for a particular URL like the web form login page to block "Suspicious Browser" based on the Challenge-Free Verification.
