Forum Discussion

lnxgeek's avatar
Mar 07, 2023

Bot Defense Deployment/Troubleshooting

Hi all

I have started to push Botd more and more as a solution for fending off ddos attacks.

However, I often end up with a lot of false positives or blocked or challenged requests which are normal clients.

For now I circumvent it by turning off botd for parts of the application with ltm policies.

Any good ideas or a better and more systematic approach would be very welcome ๐Ÿ™‚

11 Replies

  • I found this db key:

    sys db botdefense.suspicious_jsfree_score {
    value "49"
    }

    I have changed this to 60 as well.

    On 16.x the key dosl7.browser* doesn't exist, don't know if it has been replaced by something else or if it all is placed under botd now.

    • Nice!

       

      By the way  I tested on a VM on 16.1.3.3 with updated bot and browser signatures and again I am never blocked by the active javascript bot protection when using Local Traffic policies to switch between Bot profiles, so the Local Traffic policies seem to break this feature even with SPA enabled as even with bot debug https://my.f5.com/manage/s/article/K10478323 I see that F5 things that javascript verification is not configured.

       

      With Javascript verification under for example the login pages the the Impersonating/suspcious Browser category could be more accurate than just checking the HTTP header order etc.

       

      You can ask F5 TAC about this as this makes pointless to switch bot profiles as just with an irule you can change the blocking setting for a bot category for a particular URL like the web form login page to block "Suspicious Browser" based on the Challenge-Free Verification.

    • lnxgeek's avatar
      lnxgeek
      Icon for MVP rankMVP

      Hi Juergen

      Here are a couple of examples.

      This one is caught as a suspcious browser:

       

       

      POST /identity/connect/token HTTP/1.1
      Host: xxx.domingo.dk
      Connection: keep-alive
      Content-Length: 121
      sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106"
      device-type: 8
      Bitwarden-Client-Version: 2023.2.0
      sec-ch-ua-mobile: ?0
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36
      content-type: application/x-www-form-urlencoded; charset=utf-8
      accept: application/json
      Bitwarden-Client-Name: desktop
      sec-ch-ua-platform: "Linux"
      Sec-Fetch-Site: cross-site
      Sec-Fetch-Mode: cors
      Sec-Fetch-Dest: empty
      Accept-Encoding: gzip, deflate, br
      Accept-Language: en-US
      

       

       

       This is the same:

       

       

      GET /api/accounts/revision-date HTTP/1.1
      Host: xxx.domingo.dk
      Connection: keep-alive
      sec-ch-ua: "Not;A=Brand";v="99", "Chromium";v="106"
      Pragma: no-cache
      device-type: 8
      Bitwarden-Client-Version: 2023.2.0
      sec-ch-ua-mobile: ?0
      authorization: Bearer xxxx
      User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.5249.181 Safari/537.36
      accept: application/json
      Cache-Control: no-store
      Bitwarden-Client-Name: desktop
      sec-ch-ua-platform: "Linux"
      Sec-Fetch-Site: cross-sit

       

       

       

      This one is called an Android browser but is being challenged:

       

       

      POST /api/ciphers HTTP/1.1
      Content-Type: application/json; charset=utf-8
      Authorization: Bearer xxxxx
      Accept: application/json
      Device-Type: 0
      Bitwarden-Client-Name: mobile
      Bitwarden-Client-Version: 2023.2.0
      User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-G998B)
      Accept-Encoding: identity
      Cookie: TS9f79eae6029=0858c33216ab280069fed2335f832686f40bee553f364277a813f24ec7c741052c6dfd58c2e7e1949548e17d06aec041; TS7f78903d027=0858c33216ab20007d1ce8a65ebef8e54dd2bcec14433de6d8981b0e52e56dce619fda5b3bcb086
      POST /api/ciphers HTTP/1.1
      Content-Type: application/json; charset=utf-8
      Authorization: Bearer xxxx
      Accept: application/json
      Device-Type: 0
      Bitwarden-Client-Name: mobile
      Bitwarden-Client-Version: 2023.2.0
      User-Agent: Bitwarden_Mobile/2023.2.0 (Android 13; SDK 33; Model SM-F721B)
      Accept-Encoding: identity
      Cookie: TS9f79eae6029=0858c33216ab2800f9a3ea0a88ee8d54455848012a8855571e4a03813af486c0dfc70243c46674171fc6c489f05e3781; TS9f79eae6078=0858c33216ab20007dd688a601902ab986ede4a8603a3cd1541458a19896ced1f2a3d26c8b431ea

       

       

      All requests are coming from the Bitwarden app on either a desktop pc or an Android App.

      • Have you tried enabling "api access for browsers and mobile applications" https://my.f5.com/manage/s/article/K42323285 ? Single Page Protection (SPA) needs to enabled for this as your Application could be also needed this if it is AJAX.

        Also do the POST requests have a body as maybe the F5 bot signatures don't like the lack of a Body in a POST request?

        My final thought is again if your app is SPA maybe the Javarscript generates some HTTP requests that have incorect HTTP header order or something like that and F5 Bot signatures don't like this.

         

        Also as you say " Android browser" could be a mobile application as this case F5 Bot SDK feature needs to be licenced ?

  • After some tests switching the Bot protection profiles with Local Traffic policies (for the web form login url the bot profile uses the javascript based browser verification ) it seems the feature browser verification does not work even when from the local traffic policy and the logs the URL is matching the Bot protection profile that has browser verification enabled. I tested switching with verify before and after access but it does not work on 15.1.8.1  ๐Ÿ˜€

     

    Do you also see the same issue lnxgeek  ?

    • lnxgeek's avatar
      lnxgeek
      Icon for MVP rankMVP

      Nikoolayy1 I have bellow LTM policy on my VS:

      for bw_botd I have these settings to accomodate the app:

      and this to go more aggressively on the browser:

      Btw I think enabling SPA does make it work more smoothly - thanks ๐Ÿ™‚

      I'm using 16.1.3.3.

       

      • And you have tested that something like curl, browser where cookies are  or apache benchmark are blocked when accessing urls that are protected with bot profile that has "Verify After Access" and it is the javascript checks not the signatures that block them?

         

        This can be seen in the traffic logs for bot or just opening browser and blocking it to accept cookies and you will see if you are truly blocked and you will also see the javascript message.