Forum Discussion

am_gli_287451's avatar
am_gli_287451
Icon for Nimbostratus rankNimbostratus
Dec 12, 2018

ASM - Problems with curl monitoring / crawler - please enable JS

Hi,

recently I implemented an ASM policy with the Sharepoint template in transparent mode. No DoS profile.

Unfortunately I ran into an issue with - I assume - the browser check.

Our external Nagios-Monitoring does a curl request and looks for a specific keyword in the response to check the health of the external service.

Currently the response doesn't contain a page, but following output:

Please enable JavaScript to view the page content.
Your support ID is:  2863805088290756184.

I can't figure out why this is happening. I can't even find anything with this support ID, neither in App Events, nor in DoS events.

In the learning and blocking settings the "Web scraping detected" signature is deactivated.

If I add the source IP to a whitelist, it works for my test client all the time. But adding the Nagios IP doesn't work (only random).

I'm not sure what more I could check/change here - any ideas?

application delivery
ASM Advanced WAF
security
  • nathe's avatar
    nathe
    Icon for Cirrocumulus rankCirrocumulus
    Dec 12, 2018

    What TMOS version in use? Also, have you enabled BOT signatures and Proactive Bot Defence?

     

  • am_gli's avatar
    am_gli
    Icon for Altostratus rankAltostratus
    Dec 13, 2018

    13.1.1

     

    As far as I know, no. I hope we mean the same:

     

    In Security - DoS Protection - DoS Profiles, there is only the default profile "dos", which is disabled:

     

     

  • nathe's avatar
    nathe
    Icon for Cirrocumulus rankCirrocumulus
    Dec 13, 2018

    OK, so not a DOS profile issue. Suggest confirming all violations with a Block have Learn/Alarm flags set to see if the blocked events appear in the event logs. Also confirm what IP address Nagios uses to ensure the right one is whitelisted.

     

  • am_gli's avatar
    am_gli
    Icon for Altostratus rankAltostratus
    Dec 14, 2018

    Hi, it was an issue with the session awareness / Device ID. This feature also uses JS. After deactivating, it works now properly.

     

    Thanks :)

     

  • boneyard's avatar
    boneyard
    Icon for MVP rankMVP
    Dec 15, 2018

    thanks for posting the answer, still i would look into the whitelisting issue. if you ever want to use that functionality you need a solution.

     

  • samstep's avatar
    samstep
    Icon for Cirrocumulus rankCirrocumulus
    Jan 12, 2019

    ASM is treating requests curl/Nagios as a bot hence blocking it (obviously). Deactivating bot/deviceID protection completely only makes sense if this URL is not meant to be accessed publicly, otherwise you will be allowing bots to attack this content.

     

    Best way is to approach this without breaking security of your website is to whitelist the "good bot".