Forum Discussion

Mohamed_Ahmed_Kansoh's avatar
Mohamed_Ahmed_Kansoh
Icon for MVP rankMVP
Nov 09, 2022

Web Scraping Feature

I am wondering if "Web Scraping" Feature still available to configure or not.  - I know it was available under ( Security >>> Application security >>> Anomaly Detection >>> web scraping).  I am tak...
security
  • Daniel_Wolf's avatar
    Daniel_Wolf
    Nov 10, 2022

    True, the available configuration options have changed. And what was Web Scraping till 14.0 is now Bot Defense, where Web Scraping exists as a sub-feature called Web Scraping to prevent any attempts to collect application content.

    It is also stated in K42323285: "The Bot Defense profile replaces and includes  the original functionality of the DoS profile bot signatures and proactive bot defense, as well as Web Scraping."

Mohamed_Ahmed_Kansoh's avatar
Mohamed_Ahmed_Kansoh
Icon for MVP rankMVP
Nov 09, 2022

Hi Daniel_Wolf , 

Thanks for your fast response , but I think it is not configured such as old version any more , I mean that all of below intevals do not exist in V14.1 : 
and it has been reduced in V14.1 with Bot Defense with simple configuration through " Microservice " Feature when creating a Bot Profile. 


  • Daniel_Wolf's avatar
    Daniel_Wolf
    Icon for MVP rankMVP
    Nov 10, 2022

    True, the available configuration options have changed. And what was Web Scraping till 14.0 is now Bot Defense, where Web Scraping exists as a sub-feature called Web Scraping to prevent any attempts to collect application content.

    It is also stated in K42323285: "The Bot Defense profile replaces and includes  the original functionality of the DoS profile bot signatures and proactive bot defense, as well as Web Scraping."

    • Nikoolayy1's avatar
      Nikoolayy1
      Icon for MVP rankMVP
      Nov 10, 2022

      Yup as now for example " Anomalies category" in the Bot profile now includes stuff like "Session Opening" but I wish they kept the latency options atleast as system variables Latency increased/Latency reached/Minimum Latency Threshold for detection for the stress based DDOS detection as now also after 13.1 it is automatic and in some environments I see the use cases but oh well 😔