Forum Discussion

THE_BLUE's avatar
THE_BLUE
Icon for Cirrostratus rankCirrostratus
Apr 19, 2021

Script to collect urls

is there any script that might help to visit all links in website so that WAF will learn all the links ?

2 Replies

  • Also use Trusted IP and trigger the crawler from the computer with the trusted IP, so that the F5 learns things fast. If the site has authentication you may need a vunravility scanner like qualys that supports authenticaion with credentials or ssl cert.

     

     

    https://techdocs.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-implementations-13-0-0/9.html

     

    https://qualysguard.qg2.apps.qualys.com/portal-help/en/was/scans/web_crawling_and_link_discovery.htm