cancel
Showing results for 
Search instead for 
Did you mean: 
Login & Join the DevCentral Connects Group to watch the Recorded LiveStream (May 12) on Basic iControl Security - show notes included.

Script to collect urls

THE_BLUE
Cirrus
Cirrus

is there any script that might help to visit all links in website so that WAF will learn all the links ?

2 REPLIES 2

Erik_Novak
F5 Employee
F5 Employee

You could create a policy using automatic learning mode and then run this free crawler:

 

https://www.httrack.com/

 

 

Also use Trusted IP and trigger the crawler from the computer with the trusted IP, so that the F5 learns things fast. If the site has authentication you may need a vunravility scanner like qualys that supports authenticaion with credentials or ssl cert.

 

 

https://techdocs.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-implementations-13-0-0/9.html

 

https://qualysguard.qg2.apps.qualys.com/portal-help/en/was/scans/web_crawling_and_link_discovery.htm