Script to collect urls

Also use Trusted IP and trigger the crawler from the computer with the trusted IP, so that the F5 learns things fast. If the site has authentication you may need a vunravility scanner like qualys that supports authenticaion with credentials or ssl cert.

https://techdocs.f5.com/kb/en-us/products/big-ip_asm/manuals/product/asm-implementations-13-0-0/9.html

https://qualysguard.qg2.apps.qualys.com/portal-help/en/was/scans/web_crawling_and_link_discovery.htm