For the General order of the modules in F5:
Packet Filter > AFM > iRule Flow Init event> LTM(or GTM/DNS) >APM > ASM .
Also in the AFM there is DDOS at Layer 3 or 4 that is before the AFM rules (the same as the ASM). For the AFM DDOS there is general device DDOS and virtual server specific DDOS and the Genaral Device DDOS takes precedence but it has higher by default thresholds and this why during attack the Virtual server DDOS will in most cases be first activated. The Device DDOS is present even without the AFM module but when there is AFM module it can actually be controlled and configured(not only using the default values). The AFM rules themselves have a conext order(https://techdocs.f5.com/kb/en-us/products/big-ip-afm/manuals/product/network-firewall-policies-imple...). To see what part of the AFM is blocking you use the packet tracer tool:
If needed you can still place the ASM infront the APM by following:
Other F5 precedences is the GTM/DNS order :
The Local traffic object and VIP order for the LTM:
The F5 irule event order:
The picture of the F5 order is from the old F5 401 study guide:
As in the newer F5 TMOS versions the Bot defense is seperated from the DDOS Protection and as my tests confirmed first the ASM DDOS is activated then the Bot defense and after that the ASM policy and in the most F5 documentation maybe not writen good this is the case. In the older versons also first the DDOS filtered requests and then the Bot Defense further filtered the traffic before the ASM policy. As of now the Bot protection also generates support id, so if you are blocked and you see support id but in the security policy searches you can't find anything also search the support id under the Bot defence request logs as I found this the hard way.
The Bot defence can also make in some cases dynamic signatures for the DDOS in order to stop the traffic at the DDOS checks but I still have not seen this done.
For testing web DDOS attacks jmeter is a great free tool and for bigger commercial tests there is cloud platform named RedWolf but Jmeter in most cases will do just fine.