log illegal requests in ASM policy transparent mode

OOTB functionality is illustrated in the screenshot above. If you selected Log illegal requests and assigned it to the virtual server it should work. It sounds like maybe you have created a custom logging profile, which is fine. If you have a custom profile you need to assign it to the virtual server in the same way. Can you try sending a request you know is illegal, perhaps by triggering a simple attack signature? If you go to the Learning and Blocking Settings page, do any of the violations, especially attack signature violations, have the "Alarm" check box selected?