I don't think so. The only feature that can help a little is the ip intelligence as it will block bad source ip addresses from bad domains.
F5 had a MSM module but that was a long time ago:
If don't want to buy another device you may consider SMTP relay services like SendGrid etc. that are cloud based offerings. Symantec even has Email Security.cloud but from F5 point I think that they have given up on this market :
Also I just renembered that you can use the URLDB for the SWG module to filter based on FQDN/Domain names:
Still you will need to see if you can do this for the SMTP traffic and it will be hard work and maybe not worth the efford:
About the irule I don't think there is exactly what you are looking for but probably it could be written but it will take a lot of time to capture the traffic with TCP::collect and to match on the "From: email@example.com" value in the emails but I admit don't have a use case to try do it. This is why I gave you the example with the iRule with smtp-filter-and-proxy.
For PTR if your device is not the DNS/GTM you may extract the needed value from the email payloadd and with the "RESOLV::lookup" to check it as I don't know if ''NAME::lookup'' supports PTR resolution:
A good note is you can use free feed lists as alternative to the F5 URLDB service for the SWG module like Minemelt if you have AFM you can add custom feed lists and if not you may check the tabul CVS importer iRule that I also use for some things.
Another helpfull article like the one for the SMTP proxy:
Also you may try using a Stream profile than TCP::collect or SSL::collect for encrypted SMTP traffic as mentioned in: