GTM/DNS as a smart DNS server ("without" LTM)

F5 GTM(DNS) can work directly with 3-rth party servers without LTM and you may use health monitors like SNMP or ICMP or HTTP/HTTPS for the monitoring:

https://techdocs.f5.com/kb/en-us/products/big-ip-dns/manuals/product/bigip-dns-implementations-12-1-0/21.html

Better to also configure a prober pool to select which BIG-IP devices (LTM or DNS/GTM) will send the probes to the generic hosts as otherwise the F5 devices will use and automatic algoritam to delegate the monitoring to one of them that will inform the other F5 devices if the servers are up:

https://support.f5.com/csp/article/K08433560

https://support.f5.com/csp/article/K10385

You can use DNS irules to make more specific logic like for specific client ip addresses or hostnames to return the pool with generic  host servers and not the one with LTM Virtual servers:

https://community.f5.com/t5/technical-forum/gtm-irule-to-set-the-pool-based-on-which-gtm-is-responding-to/td-p/109187

https://clouddocs.f5.com/api/irules/DNS_REQUEST.html