Identity aware-based ACL problem

I want to apply ACLs based on user identity (actually, AD group membership). USers connect via RA-VPN though a full webtop. This is a piece of cake with APM using a AD Group Assignment Resource agent. BUT, the problem in this case is that there are THOUSANDS of servers and APMs ACLs does not support a hierarchy (right?). The maintenance job will be overwhelming. It is required that network resources can be added and removed with minimal work, not in several ACLs.

I looked a bit at AFM, and there you can group Addresslists and so on in policys, creating a hierarcy. But - AFM has no identity awareness (right?). And it does not seem possible to apply a AFM policy in APM VSE.

Is there a way to solve this in BIGIP? The fallback here is to use an external firewall, but I want to know if it is possible to solve this in BIGIP alone.