Limiting user instance access to VIPs.

I Guess, F5 APM can solve issue.

 

OR

 

Do you want to access specific vip from specific computer[Dynamic IP] then suggest to with client certificate authentication process.