Rescricting network access to ltm gui

We are in the process of locking things down on our network, but one thing I can't figure out how to do is restricting access to the gui. I figured out how to do ssh restricting, but not gui console. Is there a magical trick I am missing?