Keep in mind that the traffic is matched by the most specific IP forwarding virtual server. For example if you have virtual ip forwarding of 0.0.0.0:22 and 0.0.0.0:0 then the traffic for SSH will match the most specific defined.
For all my internal load-balancing VLAN’s where the LTM is the default gateway, I ALWAYS configure a wildcard (Port 0) network virtual server of type forwarding for INBOUND admin traffic. That’s usually enabled on all VLAN’s… (There are exceptions, but they’re not important here).
I tend to do the same for DMZ LTM’s. But the VS’s are usually configured so that ALL traffic BETWEEN, TO or FROM VLAN’s/DMZ’s is passed via the firewalls…
Then only the firewall needs to be concerned with whether traffic is allowed to pass from one network to another. The F5 is a Load Balancer (OK, Application Delivery Controller :), not a firewall.
Oh… ‘More Specific’ is a tricky subject with LTM… The definition changed from v4 to v9… In v9 the priority is on matching the MASK, not the port… See 404 Page not found | BIG-IP Documentation (Although I’m guessing not many people would be coming from v4 nowadays