Do I need a self-ip in the same subnet as my virtual servers, or a VLAN ID?

That depends. You don't NEED one, but if you don't have one, then BOTH the BIG-IP needs a way to route to your server from a different self-ip, either via a route or the default tmm gateway (it can NOT use the management gateway), AND the server needs a route to the self-ip it receives the traffic from. If you are on the same subnet it is easy because you don't have to configure any routing.

As for VLANs, the logic is similar, you have to make sure the ingress/egress traffic have the same VLAN tag or disable VLAN-keyed connections (not recommended).