Irule to limit access to VIP to only certain source ip addresses

In the BIG-IP management GUI, go to Network - Packet Filters. Select the "Enabled" option and then configure appropriately. See this guide for an extensive description of the packet filter feature:

Introduction to packet filtering