Using source address affinity persistence, but exluding certain IP addresses

I'm in the same boat but I'm not familar with irules. I have a pool with source persistance turned on. Unfortunately we have added 2 more servers to this pool but they aren't getting used because the hosts hitting them only have 2 servers. I would like to LB across all 4. Is there a way to turn off source persistance just for 2 incoming connections? If I have to create a new vip/pool I guess I could but it will require a lot of fw changes.