Irule persist lookup source_addr for specific virtual server

Unless 'Persist Across Virtuals' is set then the persistence is per virtual server anyway. If it is set then yes, you can control this with the command as per https://devcentral.f5.com/wiki/iRules.persist.ashx

persist lookup   [all|node|port|pool]  
"all" or no specification returns a list containing the node, port and  pool name.  
Specifying any of the other return types will return the specified item only.       
 =  | {  [any virtual|service|pool] [pool ] }        
the latter key specification is used to access persistence entries across virtuals, services, or pools.