You can use the session command to add a session table entry to map the client IP address to a proxy pool member on the client to forwarding VS connection. You could then look up the session table entry for the server to client connection.
Deb has posted a few good examples which show how to use the session table. As you'll be trying to access the same session table entries from different VIPs, you'll need to add the entries using the 'any virtual' option:
http://devcentral.f5.com/wiki/default.aspx/iRules/session
session add uie [list $myVar any virtual]
and to retrieve it:
set session_data [session lookup uie [list $myVar any virtual]]
Here are a few of Deb's posts on this to get you started:
looking inside persistence table, through different Virtual Servers
http://devcentral.f5.com/Default.aspx?tabid=53&forumid=5&tpage=1&view=topic&postid=1688717129
source address persistence
http://devcentral.f5.com/Default.aspx?tabid=53&forumid=5&tpage=1&view=topic&postid=16855
URI Session ID based persistence issue
http://devcentral.f5.com/Default.aspx?tabid=53&forumid=5&tpage=1&view=topic&postid=15647
Try reading over these posts and reply if you have any questions.
Aaron