Silverlight XAP caching

We are facing a problem when used with F5. When ever user logs into a computer for the first time or they closes down all instance of IE and open a brand new IE and hit our Silvelright Page, Silverlight end up downloading the XAP file even though the file is already in the cache. Is there a rule or something in F5 we can change not to do it if the XAP is same version? If I remove F5 and hit the server box, it does not download until unless it is a new version and if the IE is closed or not.

Thanks