In my environment, whenever I use IP address in place of hostname I get prompted for authentication no matter whether I'm going direct to CAS or via the F5. This makes sense because the bare IP address in the URL wouldn't be considered an IE "Trusted Site" so it wouldn't attempt
automatic NTLM authentication with it. It would prompt me to manually enter the authentication.
What happens if you set a host file entry for "autodiscover.site.com" pointed at the F5 IP address, then try the request in IE as "https://autodiscover.site.com/autodiscover/autodiscover.xml"? Make sure to remove the entry later after you test. :>
I've seen four main things go wrong with autodiscovery on Exchange 2010 -- the cert on the autodiscover site does not have the "autodiscover" name set as either a primary or SubjectAlternativeName, the cert for autodiscover is not able to be verified from a trust perspective (self-signed or unknown CA), the Outlook profile the wrong Authentication Type set for Outlook Anywhere connectivity, or the configured Redirect URL on the autodiscover site itself is incorrect. In your case, it sounds like when passing through the F5 VIP the system has some reason to believe that it can no longer do automatic windows integrated authentication -- this could occur as the result of a certificate mismatch or a failure to believe the site can be classified as trusted.