unexpected response from LTM

The key is the persistence cookie. (I am assuming you have a timeout value of 30 seconds in the cookie persistence profile.) When you successfully connect to the OWA pages, the persistence cookie is sent with the response. Any subsequent requests to webmail.domain.com that occur before the cookie times out will persist rather than be load balanced, as the cookie is sent with the request to BIG-IP. This explains the 404 not found when you requested "anothernonexistinguri" within that 30 second period. That request persisted rather than load balanced.

If you wait until the cookie expires to send another request, then the cookie is not sent, and the request will either load be balanced, if the URI matches one of the rules in the iRule, or fail because there is no default pool, resulting in the "page cannot be displayed" sort of message.

You will not see a log message produced from the LB_SELECTED event if the connection persisted via the cookie. Persistence matching occurs before a load balancing decision.