Implementing a Form SSO through APM or IRules?

I'm not sure if this does exactly what you need, but it would be a quick try: Did you ever check "Pass-Through" in your form-based SSO config? This should pass the requests and responses through to the client instead of having any processing just on the F5 - side. Though i'm not 100% sure if really everything is passed through to the client, you would need to quick-test it.

Another way i could think of would be to do manual authentication in an iRule through a sidebanding request. Utilizing sidebanding, you could manually open the connection, do the form-post or whatever is needed, and just take the whole response-content and return it to the client.