Terminate SSL and open TCP socket connection inside

Can you provide the VS configuration from 'tmsh list ltm virtual ' and the associated iRule?

Client-side SSL and server-side HTTP design is usually accomplished by having a VS with a clientSSL profile, no serverSSL profile, and an HTTP profile. However, if there will be a mix of SSL/non-SSL pools associated with this VS, per the mentioned iRule, you may need to tweak the iRule (check out SSL::disable serverside - https://devcentral.f5.com/wiki/iRules.SSL__disable.ashx)

The answer to this question is simple. You can't do it.

If you have a virtual server with a HTTP profile attached and you receive traffic that is not HTTP the F5 will block the traffic. Nothing will be processed through the virtual server.

The nature of virtual servers is they are a one protocol shop. You cannot process more than one type of traffic per virtual. You can strip it down to layer 4 and decode packets and route based on that information but then you lose all your layer 7 HTTP processing you are doing now. You lose access to all the HTTP commands and associated events.

In theory you are able handle more than one protocol above layer 4 but in practice its difficult to do as you have to manually decode the traffic yourself to do it.

Let me be more specific...

Virtual servers are a single protocol up to layer four. If you have the skills to write a protocol dissector in an iRule then further traffic examination is up to you. I know of only one person James D. who can write these and he was my mentor. Unsurprisingly he now works for F5. To do what your doing requires examining the first packet then making decision on virtual server configuration as a result. This is risky because their is no guarantee on the fly changes to a virtual server will work as you would expect. Your far better having a seperate virtual server setup with the right protocol then forwarding the traffic to it when you detect the traffic. But first... you have to write the iRule. There are some examples on protocol dissection here on devcentral but I have been unable to find one for HTTP. Check out DNS examples for how packets can be opened up and examined. I warn you though it is not for the feint hearted.

This brings me to my second point. If you are sending multiple protocols down a single address and port then I suggest you pause, take a step back and look at the architecture of the solution. TCP is written in such a way that the destination port is the selector for the protocol or traffic type. What you are doing pretty much ignores this. You need to ask yourself is this really necessary? Is there a better we to do this? If their is a way to have different types of traffic come in on different ports then you should as it will vastly simplify your problem. Take a big picture look at the solution and I would suggest that only if their is no other alternative should you continue. I often tell my students (teacher/consultant), always look at the overall solution before writing an iRule. Sometimes the simplest changes can remove the need for iRules or vastly simply what you need to write.

Here is the answer.

Since all traffic on the outside is SSL, collect the first decrypted SSL packet after the handshake, read it to see if it has "HTTP" in it. If it does, go on to normal HTTP processing. If it doesn't, disable the http profile and allow it to connect through to the desired pool and port.