problems connecting socket via F5

Be sure that the target pool of that virtual server is port 443 (one port) instead of 0 (all ports). If you use 0, the flow would look like this:

 client(S: ephemeral, d: 10800) --> <big-ip> --> server(S: ephemeral, d: 10800)

but if the pool was port 443 explicitly, the BIG-IP automatically translates the destination port on the server-side of the flow, 

client(S: ephemeral, d:  10800) --> <BIG-IP> --> server(S: ephemenal, d: 443)

Hello,

you try to let the client establish a Websocket Connection. Websocket by itself is still part of http. 
So your Virtual Server is misconfigured. Please configure it like the *:443 Virtual Server. Also with required SSL Profiles. 

After that configure a websocket profile too on the *:10800 Virtual Server. Then this should work.

Regards

Thanks guys, i tried both your suggestions but sadly had no effect, we are still seeing the same error presented. I am starting to wonder if this is a backend issue but then again it works fine if you bypass the f5 so..

Without packet capture analyis (don't send it on this forum), we're kind of guessing. You could try removing the HTTP profile and changing the vip to be Fast L4 to see if it's an L5-L7(ssl, http) or an L3-L4(ip, tcp) issue. 

If you can't get it worked out, Support can help by analyzing your config and a packet capture: https://my.f5.com/manage/s/article/K2633

You have turned on the websocket profile haven't you?
Otherwise the f5 wont know to allow bidirectional comms.