Forum Discussion
paolodaniele
Altostratus
AltostratusWSS LTM not passing data correctly
Hi,
i'm having some trouble about running an Virtual Server in front of a K3s ingress.
I've setted up 4 virtual server, all point to the same destination address, but different port (different pool but final address is the same, only change the port)
For Http/s, MQTT/s zero problem, but when i'm trying to redirect the http traffic using websocket i've several problem.
Destination return several errors based on the Virtual Server tuning.
Actually i've the virtual server configured in TCP with HTTP profile (Websocket enable both client and server) and in security profile i've created a wss security to allow * addresses (HTTP,HTTPS,WS,WSS)
But when we try the application it return ERR_TRUNCATE_HEADERS. I've also create a new http profile based on the default, to increse the size of che Header Payload.
Also no i'm not checking the payload, and ignore any possible malformed packet (just to let everything pass).
On the application event log i can see all the request and i see the green thick so i suppose that everything is fine on the F5 side.
How can i be sure of that? Is there a way to do extra check or am i missing something?
Thank you guys.
Paolo
Hi Paolo,
Thank you for your time today, im glad we could determine what was wrong.
A few steps we took that helped us confirm first off, that no truncating was happening in the headers, was to capture the connection in a tcpdump with the f5 high detail peer options (:nnnp), that showed us the payload on the backend as being perfect. No truncating happening.
tcpdump -vvv -s0 -ni 0.0:nnnp host <f5 VIP> and port <f5 virtual server port>One of the key factors to identifying what was the issue here was something you sent me in a private message. Removing the http profile, which subsequently resultied in the websocket, waf and clientssl profile to be removed, allowed the connection to work. The wss:// call, not ws://.
Because this was a WSS or WebSocket Secure call, the server was expecting a TLS session, and our configuration was doing TLS offloading at the f5, and sending the traffic plain text to the server. Why the application generated an "ERR_TRUNCATE_HEADERS" message is purely speculation at this point, but i suspect it was because the headers were all encrypted and thus exceeded the maximum for the application.
I would normally expect to see 400 errors with HTTP, so the backend encryption wasnt jumping out at me sooner.
Im glad that we could resolve this matter, and please let myself and f5 know if there is anything else we can do to assist you in the future.
Cheers
Mike
mdevlinEmployee
paolodaniele What version of code are you running?
Have you uploaded a qkview to ihealth?
Can you provide us the configurations of your virtual server, iRules (if applicable), policies and profiles (if you cant upload a qkview)
- mdevlin
Additionally, if you remove the ASM security profile (for testing purposes), does it function as required/desired?
paolodanieleI've tried to remove the ASM and it's not work.
Btw the version 16.1.2.2 so i've not the problem that had older version (i've done some digging before writing you!)
As i told to buulam i'm pretty noob with f5 and i didn't know the iHealth portal. I'm uploading the file to check with the tool.
If i cannot find anything i'll post you the config part (i've to blur some ip and name for gdpr privacy)
Thank you
- mdevlin
Hi Paolo,
Thank you for your time today, im glad we could determine what was wrong.
A few steps we took that helped us confirm first off, that no truncating was happening in the headers, was to capture the connection in a tcpdump with the f5 high detail peer options (:nnnp), that showed us the payload on the backend as being perfect. No truncating happening.
tcpdump -vvv -s0 -ni 0.0:nnnp host <f5 VIP> and port <f5 virtual server port>One of the key factors to identifying what was the issue here was something you sent me in a private message. Removing the http profile, which subsequently resultied in the websocket, waf and clientssl profile to be removed, allowed the connection to work. The wss:// call, not ws://.
Because this was a WSS or WebSocket Secure call, the server was expecting a TLS session, and our configuration was doing TLS offloading at the f5, and sending the traffic plain text to the server. Why the application generated an "ERR_TRUNCATE_HEADERS" message is purely speculation at this point, but i suspect it was because the headers were all encrypted and thus exceeded the maximum for the application.
I would normally expect to see 400 errors with HTTP, so the backend encryption wasnt jumping out at me sooner.
Im glad that we could resolve this matter, and please let myself and f5 know if there is anything else we can do to assist you in the future.
Cheers
Mike
- paolodaniele
Thank you Mike, Bu and all F5 Team for your help. It was very hard to find a solution because we do not know the application behaviour.
Best Regards,
P.
- buulam
Admin
Hi paolodaniele , I'm not finding anything on the error you've noted. I'm asking around to see if there's any more thoughts on this. Seeing the requests in your application event log seems to be a positive sign for the BIG-IP passing along the traffic.
- buulam
paolodaniele one thought was if you've tried to pass the traffic with an http profile attached at all to see if the traffic can do a simple pass through first?
- paolodaniele
Hi Buulam,
thank you for your answer. All i can say is that i'm quite noob with F5(i've always used other solutions) but i was able to learn quickly how it works and how to configure all things (the LTM HTTPS works fine without any problem)
All i can tell is that if i configure to pass not only TCP but instead all protocols (so no configuration of websocket or anything else) when i'm try to wscat to wss://myaddress:myport i get the Connected message and its open the wss shell. Using this configuration when i run the wss://myaddress:myport i get the answer that it expects the request ad HTTP.
This is why i'm thinking that it's me who's miss something in the config. Let me know if you want to see part of the configuration.
Thanks for your help.
Regards,
P.
- buulam
Right, so the test I'm suggesting where you remove the http profile will still actually let you pass the websocket traffic. It will do so without passing through the http proxy and can isolate things further for you.
