Forum Discussion
James_Wrubel_48
Nimbostratus
Nimbostratusserver_data workflow - what am I doing wrong?
Hi all,
I've been working for a while to set up the F5 to act as an http wrapper of sorts for a TCP-based protocol (VNC). We've gotten pretty far, but we have sequences where the VNC server is sending multiple responses that get concatenated into one where we think the iRule should be sending them as individual responses. Here's the server_data block:
when SERVER_DATA {
::phase is a global variable to handle cases where we *want* to concatenate two specific responses
if { $::phase >=2 } {
log local0.info "collecting additional data"
set ::phase 1
TCP::collect
return
}
grab the payload from the server, and base 64 encode it. Get the length of the output to be returned
with the http header
set encoded_payload [b64encode [TCP::payload]]
set encoded_payload_length [string length $encoded_payload]
if {$::debug>=2} {
log local0.info "responding with $encoded_payload"
}
empty the current payload
TCP::payload replace 0 [TCP::payload length] $encoded_payload
grab the current date so the http response is accurate
set cur_date [clock format [clock seconds] -format {%a, %d %b %Y %I:%M:%S}]
build the http response header with the current date, appropriate response length (of base64 encoded response)
set http_wrapper "$::http_version 200 OK\r\nContent-Length: $encoded_payload_length\r\nContent-Type: text/plain\r\nLast-Modified: $cur_date GMT\r\nDate: $cur_date GMT\r\n\r\n"
add the http header to the beginning of the response
TCP::payload replace 0 0 $http_wrapper
}
stop collecting and forwarding collected or modified payload data
TCP::release
if I don't call this new data won't get wrapped in HTTP headers
TCP::collect
}
What we get back is fine until we start getting a 'lot' of data from the server - Then we start to get single responses back to the client that contain multiple combinations of http_wrapper and the encoded payload. What it looks like is the f5 may not be calling release fast enough, so by the time it does the iRule has processed sometimes up to 5 passthroughs of server_data. Does that sound possible?
Jim
hoolioCirrostratus
If a developer doesn't respond to this with more information, you could open a case with F5 Support. When you get an answer, could you reply here?
Thanks,
Aaron- I don't quite understand what you're saying here. In particular, the phrase "single responses [...] that contain multiple combinations of http_wrapper and the encoded payload" doesn't make sense to me. Do you mean "single packets"? If so, then that is to be expected; there is no way to explicitly packetize responses to clients via iRules.
You're not calling any commands that would suspend iRule execution ("session", "persist", or "after"), so you're not running into the problem where new server data may suddenly "appear" before the release is processed.
I think you might be misunderstanding what TCP::release does. Because the BIG-IP is a full proxy, when you release this data on the serverside, it still has both the internal proxy and clientside TCP stack to traverse before it is sent out of the BIG-IP. It is very possible that they would repacketize the data in some fashion before sending it out to the client. You might be able to force the issue by adjusting things like the proxy buffer size and tcp profile options (window size, tcp options, &c.) so that the BIG-IP is not allowed to buffer up multiple responses into a single packet.
Again, this is all predicated on you meaning "single packet" above. If you meant something else, please try to clarify and I'll see what I can do. 
James_Wrubel_48Spark - thanks for the assist. Some background would definitely help. We have a business requirement to develop a VNC client that will connect to a specific VMWare ESX server based on some application rules. The challenge is that our end-user base sits behind networks that are well-locked down; only http/https allowed outbound. Since the ESX servers are already behind an LTM our bright idea was to use an iRule to wrap the VNC socket data as an http payload. This actually works well for the initialization routines, which are basic request-response. The issue arises when the server starts sending screen refresh data. We make a single request which is essentially 'redraw the rectangle with these coordinates', and the server responds with a series of small updates that together make up the full requested screen data. What we had intended to happen is that each of these updates (which I believe come in as individual packets, or at least that's what we see in Wireshark on the server) would get its own http header and be sent as a single distinct response. What we see on Wireshark is a single packet with multiple combinations of http headers (that we add in the iRule) and payload all in one packet. So you may be right that due to the speed of socket data coming over the wire, the iRule fires several times but the tcp stack grabs multiple responses and send them together. Since the socket data from the server is of variable size I'm not sure we can adjust the parameters to force the LTM into a single response. We should be able to inspect the payload and mmake send/don't send decisions based on application data but I was hoping to avoid that since there are six or so different message types and that could be an ugly iRule. :>- If I read this right, then you're intentionally sending multiple HTTP responses (with the encapsulated screen data) for a single request? I'm surprised that the clients' HTTP proxy/firewall allows this at all, and I'm even more surprised that it breaks when the multiple HTTP responses are contained in the same packet (which is completely legal). I'd first make sure that this really is what the situation is by crafting some test programs and not use the BIG-IP or ESX server at all.
If that is the real situation, I can come up with 2 ideas to maybe solve this problem:
1) For the screen refresh data, use the CONNECT method through the clients' HTTPS proxy, which would simplify the iRule greatly.
2) If the server really always sends responses only in single packets, then you should be able to minimally parse the responses to bundle up all of the screen data into a single HTTP response.
Also, depending on the precise nature of the clients' proxy/firewall, a series of 100 responses might be more readily accepted than a series of 200 responses. You might also need the appropriate Expect header on the client request. Just some thoughts...