Forum Discussion
CirrostratusiRule to send multipart file upload to a different pool
Hi,
I have an application that allows users to upload some very large files (GBs). The application chunks it into 10MB parts and send the requests via the F5. Currently when you get to the 1GB mark the download pauses. It can be resumed, but this is less than ideal.
It has been identified that it's the webserver behind the F5 causing this issue. A new web server has been stood up on the box specifically to handle file uploads/downloads on a non standard port. The would like upload requests to be sent to a new pool that contains members using this new port.
Is it possible to select the pool to be used on a multipart upload? URI match will be difficult because it's a portal application. How does the F5 see the chunked requests? all as one request? or as individual requests? Sorry if that should be obvious!
Thanks
raZorTTAn initial test where I was looking at http header for content-disposition and setting the pool only seemed to work on the first chunk. Subsequent connections from F5 to the pool was to the default pool.
When you use the term "chunk" do you mean true HTTP chunked transfer encoding? Or is this something custom-built the application itself does to break the large file into separate 10MB pieces, then send over separate connections? If multiple connections, you might be able to incorporate some sort of persistence. If there's any part of the payload that indicates this is one of those special large file uploads, you could use that with a universal persistence profile and an iRule to force the connection to persist to the same server as previous connections.
- raZorTT
Hi Cathy
Thanks for your response.
The upload is being handled by an Ajax component (PrimeFaces).
Using Fiddler on the client side I can see the individual HTTP POSTs which contains the following headers
- Connection: Keep-Alive
- Accept-Encoding: gzip, deflate
- X-Requested-With: XMLHttpRequest
- Content-Disposition: attachment; filename="a_big_file.zip"
- Content-Length: 10001628
- Content-Range: bytes 0-9999999/1462538240
- Content-Type: multipart/form-data; boundary=---------------------------7e11932a13e0402
Here are the entries from /var/log/ltm for the first 4 HTTP POSTs that I see in Fiddler. You can see the first request/chunk has the pool changed, but the subsequent ones use the default pool. I only see an a single HTTP_REQUEST event, which is where the logic to select pools currently is.
Aug 9 09:52:42 F5XD01 debug tmm[10989]: Rule /Common/irule_file_upload_check : file upload, change pool Aug 9 09:52:42 F5XD01 info tmm[10989]: Rule /Common/LB_events : member selected: /Common/pool_file_upload 10.161.22.45 10099 Aug 9 09:52:42 F5XD01 info tmm[10989]: Rule /Common/LB_events : server connected Aug 9 09:52:42 F5XD01 debug tmm[10989]: Rule /Common/irule_file_upload_check : request send
Aug 9 09:52:44 F5XD01 info tmm[10989]: Rule /Common/LB_events : member selected: /Common/pool_default_app 10.161.22.45 80 Aug 9 09:52:44 F5XD01 debug tmm[10989]: Rule /Common/irule_file_upload_check : request send
Aug 9 09:52:46 F5XD01 info tmm[10989]: Rule /Common/LB_events : member selected: /Common/pool_default_app 10.161.22.45 80 Aug 9 09:52:46 F5XD01 debug tmm[10989]: Rule /Common/irule_file_upload_check : request send
Aug 9 09:52:47 F5XD01 info tmm[10989]: Rule /Common/LB_events : member selected: /Common/pool_default_app 10.161.22.45 80 Aug 9 09:52:47 F5XD01 debug tmm[10989]: Rule /Common/irule_file_upload_check : request send
- raZorTT
I think I found my problem!
There is an event disable statement in another irule in the http_request event. Which, if I understand it correctly, means that the http_request event will be disabled for all future requests on the same connection.
What is considered best practice when you have multiple irules that look to specifically set the pool based on certain conditions? Set a variable in the client_accepted event to store the selected pool?
Cheers,
Simon
