Forum Discussion
Download file from AWS S3 bucket / http profile
Hello,
Resolved, add only : HTTP::header replace Host "hostname"
FYI : url is : hostname.bucket.xxxxxxaws......
Hi cpt_ri_F5 ,
Could you please disable any AWAF/ASM Policies if exists , Remove attached iRule and convert Virtual server type to performance layer 4 , If the issue is resolved so , Keep disabling ASM policy and attach http profile again , and test one more time.
If it works with http profile , so you have issue with AWAF policy and need to check yout policy and AWAF system Variables.
If the Download didn't work so you the issue is with http profile , and you need really to disable it , in this Case I recommend to add this :when HTTP_REQUEST { if { [class match [string tolower [HTTP::uri]] contains uri-data] } { HTTP::disable log local0. "Condition True in Request" return } } when HTTP_RESPONSE { HTTP::disable log local0. "Response Action achieved" }
> I addedd [Log local ] as a guide to make sure that your requests match with Conditions of if statement , and to make sure that http profile really disabled.
> I added Return to break the irule , and prefer to attach the Pool "MYPOOL" under virtual server resources.
The Last thing if this available with you, Take a Packet capture for both sides ( Client and server side ) with identifing the Download file request to see why it gives you an empty/corrupted file.
Follow the above steps , I hope this helps youJust like Mohamed_Ahmed_Kansoh said, you really need to look closer in what variables you can eliminate, and the actual data from TCP dump. Open in wireshark, and check to see that the HTTP request leaves the F5 (sounds like it does since you get an AWS error messsage). Look at the host header, make sure it is correct. Also, look at the URI and make sure it is correct on the server side of the connection. Also, make sure you are using the same protocol -- HTTP and HTTP or HTTPS and HTTPS, if you are doing a conversion it may not work if AWS is not listening on 80 or 443.
Just a few more nuggets to dig and check 😉 Definitely an interesting issue!
- cpt_ri_F5Cirrostratus
Hello,
Resolved, add only : HTTP::header replace Host "hostname"
FYI : url is : hostname.bucket.xxxxxxaws......
Excellent. Checking headers for the win! 🙂
So, 2 questions here:
- Does datagroup 'uri-data' actually exist? Do you have the URI path there for the file you are looking to download?
- The HTTP::disable in the HTTP_RESPONSE will trigger on EVERY request, regardless if there is a match in the HTTP_REQUEST. You need to specify a variable in the HTTP_REQUEST and check it in the HTTP_RESPONSE. I presume that when that particular URI is matched, you only then want to disable the HTTP profile for that session -- both client side (http_request) and server side (http_response.- cpt_ri_F5Cirrostratus
Hello,
- Yes, the 'uri-data' datagroup exists (also tested with fixed uri in code)
- Same problem, HTTP_RESPONSE with matched URI !
I presume that when that particular URI is matched, you only then want to disable the HTTP profile for that session -- both client side (http_request) and server side (http_response.
=> I don't know how it works, I found this solution in the two links shared, I'm interested in any other solutions.
My question, is there a special configuration when trying to view or download a file (http://VS_F5/path_file) with a Standard VS ?
I would also like to confirm: I access in http and the backend servers are in https, correct : no client ssl, with default serverssl ?
Thank you
Ok, so you have a virtual server configured. What do you have configured as Pool Members? Sounds like you have a standard VS and want to server side proxy to AWS S3 buckets?
Silly question, but do you have SNAT enabled on VS? Could be routing issue.
- cpt_ri_F5Cirrostratus
Hello Mohamed_Ahmed_Kansoh, whisperer
Thank you for yor answers,
- Not ASM/WAF
- L4 (with/without http) : download empty/corrupted file.
- L4 (with http + Mohamed_Ahmed_Kansoh's iRule) : download empty/corrupted file.
- Standard (with http + Mohamed_Ahmed_Kansoh's iRule + no serverssl ) :
<HTTP_REQUEST>: Condition True in Request
download empty/corrupted file.
- Standard (with http + Mohamed_Ahmed_Kansoh's iRule + serverssl ) :
S3 XML Error :
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist</Message>
<HTTP_REQUEST>: Condition True in Request
<HTTP_RESPONSE>: Response Action achievedI am in http (client) and https (server), MYPOOL S3 := 443
I keep looking at the pcap....
Thank you.
Hi cpt_ri_F5 ,
Can you bypass bigip and try to download it directly from server ? To make sure that bigip is point of interrupt your download.
Pcap may give us some insights , you may find TCP-Window-FULL and TCP-Zero-Window.
you may need to increase bigip buffer of window scale from bigip.
But test first from server itself directly without bigip in path.
- cpt_ri_F5Cirrostratus
+
from F5 :
curl http//fqdn/path_file
=> okcurl https//fqdn/path_file
=> okcurl http://IP_node/path_file
=> same S3 XML Errorcurl https://IP_node/path_file
=> SSL: certificate subject name 's3.xx-xxxx-x.amazonaws.com' does not match target host name IP_node
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com