13-Jun-2023 10:33
Solved! Go to Solution.
14-Jun-2023 03:20
Hello,
Resolved, add only : HTTP::header replace Host "hostname"
FYI : url is : hostname.bucket.xxxxxxaws......
13-Jun-2023 12:17
So, 2 questions here:
- Does datagroup 'uri-data' actually exist? Do you have the URI path there for the file you are looking to download?
- The HTTP::disable in the HTTP_RESPONSE will trigger on EVERY request, regardless if there is a match in the HTTP_REQUEST. You need to specify a variable in the HTTP_REQUEST and check it in the HTTP_RESPONSE. I presume that when that particular URI is matched, you only then want to disable the HTTP profile for that session -- both client side (http_request) and server side (http_response.
13-Jun-2023 12:43
Hello,
- Yes, the 'uri-data' datagroup exists (also tested with fixed uri in code)
- Same problem, HTTP_RESPONSE with matched URI !
I presume that when that particular URI is matched, you only then want to disable the HTTP profile for that session -- both client side (http_request) and server side (http_response.
=> I don't know how it works, I found this solution in the two links shared, I'm interested in any other solutions.
My question, is there a special configuration when trying to view or download a file (http://VS_F5/path_file) with a Standard VS ?
I would also like to confirm: I access in http and the backend servers are in https, correct : no client ssl, with default serverssl ?
Thank you
13-Jun-2023 12:46 - edited 13-Jun-2023 12:48
Ok, so you have a virtual server configured. What do you have configured as Pool Members? Sounds like you have a standard VS and want to server side proxy to AWS S3 buckets?
Silly question, but do you have SNAT enabled on VS? Could be routing issue.
13-Jun-2023 12:56
yes, a standard VS, pool fqdn (up)
wWy proxy? I just want to retrieve the file in http, curl or wget
Not snat, automap (route ok)
Thanks
13-Jun-2023 13:26
Your response is confusing. "Not snat" - so this is not enabled? You are also using a pool of FQDNs. Do you have DNS enabled on the F5 and is this being properly resolved to the IP addresses that host your S3 bucket? Have you tried to perform a TCP dump on both client side and server side of the connection? Have you tried to tail -f /var/log/ltm to see if there are any errors flagged?
13-Jun-2023 14:27
Sorry, snat with automap, fqdn proprly resolved (pool up), no error /var/log/ltm, tcpdump client, serveur ok ( Syn, Syn Ack, Ack, ...GET, Continuation ...)
Now it shows me: "the specified bucket does not exist",
But, the file exists on S3! + "curl backend/path" is ok from F5
Thanks
13-Jun-2023 16:21 - edited 13-Jun-2023 16:24
Hi @cpt_ri_F5 ,
Could you please disable any AWAF/ASM Policies if exists , Remove attached iRule and convert Virtual server type to performance layer 4 , If the issue is resolved so , Keep disabling ASM policy and attach http profile again , and test one more time.
If it works with http profile , so you have issue with AWAF policy and need to check yout policy and AWAF system Variables.
If the Download didn't work so you the issue is with http profile , and you need really to disable it , in this Case I recommend to add this :
when HTTP_REQUEST {
if { [class match [string tolower [HTTP::uri]] contains uri-data] } {
HTTP::disable
log local0. "Condition True in Request"
return
}
}
when HTTP_RESPONSE {
HTTP::disable
log local0. "Response Action achieved"
}
> I addedd [Log local ] as a guide to make sure that your requests match with Conditions of if statement , and to make sure that http profile really disabled.
> I added Return to break the irule , and prefer to attach the Pool "MYPOOL" under virtual server resources.
The Last thing if this available with you, Take a Packet capture for both sides ( Client and server side ) with identifing the Download file request to see why it gives you an empty/corrupted file.
Follow the above steps , I hope this helps you
13-Jun-2023 17:53
Just like @Mohamed_Ahmed_Kansoh said, you really need to look closer in what variables you can eliminate, and the actual data from TCP dump. Open in wireshark, and check to see that the HTTP request leaves the F5 (sounds like it does since you get an AWS error messsage). Look at the host header, make sure it is correct. Also, look at the URI and make sure it is correct on the server side of the connection. Also, make sure you are using the same protocol -- HTTP and HTTP or HTTPS and HTTPS, if you are doing a conversion it may not work if AWS is not listening on 80 or 443.
Just a few more nuggets to dig and check 😉 Definitely an interesting issue!
14-Jun-2023 01:51
Hello Mohamed_Ahmed_Kansoh, whisperer
Thank you for yor answers,
- Not ASM/WAF
- L4 (with/without http) : download empty/corrupted file.
- L4 (with http + Mohamed_Ahmed_Kansoh's iRule) : download empty/corrupted file.
- Standard (with http + Mohamed_Ahmed_Kansoh's iRule + no serverssl ) :
<HTTP_REQUEST>: Condition True in Request
download empty/corrupted file.
- Standard (with http + Mohamed_Ahmed_Kansoh's iRule + serverssl ) :
S3 XML Error :
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist</Message>
<HTTP_REQUEST>: Condition True in Request
<HTTP_RESPONSE>: Response Action achieved
I am in http (client) and https (server), MYPOOL S3 := 443
I keep looking at the pcap....
Thank you.
14-Jun-2023 02:23 - edited 14-Jun-2023 02:23
Hi @cpt_ri_F5 ,
Can you bypass bigip and try to download it directly from server ? To make sure that bigip is point of interrupt your download.
Pcap may give us some insights , you may find TCP-Window-FULL and TCP-Zero-Window.
you may need to increase bigip buffer of window scale from bigip.
But test first from server itself directly without bigip in path.
14-Jun-2023 01:51
+
from F5 :
curl http//fqdn/path_file
=> ok
curl https//fqdn/path_file
=> ok
curl http://IP_node/path_file
=> same S3 XML Error
curl https://IP_node/path_file
=> SSL: certificate subject name 's3.xx-xxxx-x.amazonaws.com' does not match target host name IP_node
14-Jun-2023 03:20
Hello,
Resolved, add only : HTTP::header replace Host "hostname"
FYI : url is : hostname.bucket.xxxxxxaws......
14-Jun-2023 04:28
Excellent. Checking headers for the win! 🙂