Forum Discussion
NimbostratusCreating an iRule to send/extract the payload of HTTP request to remote server
MVPHi Victor,
it is possible, you can use this iRule as a basis to start from and modify it to your needs:
Log large HTTP payloads in chunks locally and remotely
From a performance perspective it is OK, since it uses HSL for remote logging. Beware to set the limit for the collected payload to 32 MB, if you try to log anything larger than 32 MB your TMM will crash.
But maybe if you explain the use case, there is a better solution than doing this with iRules and also maybe there is a better way than doing this on the BIG-IP.
KR
Daniel
NimbostratusHello Daniel!
Firs of all, thank you so much for your information!
Let me explain a little bit more.
Today, we have 2 remote servers with booth of them are receiving the logs of our F5 Appliances. We are working to future desable the local logs of our appliances for only make troubleshooting with our 2 remote servers that are receiving this logs internal only. The problem is, we're sending to every alert/block the "query_string" field, but in some cases, the field "query_string" don't show anything, the field are there, but without any information, we detected that some type of signatures like XSS and SQL do not send this values of the exacly query string that match with the attack signature, but if we see it in our local log in F5 we can see the string that matches with the Attack Signature.
Best Regards,
Victor.
gersbahCirrostratus
Maybe I misunderstood your question, but you can select the "request" field in your log profile to log the entire HTTP request (headers + body).
Of course for big requests this will still be limited by the max entry length you also configure in the log profile, but maybe the 64k or whatever the max length for standard log profiles is, is already enough for your use case.
If you do use this, make sure it's the last field in your log format. That way if a request does exceed the limit, you don't lose any other log fields.
- gersbah
Oh also, if you're looking for troubleshooting information, the "violation_details" field is very helpful. That's an XML blob that contains (partially base64 encoded) information about what exactly matched where and why. This one can also get very big, depending on the number of violations.
- Victor_Soares
Hi Gersbah, that's the thing!, we're sending all of fileds thath contains in F5 logging profile, if we can only receive always the field "query_string" with the match parameter or string that matches an a attack signature, that way is sufficient for us. Thinking about exacly what you say, if the request it is to long, we'll have a problema with the lenght, so, there is a way to "fix" when the query_string come without any information ? even though on the local log of our appliances ?
Thank you,
Victor.
- gersbah
If you try to log literally everything remotely, you will probably always run into length issues.
Remote logging in my opinion is meant to be selective.
Something that could work (but I've never done it myself) is to set multiple remote logging profiles to overcome the length limitations. For example one for your "standard" log with whatever log fields you typically need as your standard "access log". Then a second profile with only support ID (so you can identify the log entry) and request (to have as much of the full HTTP request as possible). And a third profile with only support ID and violation_details (again ID to match and then as much as possible of the violation_details field). And you can send those to the same or different log destinations.
All in all, I'm not convinced it's a good idea to try to completely move away from the local log of the bigip. I don't think you will get the same quality and user-friendliness with remote logging.
- Daniel_Wolf
Hi Victor,
somehow I didn't understand from your question that you are using AdvWAF.
Did you try to reorder the fields and move the query_string itm further up in the list of selected items in the Logging Profile?
I can't reproduce it in my lab, but I think it is worth a try.
KR
Daniel
- Victor_Soares
Daniel,
I have not performed this test yet, but i'll try to further up the query_string and see if this change will fix our problem.
Thank You so Much!
