Forum Discussion
Greg_33558
Nimbostratus
NimbostratusWhere to find raw HTTP Request for ASM violations?
Where can I find the raw HTTP Request that is logged with ASM violations?
The version that is printed to the browser in Security->Event Logs->Application->Requests appears to be inserting a Replacement Character at the location of (presumably) a multibyte UTF-8 character. If I cut and paste the text from the browser, it contains a UTF-8 Left-to-Right Embedding character and the Replacement Character (with words in between). But I'm guessing that Replacement Character is not what the client actually sent, just what I'm ending up with after it's been processed by the web stack.
I have explored the underlying MySQL database a little and the violation itself is logged in PRX.REQUEST_LOG, but the HTTP Request that gets shown when I browse in the web UI is not in that table, nor do I see it in any other table in PRX. It's in the database somewhere, because I can string match it in ibdata1.
Does anyone know which table stores the HTTP Request raw data for ASM violations?
Is there any documentation for the schema of the databases in MySQL on the F5 (DCC, PLC, PRX, WIZ)?
Martin_VlaskoAltocumulus
Hi Greg,
I found a solution for myself and I believe it could help you too. If you are a friend with Python, you can expand existing version of f5-common-python SDK to provide access to raw Requests data.
I have posted my solution on project forum:
https://github.com/F5Networks/f5-common-python/issues/1486
The example I mentioned there is for Violations. But you can apply very similar code to fetch Requests as well.
I have used this SDK to write a script to export ASM learning suggestions to an Excel spreadsheet with almost all information which are normally available via GUI.
boneyardMVP
nice, thanks for sharing
Martin_Vlasko_1Altostratus
Hi Greg,
I found a solution for myself and I believe it could help you too. If you are a friend with Python, you can expand existing version of f5-common-python SDK to provide access to raw Requests data.
I have posted my solution on project forum:
https://github.com/F5Networks/f5-common-python/issues/1486
The example I mentioned there is for Violations. But you can apply very similar code to fetch Requests as well.
I have used this SDK to write a script to export ASM learning suggestions to an Excel spreadsheet with almost all information which are normally available via GUI.
- boneyard
nice, thanks for sharing
gowenfawrI have found the raw HTTP request in a file on the F5:
/var/adm/request_log/0000000001
This file appears to be a binary format which contains the raw requests and various other recognizable information about the request. For example, among a lot of binary character mush I can see 'no_ext', the URI, and the IP address of the virtual involved as well as the raw HTTP request. Clearly this file contains info about multiple requests, one after the other, with associated data.
It was possible to pull out the raw request and find what I suspected was true - the request contained a 0xe2 byte, which is a valid UTF-8 multibyte prefix, but without valid multibyte data following it (in fact, the next characters are the '
I would love to see tools or documentation for dissecting this file, but for my purposes it was sufficient to go in and search for the string I needed to extract and write out the lines of the HTTP request.
Amit_KarnikFor what it is worth....
I routinely use the export to export the violation data into Excel. You can then add some Excel formulas to mine the data off the excel sheet into additional columns.
It is not straightforward but most of the times it works for us with a little tweaking.
Please post to this forum if you do find a good way to analyze the data.
Best.