I set up a global log receiver and I'm getting the logs, but the request and response bodies are "UNKNOWN".
What do I need to do in order to get the payloads?
- I use a generic http server as the receiver
- I chose request logs because I care about the data coming from requests and responses
- I could only select current namespace for the Global Log Receiver which put it in system
I'm still learning it all so thank you for the help.
Couple of questions:
1. If it's not possible why does the request log send `req_body` and `rsp_body` keys anyways?
2. Is it possible to send the payload to a third party server in any other feature?
Thanks for the help.
Could you explain the use-case/need for logging request and response bodies? At present, we dont support it as a feature, but for customers who enabled "API Discovery" on their http lb, we log request headers and body, response headers and body for a sample of requests . So, if you have purchased API Security SKU and enabled API Discovery feature, a sample of the request logs will have these fields populated and the rest of the logs these fields will be shown as "unknown"
The video attached to the following article walks you through enabling the API protection capabilities, including discovery. The configuration walk through is covered in the first couple minutes of the video.
I am not aware of any way you can control sampling process. Sudhir may have a better answer to that piece.
Thank you for the reply.
I started using the API Discovery sampling but I've noticed that even when it's a log that sampled the data it could still be missing some data.
What do I mean is when it is not sampled data req_body and rsp_body are empty, req_headers is "null" and rsp_headers doesn't exist.
When it is sampled data is should be that all four have the same value from the requests and responses, but that isn't true a lot of the time.
A lot of the time even if I have the bodies, the req_headers could still be "null" or rsp_headers could be empty and vice-verse, if I have the headers I still see sometimes that the bodies are empty.
Is this a bug? Or is it on purpose sending partial data even with the API Discovery feature enabled?
Thank you for bringing up the topic of the sampling mechanism in our requests and responses. I understand that there might be some confusion, so let me clarify the situation for you.
The sampling mechanism we have in place is indeed asymmetric for requests and responses. This means that it is not guaranteed that all four components (Req_header, Req_body, Res_Header, Res_Body) will always appear in the same record. In some cases, you might receive only a subset of these components in a single record, depending on the data being collected.
On the other hand, while there is asymmetry in how the data is sampled within a single record, there are symmetries in the collection of headers and body for separate requests or separate responses. This means that across multiple records, you can expect to find a balanced representation of headers and bodies for both requests and responses.
We understand the importance of obtaining comprehensive data, and we are continuously working to optimize our sampling mechanisms to improve the overall data collection process. If you have any specific requirements or concerns about the data you are receiving, please feel free to let us know, and we'll be more than happy to assist you further.