We have a VIP where one specific request (GET/Terms) which sometimes fails on specific nodes in the pool. In AppInsights we see 400 errors for that specific request, but we don't see anything in the IIS logs. So we're looking at the F5 to try and see if we can get any ideas why this is happening. In the past I've used tcpdump's to filter down to those affected items and capture the traffic, however this problem is very infrequent. (We haven't seen it happen in the last 12 hours).
What would be the best approach to tracking down this traffic and error so we can see what is going on? What would you do to see the traffic?
@SteveEason Unless the F5 is configured to respond with the 400 status code this would not show up in the LTM logs. If you all have looked at the server and never see the 400 response then you might check on the client side and see what is the value in the HTTP header "Server:" to see if you can have a better understanding of what device is responding with that 400. In addition to this I would setup a rotating tcpdump on the F5 that saves to a file and then overwrites that file once it reaches a certain file size. Doing this rotation should allow you enough time, assuming this isn't a high traffic virtual server, to log into the F5 and export the capture and then parse through it for the specific HTTP response status code of 400. Troubleshooting intermitent issues is always troublesome because it makes it difficult to see exactly what was happening at the specific time of the clients query.