Forum Discussion

PSFletchTheTek's avatar
Jan 14, 2022

ELK Logs Vs F5 - Is there a complete solution anywhere? - If not, can we make one?

Hi All,

This has been bothering me for some time now, I've used f5 for many years now, and its many different modules and log types have always bothered me when looking at external logging. Identifying them, and also understanding how to read them.

But this post is more about identifying them not making comment about it! I'm more interested in getting a common community driven approach into f5 log management and driving it into the elk stack. Mainly because its open source and great for my home lab, but also the release of Elastic Security to the open source branch means many of us can make use of it from home and input that back into our businesses.

 

So what I was hoping for in the first instance is help with identifying people or posts showing work already done which can be reviewed.

For example I've found snippets for ASM and logstash, beats for APM and AFM but from that LTM and DNS seem to be missing. None of these approaches are similar. But all have some brilliant work which just needs pulling together.

With similarity in mind, it would also be good to try to align these output into the elasticseach ecs format so the Security module and other searches can pick up this data easily. There is also many discussions about Logstash or Beats and which one to focus on. My first off thought is to use lgostash as its more open in its development and once we have something working it looks possible to transfer this config into a beats.

 

So any help and advise would be nice from the community please!

 

Thanks - Pete

 

1 Reply

  • Hey Pete
    I actually wrote a logstash parser for a customer of mine a few years ago.
    Too bad I could not share it due to NDAs.

    However, I can share one thing which sprang out of the excercise:
    https://loadbalancing.se/2020/03/11/logstash-testing-tool/

    It's not what you're looking for, but it might help when writing the pipeline. At least it helped me a lot when developing parsers.

    Also wanted to input that part of the reason why this was a bit painful:

    • In order to get synergy from parsing the logs the field names should match those of other sources. Makes it easier to correlate data. I have not found an opinionated database with recommended field names, but I feel that if there is a need for such. 
    • F5 logs are pretty much free text after the log header. It was (somewhat) easy to catch the common things, but I found after a while that there were subtle differences between even the standard log messages (pool member down etc) and I pulled my hair multiple times when trying to figure it out.

    Kind regards,
    Patrik