Integrate F5 Distributed Cloud remote logging with ELK

Scope

This technical article is useful for F5 XC users familiar with HTTP Load Balancer with WAF security and the implementation and use of the ELK Stack.

The focus is for F5 XC security logs integration with ELK exclusively.

Introduction

F5 Distributed Cloud (XC) WAAP provides SaaS based services which consist of Web Application Firewall (WAF), API Security, Bot Defense, DDos Mitigation. The platform also provides centralized dashboard for management, monitoring service visibility and observability.

Even F5 XC platform has native dashboard for monitoring, many customers prefer to consolidate log/telemetry from various systems/platforms (including F5 XC) to their centralized SIEM system e.g., Splunk, QRadar, ELK, etc. For this reason, Global Log Receiver on F5 XC platform was released to fulfil the need.

The following is example use cases which make customer prefer to export F5 XC log to their centralized SIEM

• More control of log retention and data analytic (F5 XC has 30 days log retention included in based package)
• To comply with company security policy for log handling (keep it on-prem instead of on-cloud)

F5 XC and ELK connection diagram

 

ELK Deployment and HTTP Receiver configuration

Note: ELK deployment should be separated on each component (Elasticsearch, Logstash, Kibana) and apply HA configuration for resiliency if deployed on production environment. In this tutorial I just demonstrate how to integrate F5 XC with ELK so ELK on Docker was chosen for easy deployment.

Step:

• Type

  git clone https://github.com/deviantony/docker-elk.git ​

  to clone ELK stack files from github project
•  Modify ./docker-elk/docker-compose.yml (by adding http receiver port 8080 under logstash section as shown below screenshot)

• Modify ./docker-elk/logstash/pipeline/logstash.conf (by adding the following entry in red box)

• Type

  sudo docker-compose up -d ​

  to start ELK stack in background
• Type

  sudo docker ps​

   to check status of ELK stack containers

• Once the ELK stack already up and running, then you can access to ELK GUI http://<ELK IP>:5601 using default username/password (elastic/changeme)

F5 XC Global Log Receiver configuration

Step:

• On XC console, navigate to Shared Configuration > Manage > Global Log Receiver > Add Global Log Receiver
• Apply the following configuration, once it done, click “Save and Exit”

Verify XC log populated on ELK

Step:

• Generate attack traffic to the HTTP Load Balancer FQDN which is protected by F5 XC WAAP
• On ELK GUI, navigate to Analytics > Discover and choose “logs-*” as data view. You will be able to see XC security log.

Create ELK Dashboard using parsed XC log

Step:

• After XC log has been parsed, then you can create ELK dashboard using XC log data.
• ELK Dashboard can be accessed on http://<ELK IP>:5601
• The screenshot below shown ELK dashboard which contains the number of security events, attack percentage of each type of attack, demographic of origin country of attacker, URI path which was attacked.

Additional Links

F5 Distributed Cloud (XC) Services

F5 XC Global Log Receiver