Forwarding Logs to SIEM Tools via HTTP Proxy for F5 Distributed Cloud Global Log Receiver
Purpose This guide provides a solution for forwarding logs to SIEM tools that support syslog but lack HTTP/HTTPS ingestion capabilities. It covers the deployment and tuning of an HTTP Proxy log receiver configured to work with F5 Distributed Cloud (XC) Global Log Receiver settings. Audience: This guide is intended for technical professionals, including SecOps teams and Solution Architects, who are responsible for integrating SIEM tools with F5 XC Global Log Receiver. Readers should have a solid understanding of HTTP communication (methods, request body, reverse proxy), syslog, and data center network architecture. Familiarity with F5 XC concepts such as namespaces, log types, events, and XC-GLR is also required. Introduction: Problem Statement: SIEM tools often support syslog ingestion but lack HTTP/HTTPS log reception capabilities. Objective: Explain how to deploy and configure an HTTP Proxy to forward logs to F5 Distributed Cloud Global Log Receiver. Solution Overview: Architecture Diagram and workflow: Configuration Steps: Configure Global Log Receiver in F5 Distributed Cloud Console Navigate to: Home → Shared Configuration → Global Log Receiver Create or edit the Global Log Receiver settings for HTTP receiver Ensure the Global Log Receiver batch size is based on the payload size expected from F5 NGINX. Example configuration snap: Set Up NGINX as an HTTP Log Receiver Install NGINX on your designated server. Configure log_format Configure NGINX to accept HTTP POST requests only and forward access logs to syslog Example configuration snippet: log_format custom_log_format_1 escape=json $request_body; # Example: include request body only server { listen 443 ssl; server_name <logreceiver_server_name>; ssl_certificate /etc/ssl/<logreceiver_server_cert>; ssl_certificate_key /etc/ssl/<logreceiver_server_key>; # Other SSL/TLS configurations (e.g., protocols, ciphers) ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; client_body_in_single_buffer on; # The directive is recommended when using the $request_body variable, to save the number of copy operations involved client_body_in_file_only off; #default client_max_body_size 32M; # based on tuning gzip on; location /log_endpoint { # Allow only POST requests for sending log data limit_except POST { deny all; } # Configure access_log to write incoming data to a file # access_log /var/log/nginx/log_receiver.log custom_log_format_1; access_log syslog:server=127.0.0.1:514,facility=local7,tag=nginx,severity=info custom_log_format_1; proxy_pass http://localhost:8091/; # This dummy Internal server required to collect request_body variable. } } # dummy internal server to respond back 200 ok server { listen 8091; server_name localhost; location / { return 200 "Log received successfully."; } } Set Up rsyslog server Install/configure rsyslog on your designated server. Configure 60-nginx.conf file in /etc/rsyslog.d/ directory Sample 60-nginx.conf file #nginx.* @@127.0.0.1:514 :syslogtag, isequal, "[nginx]" /var/log/nginx-syslog/nginx-access-log.log References: F5 Distributed Cloud Global log receiver supports many log receivers natively: F5 Distributed Cloud Technical Knowledge page on "Configure Global Log receiver" Prerequisites: An external log collection system reachable publicly. The following IP address ranges are required to be added to your firewall's allowlist: 193.16.236.64/29 185.160.8.152/29
BIG-IP SysLog appearing in ossec.log
F5 BigIP Syslog Integration: Logs Appearing in ossec.log instead of archives.log with Size Limitation Error Environment Wazuh server F5 BigIP Current Setup F5 BigIP is configured to send logs via syslog since running a Wazuh agent is not possible (BigIP standard practices restrict installing new packages). Issue Description When sending logs from F5 BigIP to Wazuh using syslog: Logs are appearing in ossec.log instead of archives.log as specified in the official documentation The logs are being received in hexadecimal format The logs appear to be incomplete with an error message indicating "to big size above" Current Configurations Wazuh Configuration F5 BigIP Syslog Configuration Expected Behavior Logs should be written to archives.log Logs should be complete and properly decoded No size limitation errors should occur Actual Behavior Logs are being written to ossec.log Logs are in hexadecimal format Receiving error: "to big size above" Logs are incomplete Troubleshooting Steps Attempted Network Connectivity Verification: Performed tcpdump analysis - confirmed packets are being transmitted correctly No network-level issues identified Wazuh Configuration Adjustments: Modified client_buffer settings - no impact on the issue Tested multiple port configurations - issue persists Port Testing: Attempted communication through different ports Issue remained consistent across all port configurations Additional Attempts: Exhausted various other configuration combinations No successful resolution achieved through standard troubleshooting methods Debug Information Decoded Hex Log Sample Additional Notes The incomplete hex format suggests potential issues with message size limitations or parsing Willing to provide additional information or troubleshooting details through a call if needed Questions Is this a known issue with F5 BigIP syslog integration? Are there specific size limitations that need to be configured? Is there a configuration parameter that needs to be modified to direct logs to archives.log?Load Balancing TCP TLS Encrypted Syslog Messages
Syslog messages sent via TCP are not always evenly distributed among backend syslog servers because multiple syslog messages can be sent in a single TCP connection. This article utilizes the F5 BIG-IP Generic Message Routing Framework (MRF) to evenly distribute syslog messages among backend syslog pool members. This solution also uses TLS to protect the confidentiality of the syslog messages. This article is based off the work done by Mark Lloyd in this DevCentral Technical Article: A Simple One-way Generic MRF Implementation to load balance syslog message. In his article, Mark explains how to setup Generic Message Routing Framework (MRF) to distribute syslog messages sent via TCP to a pool of syslog servers. This article adds the necessary configuration to TLS encrypt, decrypt, and re-encrypt the messages. This was tested on BIG-IP version 17.1.0.1. The first step is to define the message routing protocol. The difference between the default protocol (genericmsg) is the field no-response must be configured to yes if this is a one-way stream. Otherwise, the server side will allocate buffers for return traffic that will cause severe free memory depletion. Note: The message-terminator is set to "%0a", this represents a newline character in hex and is used to separate the syslog messages. This value can be changed if a different delineator is required. ltm message-routing generic protocol simple_syslog_protocol { app-service none defaults-from genericmsg description none disable-parser no max-egress-buffer 32768 max-message-size 32768 message-terminator %0a no-response yes } An iRule must be configured on both the Virtual Server and Generic Transport Config. This iRule must be linked as a profile in both the virtual server and the generic transport configuration. ltm rule mrf_simple { when CLIENT_ACCEPTED { GENERICMESSAGE::peer name "[IP::local_addr]:[TCP::local_port]_[IP::remote_addr]:[TCP::remote_port]" } when SERVER_CONNECTED { GENERICMESSAGE::peer name "[IP::local_addr]:[TCP::local_port]_[IP::remote_addr]:[TCP::remote_port]" } } The next item to configure is the generic transport config. The generic transport config has the generic protocol configured along with the iRule to setup the server-side peers. A server-side SSL profile is also configured here to TLS encrypt the traffic to the backend syslog servers. ltm message-routing generic transport-config simple_syslog_tcp_tc { ip-protocol tcp profiles { serverssl-insecure-compatible { } simple_syslog_protocol { } tcp { } } rules { mrf_simple } } Nodes are defined for the backend Syslog servers. ltm node 10.1.20.201 { address 10.1.20.201 } ltm node 10.1.20.202 { address 10.1.20.202 } A pool is created containing the nodes. ltm pool syslog_pool { members { 10.1.20.201:514 { address 10.1.20.201 } 10.1.20.202:514 { address 10.1.20.202 } } } The next step is to create the generic message routing peer. This peer is used to identify the pool of syslog servers that the syslog messages will be routed to. ltm message-routing generic peer simple_syslog_peer { pool syslog_pool transport-config simple_syslog_tcp_tc } Now that the peer is defined, a generic route can be created to send traffic to the peer. ltm message-routing generic route simple_syslog_route { peers { simple_syslog_peer } } A generic router is configured with the generic route. ltm message-routing generic router simple_syslog_router { app-service none defaults-from messagerouter description none ignore-client-port no max-pending-bytes 23768 max-pending-messages 64 mirror disabled mirrored-message-sweeper-interval 1000 routes { simple_syslog_route } traffic-group traffic-group-1 use-local-connection yes } A client-ssl profile is configured to associate the certificates to the Virtual Server. ltm profile client-ssl syslog-ng_client { app-service none cert-key-chain { syslog-ng-new_f5demos_ca_0 { cert syslog-ng-new chain f5demos_ca key syslog-ng-new } } defaults-from clientssl inherit-ca-certkeychain true inherit-certkeychain false } A virtual server is created to receive incoming TLS-encrypted TCP syslog messages. ltm virtual mrftest_simple { creation-time 2024-10-08:09:37:52 destination 10.1.10.101:514 ip-protocol tcp last-modified-time 2024-10-08:13:55:49 mask 255.255.255.255 profiles { simple_syslog_protocol { } simple_syslog_router { } syslog-ng_client { context clientside } tcp { } } rules { mrf_simple } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled vs-index 2 } Conclusion This example is from a use case where a single syslog client was sending to a TCP load balancer. The load balancer was not evenly distributing the load among the backend servers because multiple messages were being sent as part of a single TCP connection. This solution utilizes a generic Message Routing Framework to evenly distribute TCP syslog messages. TLS encryption is also used on the client to load balancer connection as well as the load balancer to backend server connection to protect the confidentiality of the syslog messages.
F5 APM Syslog-NG parser
Hello everybody, I use the VPN big-ip Edge client F5 and I would like to generate a log with all theses session variables "session.ldap.last.attr.userPrincipalName + session.check_machinecert.last.cert.subject + session.assigned.clientip". My problem is that the "session.assigned.clientip" isn't populate in the session variable so I can't use a log message to make a custom log with all of theses values. So my question is , is it possible to parse theses logs with syslog-ng and concatenate all the syslog trame with the session ID ? and forward the log concatenated to an another syslog instance ? Don't know if my question is very undertandable ? Regards, Miguel
Need help on syslog cli configuration
Hello team , I have to correct the syslog config with below commands , will there be any impact of running the below commands , Which list commad I can run to perform pre-checks and how to confirm if the configuration working correctly post implementation ? what is the use of both the commands ? tmsh modify sys syslog {include "destination remote_server {tcp(10.10.10.8 port (1528));};filter f_alllogs {level (debug...emerg);};log {source(local);filter(f_alllogs);destination(remote_server);};options {use_fqdn(yes); keep_hostname(yes);};" tmsh modify sys syslog {remote-servers replace-all-with {remotesyslog1 {host 10.1.20.1 local-ip 192.168.101.1 remote-port 5528} remotesyslog2 {host 10.2.20.1 local-ip 192.168.101.1??? remote-port 5528}}}
Can BIG-IQ forward ASM event log which receive from BIG-IP to syslog server?
Hi Right now we have all BIG-IP send ASM event log to BIG-IQ. Question is Can BIG-IQ forward ASM event log which receive from BIG-IP to syslog server? or I need to config on each BIG-IP to send ASM event log to both (BIG-IQ and syslog server) instead.
iRule to log HTTP Request
Hello, I have two F5 Big-IP: 1 * BIG-IP 11.4.1 Build 647.0 Hotfix HF4 1 * BIG-IP 10.2.4 Build 817.0 Hotfix HF7 I want to setup logging for HTTP traffic. On the first F5 (11.4), no problem. I have created a request logging profile with this template: $DATE_NCSA client=$CLIENT_IP:$CLIENT_PORT request=$HTTP_REQUEST virtual-server=$VIRTUAL_NAME($VIRTUAL_IP:$VIRTUAL_PORT) member=$SERVER_IP:$SERVER_PORT On the second F5 (10.2), it is more complicated since the "Request logging profile" does not exist.. And i can't upgrade the device. I decided to create an iRule which produce a log with the same format (HTTP request).. But i don't know how to do.. Especially to generate "$DATE_NCSA" through the irule. Any help would be very appreciated ! 🙂 Thanks PS: please excuse my english 🐵
