Load Balancing TCP TLS Encrypted Syslog Messages
Syslog messages sent via TCP are not always evenly distributed among backend syslog servers because multiple syslog messages can be sent in a single TCP connection. This article utilizes the F5 BIG-IP Generic Message Routing Framework (MRF) to evenly distribute syslog messages among backend syslog pool members. This solution also uses TLS to protect the confidentiality of the syslog messages. This article is based off the work done by Mark Lloyd in this DevCentral Technical Article:A Simple One-way Generic MRF Implementation to load balance syslog message. In his article, Mark explains how to setup Generic Message Routing Framework (MRF) to distribute syslog messages sent via TCP to a pool of syslog servers. This article adds the necessary configuration to TLS encrypt, decrypt, and re-encrypt the messages. This was tested on BIG-IP version 17.1.0.1. The first step is to define the message routing protocol. The difference between the default protocol (genericmsg) is the field no-responsemust be configured toyes if this is a one-way stream. Otherwise, the server side will allocate buffers for return traffic that will cause severe free memory depletion. Note: The message-terminator is set to "%0a", this represents a newline character in hex and is used to separate the syslog messages. This value can be changed if a different delineator is required. ltm message-routing generic protocol simple_syslog_protocol { app-service none defaults-from genericmsg description none disable-parser no max-egress-buffer 32768 max-message-size 32768 message-terminator %0a no-response yes } An iRule must be configured on both the Virtual Server and Generic Transport Config. This iRule must be linked as a profile in both the virtual server and the generic transport configuration. ltm rule mrf_simple { when CLIENT_ACCEPTED { GENERICMESSAGE::peer name "[IP::local_addr]:[TCP::local_port]_[IP::remote_addr]:[TCP::remote_port]" } when SERVER_CONNECTED { GENERICMESSAGE::peer name "[IP::local_addr]:[TCP::local_port]_[IP::remote_addr]:[TCP::remote_port]" } } The next item to configure is the generic transport config. The generic transport config has the generic protocol configured along with the iRule to setup the server-side peers. A server-side SSL profile is also configured here to TLS encrypt the traffic to the backend syslog servers. ltm message-routing generic transport-config simple_syslog_tcp_tc { ip-protocol tcp profiles { serverssl-insecure-compatible { } simple_syslog_protocol { } tcp { } } rules { mrf_simple } } Nodes are defined for the backend Syslog servers. ltm node 10.1.20.201 { address 10.1.20.201 } ltm node 10.1.20.202 { address 10.1.20.202 } A pool is created containing the nodes. ltm pool syslog_pool { members { 10.1.20.201:514 { address 10.1.20.201 } 10.1.20.202:514 { address 10.1.20.202 } } } The next step is to create the generic message routing peer. This peer is used to identify the pool of syslog servers that the syslog messages will be routed to. ltm message-routing generic peer simple_syslog_peer { pool syslog_pool transport-config simple_syslog_tcp_tc } Now that the peer is defined, a generic route can be created to send traffic to the peer. ltm message-routing generic route simple_syslog_route { peers { simple_syslog_peer } } A generic router is configured with the generic route. ltm message-routing generic router simple_syslog_router { app-service none defaults-from messagerouter description none ignore-client-port no max-pending-bytes 23768 max-pending-messages 64 mirror disabled mirrored-message-sweeper-interval 1000 routes { simple_syslog_route } traffic-group traffic-group-1 use-local-connection yes } A client-ssl profile is configured to associate the certificates to the Virtual Server. ltm profile client-ssl syslog-ng_client { app-service none cert-key-chain { syslog-ng-new_f5demos_ca_0 { cert syslog-ng-new chain f5demos_ca key syslog-ng-new } } defaults-from clientssl inherit-ca-certkeychain true inherit-certkeychain false } A virtual server is created to receive incoming TLS-encrypted TCP syslog messages. ltm virtual mrftest_simple { creation-time 2024-10-08:09:37:52 destination 10.1.10.101:514 ip-protocol tcp last-modified-time 2024-10-08:13:55:49 mask 255.255.255.255 profiles { simple_syslog_protocol { } simple_syslog_router { } syslog-ng_client { context clientside } tcp { } } rules { mrf_simple } serverssl-use-sni disabled source 0.0.0.0/0 source-address-translation { type automap } translate-address enabled translate-port enabled vs-index 2 } Conclusion This example is from a use case where a single syslog client was sending to a TCP load balancer. The load balancer was not evenly distributing the load among the backend servers because multiple messages were being sent as part of a single TCP connection. This solution utilizes a generic Message Routing Framework to evenly distribute TCP syslog messages. TLS encryption is also used on the client to load balancer connection as well as the load balancer to backend server connection to protect the confidentiality of the syslog messages.
F5 APM Syslog-NG parser
Hello everybody, I use the VPN big-ip Edge client F5 and I would like to generate a log with all theses session variables "session.ldap.last.attr.userPrincipalName + session.check_machinecert.last.cert.subject + session.assigned.clientip". My problem is that the "session.assigned.clientip" isn't populate in the session variable so I can't use a log message to make a custom log with all of theses values. So my question is , is it possible to parse theses logs with syslog-ng and concatenate all the syslog trame with the session ID ? and forward the log concatenated to an another syslog instance ? Don't know if my question is very undertandable ? Regards, Miguel
Need help on syslog cli configuration
Hello team , I have to correct the syslog config with below commands , will there be any impact of running the below commands , Which list commad I can run to perform pre-checks and how to confirm if the configuration working correctly post implementation ? what is the use of both the commands ? tmsh modify sys syslog {include "destination remote_server {tcp(10.10.10.8 port (1528));};filter f_alllogs {level (debug...emerg);};log {source(local);filter(f_alllogs);destination(remote_server);};options {use_fqdn(yes); keep_hostname(yes);};" tmsh modify sys syslog {remote-servers replace-all-with {remotesyslog1 {host 10.1.20.1 local-ip 192.168.101.1 remote-port 5528} remotesyslog2 {host 10.2.20.1 local-ip 192.168.101.1??? remote-port 5528}}}
Can BIG-IQ forward ASM event log which receive from BIG-IP to syslog server?
Hi Right now we have all BIG-IP send ASM event log to BIG-IQ. Question is Can BIG-IQ forward ASM event log which receive from BIG-IP to syslog server? or I need to config on each BIG-IP to send ASM event log to both (BIG-IQ and syslog server) instead.
iRule to log HTTP Request
Hello, I have two F5 Big-IP: 1 * BIG-IP 11.4.1 Build 647.0 Hotfix HF4 1 * BIG-IP 10.2.4 Build 817.0 Hotfix HF7 I want to setup logging for HTTP traffic. On the first F5 (11.4), no problem. I have created a request logging profile with this template: $DATE_NCSA client=$CLIENT_IP:$CLIENT_PORT request=$HTTP_REQUEST virtual-server=$VIRTUAL_NAME($VIRTUAL_IP:$VIRTUAL_PORT) member=$SERVER_IP:$SERVER_PORT On the second F5 (10.2), it is more complicated since the "Request logging profile" does not exist.. And i can't upgrade the device. I decided to create an iRule which produce a log with the same format (HTTP request).. But i don't know how to do.. Especially to generate "$DATE_NCSA" through the irule. Any help would be very appreciated ! 🙂 Thanks PS: please excuse my english 🐵
Log the count of the STREAM hits
I'm trying to figure out how it will be possible, how many times a STREAM::expression is being executed. when HTTP_RESPONSE { if { $http_host equals "avv.com" or $http_host equals "acc.com" }{ STREAM::expression "@aa@bb@" STREAM::expression "@rr@ff" STREAM::expression "@gg@qaqa@" STREAM::enable log local0. "RESPONSE: $http_host to IP: [IP::client_addr]" } } So in the log i want to see like: Total STREAM hit 80 (40aa - 20rr - 20gg)Pool status in Splunk for F5 Networks
Hi all, I made some tests on Splunk with the 11.5.0 TMOS version. My tests were on AFM, LTM and also syslog events. LTM (with the iRule included) and AFM work fine, but for syslog events there's something wrong. If you want to have your pool status statistics on your Splunk, you have to parse syslog events. But those events have changed with the 11.5.0 version so here is the newest regex you'll need. /\]:\s(........:.):\sPool\s(\S+)\smember\s(\S+)\smonitor\sstatus\s(\S+)\.\s?\[?\s?(?:\S+)?\:?\s?(?:\S+)?\s?\]?\s+?\[\swas\s(\S+)\sfor\s(\S+)/ This regex goes to /opt/splunk/etc/apps/SplunkforF5Networks/default/transforms.conf under [f5-syslog-eventcode] I still have something missing into my Splunk configuration because I don't have all my pool status. If anybody has already play with it, could you tell me where I'm wrong ? Thanks.
BIG-IP send all syslog messages as local5 facility
Hello Devs! Our client wants every syslog message sent by the BIG-IP to be on local5 facility. I understand that this is not the behavior of syslog-ng. But is it possible? I tinkered around the syslog options and could not find this option, not even on the CLI (we are running v15.1.0.5). They're trying to emulate a Netscaler config add audit syslogAction AUDIT_SRV_SYSLOG 1.2.3.4 -logLevel EMERGENCY ALERT CRITICAL ERROR WARNING NOTICE -dateFormat DDMMYYYY -logFacility LOCAL5 -timeZone LOCAL_TIME This config sends all the messages to syslog server 1.2.3.4 as local5. Thanks, Rafael.
