Log message code list?
SOL16197: Reviewing BIG-IP log files describes local traffic log message format as: Message code is split into: Message code: The code that is associated with the message. The code is comprised of the following sub-codes: Product Code: The first two hex digits form the product code. For example, 0x01 is the BIG-IP product code. Subset Code: The third and fourth hex digits are the subset code. For example, 0x2a is the subset code for LIBHAL. Message Number: The next four digits form the message number within a module. Severity Level: The last digit between the colon symbols is the severity level, with 0 being the highest severity level. Are the Product and Subset codes listed anywhere? Would help in processing log messages further in Splunk or similar tool.
Logging Configuration in LTM HA
Hello everyone I'm doing a logging lab and I'm asking for your help to understand some things. I have two BIG-IP LTM in HA and a Qradar logging server. I have configured the Qradar as syslog servers at each HA node. At the Qradar level I receive the active logs but the standby only sends errors like "BIGIP_TMM_TIMMERR_PMBR_BACK_UP. I'd like to know if it's normal for the standby to send only error messages, and in general I'd like to understand how logging works in HA and what type of event each device sends to the server. thanks in advanceF5 APM Webtop - RDP Session Logging
Hello F5 Experts, I am relatively new to the F5 advanced ecosystem, am trying to generate useful logs from our APM Webtop environment and am hoping that someone can point me in the correct direction. I am trying to log the following things from our environment: Initial login's to the Webtop including ClientIP, Webtop portal address, Browser UserAgent, Client Username. (Optional) Client group membership/published resources when they log into Webtop. When a client opens a Web Portal Access from within Webtop including,ClientIP, Webtop portal address, Browser UserAgent, Client Username, Web Portal Access Address. (It would be good to get their session duration for this but that might not be fesable). When a client opens a RDP link from within the Webtopincluding,ClientIP, Webtop portal address, Browser UserAgent, Client Username, RDP Address, SessionCookie(?). When a client connects to a RDP session though the Webtop using one of the downloaded links,ClientIP, Webtop portal address, Client Username, RDP Address, SessionCookie(?), Session start and end time (Maybe two different log events?). From what I can tell this is likely to be an iRule. I think I have an idea how to do the Webtop portal logging, but what is really eluding me is how to log the RDP session connection and duration. Any help or a direction where to look would be greatly appreciated. Thank you,APM - How to configure logging of snat addresses for network access and app tunnels
Hello everyone, we are using BIG-IP Access Policy Manager to enable administrative access to systems via App Tunnel and Network Access resources. For security reasons, we need to be able to map requests logged on backend resources/systems (e.g. in SSH audit logs) to the session or user accessing said backend resource via App Tunnel or Network Access in APM. Currently, the following request information is logged. Network Access: May 17 14:42:00 tmm0 tmm[22565]: 01580002:5: /APM/ap_rmgw:Common:c1237463: allow ACL: #app_tunnel_/APM/Some_App-Tunnel@c1237463:15 packet: tcp 192.168.12.18:58680 -> 10.0.0.1:22 App Tunnels: May 17 14:41:10 tmm1 tmm1[22565]: 01580002:5: /APM/ap_rmgw:Common:c6787463: allow ACL: #app_tunnel_/APM/Some_App-Tunnel@c6787463:0 packet: tcp 89.229.152.144:63252 -> 10.0.0.1:2 For Network Access requests, an IP address of the lease pool configured in the Network Access resource is logged as the client IP. For App Tunnel requests, the public IP of the client accessing APM is logged as the client IP. In our setup, both requests will be NATed by APM before hitting the target system (through a snat pool in case of a Network Access request, through the active appliances backend IP in case of App Tunnels). Therefore, the APM self IPs (snat pool/appliance backend) will be logged on the target host, leading to us not being able to correlate logs in APM with logs on the target systems. Is there any way to log the SNAT/NAT addresses and ports used to access target systems through APM? I've tried using ACCESS_ACL_ALLOWED in an iRule to log additional information, unfortunately this event only seems to trigger on Portal Access resources, not when using App Tunnels or Network Access resources. Thank you, Fabian
Most efficient methods for Connection logging?
Does anyone have real world experience with logging connections at a high rate? If so, which methods are you using to collect and transmit the data? We have a requirement to log all connections going through our F5 devices. Things like the client/server-side IPs/ports as well as HTTP details for HTTP VIPs and DNS details from our GTMs. It's the Whitehouse M-21-31 mandate if anyone if familiar with it. I've used Request Logging Profiles and various iRules with HSL to collect this type of data before, but I've never been too concerned about overhead because I would only apply them as needed, like when t-shooting an issue with a VIP. Our busiest appliance pushes around 150k conn/sec and 5k HTTP req/sec, so I now have consider the most efficient methods to avoid any kind of impact to traffic flows. I've done some lab testing with several different methods but I can't do any meaningful load tests in that environment.Below are some of my opinions based on my lab testing so far. Data Collection AVR - I like that this single feature can meet all the requirements for collecting TCP, HTTP, and DNS data. It would also be relatively easy to perform audits to ensure the VIPs have the necessary Analytics profiles as we can manage it from the AVR profiles themselves. My main concern is the overhead that results from the traffic analysis. I assume it has to maintain a large database where it stores all the analyzed data even if we just ship it off to Splunk. Even the data shipped off to Splunk includes several different logs for each connection (each with a different 'Entity'). Request Logging Profile- This is fairly flexible and should have low overhead since the F5 doesn't need to analyze any of the data like AVR does. This only collects HTTP data so we still need another solution to collect details for non HTTP VIPs. It would be a pain to audit since we don't have use any kind of deployment templates or automation. iRule - This provides a lot of flexibility and it is capable of collecting all the necessary data, but I don't know how well performance overhead compares to AVR. This would also be a pain to audit due to lack of deployment templates and automation. Data Transmission HSL UDP Syslog- I imagine this is the most efficient method to send events, but it's likely only a matter of time before we are required to use TCP/TLS. Telemetry Streaming - This is the more modern method and it offers some interesting features like System Poller, which could eventually allow us to move away from SNMP polling. We would need a workaround for our GTM-only devices because they cannot run a TS listener.
Attaching w3c iRule to VS
Hi, I'm in midst of preparation to attach w3c iRule to all the VS in my internet facing BIGIP. I would be attaching to all the VS with http profile. But just wondering, if there would be any impact on the configuration (or change in Properties of VS) when I attach the iRule in question to the VS. Please confirm. Thanks, MSK
Logging SSL VPN Client Outbound Traffic
Hi all, I've searched around and found a few bits mentioned regarding the use of wildcard forwarding proxies and related rules but can't seem to find a definitive answer. We have an SSLVPN that we'd like some enhanced logging enabled on for security compliance. I've managed to get the majority of this working using the following iRule, however I'm missing one vital piece of information, the true destination of the traffic. when HTTP_REQUEST { set remote [IP::remote_addr]:[TCP::remote_port] set vip [IP::local_addr]:[TCP::local_port] set user [ACCESS::session data get session.logon.last.username] set session [HTTP::cookie value LastMRH_Session] set clientip [ACCESS::session data get session.user.clientip] set IntIP [ACCESS::session data get session.assigned.clientip] set url [HTTP::header Host][HTTP::uri] log "Rule TCP_logging fired, from $remote to vip $vip, user $user, session $session, client IP $clientip, InternalIP $IntIP, url $url" } This gives me a nice log entry with source and internal IP, username, session ID which is great! I've tried using the [IP::server_addr] value however that just returns an error (I believe because it's not actually load balancing), I've also tried the various [HTTP] variables however they just return the URL of the VIP itself not the destination traffic. All I want to see is if a user connected to the VPN hits a URL that this is recorded in the logs alongside the information I've collected above. I would appreciate any help possible! Kind Regards SpencerLogs have pool member ip as client ip. WHY?
I have one virtual server which is load balancing and have an ip of 10.50.171.5(external vlan). Pool members are 10.50.169.14, 10.50.169.16. Now the problem is that when i attach iRule for logging on remote syslog server with VS. when CLIENT_ACCEPTED { log 10.50.242.239 local0.info "Client Connected, IP: [IP::client_addr]" } I start to receive logs on remote syslog server with about 20 logs per second and all logs have the pool member ip as client ip. WHY? Although there is not traffic yet from outside and not one is dialing 10.50.171.5. It inline configuration and VLAN GROUP is enable. Please help
HTTP Payload logging breaks HTTP Keep-Alive
Afternoon all, I've written an iRule to record the request/response payload on a REST HTTP API. The rule looks like: when CLIENT_ACCEPTED { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "Processing CLIENT_ACCEPTED." } Set the payload logging flag set log_payload 1 } when HTTP_REQUEST priority 800 { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_REQUEST at priority 800..." } Skip logging if no members available if {$splunk_bypass}{ if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Splunk HSL pool is down. Bypassing logging..." } return } Don't allow data to be chunked if { [HTTP::version] eq "1.1" } { if { [HTTP::header is_keepalive] } { HTTP::header replace "Connection" "Keep-Alive" } if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Switching to HTTP Version 1.0." } HTTP::version "1.0" } Split out request headers and munge into string set headers [HTTP::header names] if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Request HTTP Headers = $headers" } set request_headers "'" foreach header $headers { set value [HTTP::header value $header] if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Request HTTP Header $header value = $value" } append request_headers "$header=$value " } set request_headers [string trimright $request_headers " "] append request_headers "'" } when HTTP_REQUEST_DATA { if { $static::PayloadLoggerDebug or $f5_connection_debug }{ log local0.debug "$log_prefix: Collected [HTTP::payload length] bytes."} set request_payload [HTTP::payload] } when HTTP_RESPONSE priority 50 { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_RESPONSE at priority 50..." } Skip logging if no members available if {$splunk_bypass}{ if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Splunk HSL pool is down. Bypassing logging..." } return } Split out the response headers and munge into string set headers [HTTP::header names] if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Response HTTP Headers = $headers" } set response_headers "'" foreach header $headers { set value [HTTP::header value $header] if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Response HTTP Header $header value = $value" } append response_headers "$header=$value " } set response_headers [string trimright $response_headers " "] append response_headers "'" Collect the response if { $response_length > 0 } { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Collecting $response_length bytes from response." } HTTP::collect $response_length } Calculate actual content-length set response_length_real [HTTP::payload length] if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Real response content-length = $response_length_real." } Correct the response_length to correct value if required. if { $response_length != $response_length_real } { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Updated \$response_length value." } set response_length $response_length_real if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: New \$response_length value is $response_length." } } } when HTTP_RESPONSE_DATA { if { $static::PayloadLoggerDebug or $f5_connection_debug } { log local0.debug "$log_prefix: Processing HTTP_RESPONSE_DATA." } Gather response data set response_payload "[HTTP::payload]" } A lot of the above feeds into a larger iRule framework. E.g. we use a SplunkHTTPS iRule to actually do the HSL logging out to a syslog server. The 'HTTP::collect' is also called from that iRule aswell. However when testing this iRule, I've identified an issue whereby it appears to be breaking HTTP Keep-alive connections to some of our GF3 application servers. Removing this rule restores the keep-alive functionality. Any pointers on how I can maintain the keep-alive functionality and also be able to log the request/response data? Cheers Gavin