Adding the body of requests/responses to the data being logged to Splunk via iRule.
Hi All, We are presently using the iRule below to log request / response data to splunk. I'd like to add the body of the requests to our splunk logging. I had tried to user HTTP::payload as part of HTTP_REQUEST however it seems that the irule no longer functions when I place this there. When I add HTTP_REQUEST_DATA to the iRule to cater for HTTP:payload, I break the app - I expect that this is my implementation of HTTP_REQUEST_DATA. Is there an easy way to add the logging of the body of the request and response to what is sent to splunk? Thanks in advance when CLIENT_ACCEPTED { set client_address [IP::client_addr] set vip [IP::local_addr] } when HTTP_REQUEST { set http_host [HTTP::host]:[TCP::local_port] set http_uri [HTTP::uri] set http_url $http_host$http_uri set http_method [HTTP::method] set http_version [HTTP::version] set http_user_agent [HTTP::header "User-Agent"] set http_content_type [HTTP::header "Content-Type"] set http_referrer [HTTP::header "Referer"] set tcp_start_time [clock clicks -milliseconds] set req_start_time [clock format [clock seconds] -format "%Y/%m/%d %H:%M:%S"] set cookie [HTTP::cookie names] set user [HTTP::username] set virtual_server [LB::server] if { [HTTP::header Content-Length] > 0 } then { set req_length [HTTP::header "Content-Length"] } else { set req_length 0 } } when HTTP_RESPONSE { set res_start_time [clock format [clock seconds] -format "%Y/%m/%d %H:%M:%S"] set node [IP::server_addr] set node_port [TCP::server_port] set http_status [HTTP::status] set req_elapsed_time [expr {[clock clicks -milliseconds] - $tcp_start_time}] if { [HTTP::header Content-Length] > 0 } then { set res_length [HTTP::header "Content-Length"] } else { set res_length 0 } set hsl [HSL::open -proto TCP -pool p-remote-logging] HSL::send $hsl "<190>,f5_irule=Splunk-iRule-HTTP,src_ip=$client_address,vip=$vip,http_method=$http_method,http_host=$http_host,http_uri=$http_uri,http_url=$http_url,http_version=$http_version,http_user_agent=\"$http_user_agent\",http_content_type=$http_content_type,http_referrer=\"$http_referrer\",req_start_time=$req_start_time,cookie=\"$cookie\",user=$user,virtual_server=\"$virtual_server\",bytes_in=$req_length,res_start_time=$res_start_time,node=$node,node_port=$node_port,http_status=$http_status,req_elapsed_time=$req_elapsed_time,bytes_out=$res_length\r\n" } when LB_FAILED { set hsl [HSL::open -proto TCP -pool p-remote-logging] HSL::send $hsl "<190>,f5_irule=Splunk-iRule-LB_FAILED,src_ip=$client_address,vip=$vip,http_method=$http_method,http_host=$http_host,http_uri=$http_uri,http_url=$http_url,http_version=$http_version,http_user_agent=\"$http_user_agent\",http_content_type=$http_content_type,http_referrer=\"$http_referrer\",req_start_time=$req_start_time,cookie=\"$cookie\",user=$user,virtual_server=\"$virtual_server\",bytes_in=$req_length\r\n" }Splunk Log Publisher with iRule HSL
Hi, I'm having some difficulty understanding the relationship between the HSL iRule commands and formatted log publishers (Splunk, in this case) in 11.5.1. Sorry if the formatting isn't what the community expects - I'm fairly new to this ecosystem. Setup One node called "splunk-universal-forwarder" running a Splunk universal forwarder listening on 9996/tcp. A pool called "remote-logging-pool" with the splunk-universal-forwarder node listening on 9996/tcp. Log Destination "splunk-rhsl" of type Remote HSL pointed at the remote-logging-pool over TCP. Log Destination "splunk-formatted" of type "Splunk" forwarding to "splunk-rhsl" Log Publisher "splunk-publisher" pointing at the splunk-formatted destination. Log Publisher "rhsl-publisher" pointing at the splunk-rhsl destination. A virtual server backed by a pool with a single node serving up content http://10.1.1.2 (the IP address is irrelevant for this question). Given the iRule: when CLIENT_ACCEPTED { set hsl [HSL::open -publisher /tst/splunk-publisher] set hsl [HSL::open -proto TCP -pool remote-logging-pool] set hsl [HSL::open -publisher /tst/Rhsl-publisher] } when HTTP_REQUEST { HSL::send $hsl "<190>|[IP::local_addr]|[HTTP::uri]\n" } The bottom two set statements in the when CLIENT_ACCEPTED command result in a messaging going successfully to the Splunk forwarder. The first set statement results in no messages going to the Splunk forwarder. Questions Are HSL commands in iRules unable to use formatted publishers? What exactly does a formatted publisher do? It seems like it would write data to the remote receiver in a format that particular receiver expects, but I haven't found a lot of documentation on this yet. Is there an interactive way to generate dummy log events to send to different destinations and publishers to see what the effect is on remote receiver? Thanks for your help!APM integration with splunk
Hey, i configured the free version of splunk and managed to get the \var\log\apm files, i can see the logs but the built-in dashboard for apm isnt showing data (except one chart) i think its because we run version 13 and this template of splunk was built for 11.6 is anyone tried to modify this template ? or found other creative solution ? i have to make this thing work!
ASM Logging to Splunk Anomoly
Hello, The ASM logs we're sending to Splunk have random (Splunk assigned?) field names. For example, violation_rating is named cn2 in Splunk, attack_type shows up as cs4, user_agent is called pm_fpua in Splunk, and so on. Does anyone know if this is a Splunk issue or a logging profile issue? The profile I inherited was configured with a logging format of Common Event Format (ArcSight) although we're talking to Splunk. I assumed changing it back to Key-Value Pairs (Splunk) might fix the issue but it's still jacked up. Thanks, ToneGTM to Splunk
Hi Everyone, We have a requirement where in we need to send GTM Logs/Statistics to our Splunk logging server. May we know what parameters can be pulled from the GTM Logs/Statistics that we can send to the Splunk? An example is which WIP uses the most bandwidth or the trend of the ISP Bandwidth usage to determine the time/day the peak is reached. our F5 DNS is running on 13.1.0.4 software version if possible, how can we implement this in our GTM and Splunk? Hope someone can give us an article. Thank you.
F5 Splunk logging issue
Hi Team, We have a requirement where all of the F5s in our environment need to send logs to the Splunk VIP configured on a different F5. I want to use F5 Analytics iApp to simplify the deployment. The Splunk VIP (on a different F5) has 2 Splunk servers as the Pool members. When we are creating "HTTP Event Collector" on Spunk, it generates a Token which I need to use in the iApp. However, the issue is each Splunk server is generating a separate Token (so total 2 tokens) whereas the iApp is accepting only one token. How can we resolve this issue? Thanks in advance!! Regards Vijay
F5 Analytics 3.7.x - Questions and Problems
Hello Community, I stumbeld upon the F5 Analytics iApp and so I wanted to give it a try. My environmental requirement: Splunk Version 6.5 Communication from BigIP to Splunk HTTP Event Collector is working (telnet to Splunk port 8088) Syslog/Analytics data MUST use Management interface of BigIP no HSL 1st try: iApp Version 3.7.1 (and also 3.7.2rc4)* BIG-IP v12.1.2 (Build 2.0.276) and BIG-IP v13.1.1.2 (Build 0.0.4) Result: failed* Error in /var/log/ltm: Script (/Common/F5-Analytics-send_stats) generated this Tcl error: (script did not successfully complete: (01020036:3: The requested RADIUS Server (/Common/F5-Analytics.app) was not found. while executing "tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth" invoked from within "lindex [tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth] 0" invoked from within "set obj [lindex [tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth] 0]" line:41)) 2nd try: iApp Version 3.7.0 BIG-IP v12.1.2 (Build 2.0.276) Result: not quite sure Some Error in /var/log/ltm: Stats Response for F5-Analytics 1542956978 0 fail Data publisher not found or not implemented when processing request (unknown request), tag (2901). Thus I seem to get Data but I don't know if this is correct. Health values on "Start" Dashboard are always 0 Device Health 0 as well, or even NaN, Most field just have no data SSL Dashboard: no data Question I still have Do I have to configure Analytic Profiles on the BigIP itself to get more data? when you can use only the Mgmt-Port, what are the best settings in the iApp? Can i user "Module High Speed Logging Streams -> yes" when I can only use the Mgmt-Port? Is there a guide, that shows how to setup the BigIP itself (i.e. analytic profiles on VS) to get the most out of the app? I followed the "Deploying the F5 Analytics iApp Template" but the result is not overwhelming Is there an explanation for the meaning of all the diagramms and stuff you get? Example: What does "Server Latency Health 100,0%" mean? I'm grateful for any hint, tip, advice from you guys. Cheers Ichnafi
Pool status in Splunk for F5 Networks
Hi all, I made some tests on Splunk with the 11.5.0 TMOS version. My tests were on AFM, LTM and also syslog events. LTM (with the iRule included) and AFM work fine, but for syslog events there's something wrong. If you want to have your pool status statistics on your Splunk, you have to parse syslog events. But those events have changed with the 11.5.0 version so here is the newest regex you'll need. /\]:\s(........:.):\sPool\s(\S+)\smember\s(\S+)\smonitor\sstatus\s(\S+)\.\s?\[?\s?(?:\S+)?\:?\s?(?:\S+)?\s?\]?\s+?\[\swas\s(\S+)\sfor\s(\S+)/ This regex goes to /opt/splunk/etc/apps/SplunkforF5Networks/default/transforms.conf under [f5-syslog-eventcode] I still have something missing into my Splunk configuration because I don't have all my pool status. If anybody has already play with it, could you tell me where I'm wrong ? Thanks.Splunk "sticky" persistence issue - random disconnects
I'm load balancing a pool of three Splunk servers using "cookie" as my primary persistence, and "source_addr" as my fallback, however I've received complaints of random disconnects from users. Might this be a persistence issue? Has anyone else experienced this? The F5's running BIG-IP 12.1.3.6 Build 0.0.3. Thanks!