Splunk Log Publisher with iRule HSL
Hi, I'm having some difficulty understanding the relationship between the HSL iRule commands and formatted log publishers (Splunk, in this case) in 11.5.1. Sorry if the formatting isn't what the community expects - I'm fairly new to this ecosystem. Setup One node called "splunk-universal-forwarder" running a Splunk universal forwarder listening on 9996/tcp. A pool called "remote-logging-pool" with the splunk-universal-forwarder node listening on 9996/tcp. Log Destination "splunk-rhsl" of type Remote HSL pointed at the remote-logging-pool over TCP. Log Destination "splunk-formatted" of type "Splunk" forwarding to "splunk-rhsl" Log Publisher "splunk-publisher" pointing at the splunk-formatted destination. Log Publisher "rhsl-publisher" pointing at the splunk-rhsl destination. A virtual server backed by a pool with a single node serving up content http://10.1.1.2 (the IP address is irrelevant for this question). Given the iRule: when CLIENT_ACCEPTED { set hsl [HSL::open -publisher /tst/splunk-publisher] set hsl [HSL::open -proto TCP -pool remote-logging-pool] set hsl [HSL::open -publisher /tst/Rhsl-publisher] } when HTTP_REQUEST { HSL::send $hsl "<190>|[IP::local_addr]|[HTTP::uri]\n" } The bottom two set statements in the when CLIENT_ACCEPTED command result in a messaging going successfully to the Splunk forwarder. The first set statement results in no messages going to the Splunk forwarder. Questions Are HSL commands in iRules unable to use formatted publishers? What exactly does a formatted publisher do? It seems like it would write data to the remote receiver in a format that particular receiver expects, but I haven't found a lot of documentation on this yet. Is there an interactive way to generate dummy log events to send to different destinations and publishers to see what the effect is on remote receiver? Thanks for your help!
F5 Analytics 3.7.x - Questions and Problems
Hello Community, I stumbeld upon the F5 Analytics iApp and so I wanted to give it a try. My environmental requirement: Splunk Version 6.5 Communication from BigIP to Splunk HTTP Event Collector is working (telnet to Splunk port 8088) Syslog/Analytics data MUST use Management interface of BigIP no HSL 1st try: iApp Version 3.7.1 (and also 3.7.2rc4)* BIG-IP v12.1.2 (Build 2.0.276) and BIG-IP v13.1.1.2 (Build 0.0.4) Result: failed* Error in /var/log/ltm: Script (/Common/F5-Analytics-send_stats) generated this Tcl error: (script did not successfully complete: (01020036:3: The requested RADIUS Server (/Common/F5-Analytics.app) was not found. while executing "tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth" invoked from within "lindex [tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth] 0" invoked from within "set obj [lindex [tmsh::get_config auth radius-server /Common/$appname.app/$radius_ihealth] 0]" line:41)) 2nd try: iApp Version 3.7.0 BIG-IP v12.1.2 (Build 2.0.276) Result: not quite sure Some Error in /var/log/ltm: Stats Response for F5-Analytics 1542956978 0 fail Data publisher not found or not implemented when processing request (unknown request), tag (2901). Thus I seem to get Data but I don't know if this is correct. Health values on "Start" Dashboard are always 0 Device Health 0 as well, or even NaN, Most field just have no data SSL Dashboard: no data Question I still have Do I have to configure Analytic Profiles on the BigIP itself to get more data? when you can use only the Mgmt-Port, what are the best settings in the iApp? Can i user "Module High Speed Logging Streams -> yes" when I can only use the Mgmt-Port? Is there a guide, that shows how to setup the BigIP itself (i.e. analytic profiles on VS) to get the most out of the app? I followed the "Deploying the F5 Analytics iApp Template" but the result is not overwhelming Is there an explanation for the meaning of all the diagramms and stuff you get? Example: What does "Server Latency Health 100,0%" mean? I'm grateful for any hint, tip, advice from you guys. Cheers Ichnafi
Pool status in Splunk for F5 Networks
Hi all, I made some tests on Splunk with the 11.5.0 TMOS version. My tests were on AFM, LTM and also syslog events. LTM (with the iRule included) and AFM work fine, but for syslog events there's something wrong. If you want to have your pool status statistics on your Splunk, you have to parse syslog events. But those events have changed with the 11.5.0 version so here is the newest regex you'll need. /\]:\s(........:.):\sPool\s(\S+)\smember\s(\S+)\smonitor\sstatus\s(\S+)\.\s?\[?\s?(?:\S+)?\:?\s?(?:\S+)?\s?\]?\s+?\[\swas\s(\S+)\sfor\s(\S+)/ This regex goes to /opt/splunk/etc/apps/SplunkforF5Networks/default/transforms.conf under [f5-syslog-eventcode] I still have something missing into my Splunk configuration because I don't have all my pool status. If anybody has already play with it, could you tell me where I'm wrong ? Thanks.F5 Analytics iApp - Mapping Regex
I'm trying to write a mapping regex for the F5 Analytics iApp (3.7.0). I'm struggling to find a regex combination that works with the naming convention used for the virtual servers. Naming Convention ...vs Example app1.dev.443.vs app1.dev.80.vs app2.443.vs I'm looking for a regex that will pare the virtual server name down to just the app name. Using regex101.com, I was able to verify this regex does what I want. (.*?)\..* When I add it to the mapping table of the iApp, Splunk returns zero results. It seems either the iApp or TMOS itself does not honor the ? as it should. If I omit the ?, the iApp returns everything up to and including the .v of .vs instead of stopping at the first . after the app name. Does anyone have any regex suggestions to just return the app name?
F5 analytics iApp on v14.1
Hi everybody. I'm doing some testing with v14.1, and I've tried to install the F5 Analytics iApp. I've installed the f5.analytics.v3.7.2rc4 template and configured it with default values more or less. The iApp deploys without any problems, but is logging an error message when trying to run the iCall script: hostname="bigip1.home.local",errdefs_msgno="014f0013:3:",msg="Script (/Common/analytics-send_stats1) generated this Tcl error: (script did not successfully complete: (can't read \"splunkdestinationip_withrd\": no such variable\n while executing\n\"string compare \"$splunkdestinationip_withrd\" \"0\" \"\n (\"Splunk\" arm line 5)\n invoked from within\n\"switch -- $format {\n \"Splunk\" -\n \"F5 Analytics\" {\n catch {exec /bin/gzip $filename$i$ext -f}\n ...\"\n (\"for\" body line 4)\n invoked from within\n\"for {set i 0} {$i <= $currentfile} {incr i} {\n set curl_response \"fail\"\n set ext \"\"\n switch -- $format {\n ...\" line:2619))" I don't have any route domains or partition configured on my BigIP. Any ideas?Adding the body of requests/responses to the data being logged to Splunk via iRule.
Hi All, We are presently using the iRule below to log request / response data to splunk. I'd like to add the body of the requests to our splunk logging. I had tried to user HTTP::payload as part of HTTP_REQUEST however it seems that the irule no longer functions when I place this there. When I add HTTP_REQUEST_DATA to the iRule to cater for HTTP:payload, I break the app - I expect that this is my implementation of HTTP_REQUEST_DATA. Is there an easy way to add the logging of the body of the request and response to what is sent to splunk? Thanks in advance when CLIENT_ACCEPTED { set client_address [IP::client_addr] set vip [IP::local_addr] } when HTTP_REQUEST { set http_host [HTTP::host]:[TCP::local_port] set http_uri [HTTP::uri] set http_url $http_host$http_uri set http_method [HTTP::method] set http_version [HTTP::version] set http_user_agent [HTTP::header "User-Agent"] set http_content_type [HTTP::header "Content-Type"] set http_referrer [HTTP::header "Referer"] set tcp_start_time [clock clicks -milliseconds] set req_start_time [clock format [clock seconds] -format "%Y/%m/%d %H:%M:%S"] set cookie [HTTP::cookie names] set user [HTTP::username] set virtual_server [LB::server] if { [HTTP::header Content-Length] > 0 } then { set req_length [HTTP::header "Content-Length"] } else { set req_length 0 } } when HTTP_RESPONSE { set res_start_time [clock format [clock seconds] -format "%Y/%m/%d %H:%M:%S"] set node [IP::server_addr] set node_port [TCP::server_port] set http_status [HTTP::status] set req_elapsed_time [expr {[clock clicks -milliseconds] - $tcp_start_time}] if { [HTTP::header Content-Length] > 0 } then { set res_length [HTTP::header "Content-Length"] } else { set res_length 0 } set hsl [HSL::open -proto TCP -pool p-remote-logging] HSL::send $hsl "<190>,f5_irule=Splunk-iRule-HTTP,src_ip=$client_address,vip=$vip,http_method=$http_method,http_host=$http_host,http_uri=$http_uri,http_url=$http_url,http_version=$http_version,http_user_agent=\"$http_user_agent\",http_content_type=$http_content_type,http_referrer=\"$http_referrer\",req_start_time=$req_start_time,cookie=\"$cookie\",user=$user,virtual_server=\"$virtual_server\",bytes_in=$req_length,res_start_time=$res_start_time,node=$node,node_port=$node_port,http_status=$http_status,req_elapsed_time=$req_elapsed_time,bytes_out=$res_length\r\n" } when LB_FAILED { set hsl [HSL::open -proto TCP -pool p-remote-logging] HSL::send $hsl "<190>,f5_irule=Splunk-iRule-LB_FAILED,src_ip=$client_address,vip=$vip,http_method=$http_method,http_host=$http_host,http_uri=$http_uri,http_url=$http_url,http_version=$http_version,http_user_agent=\"$http_user_agent\",http_content_type=$http_content_type,http_referrer=\"$http_referrer\",req_start_time=$req_start_time,cookie=\"$cookie\",user=$user,virtual_server=\"$virtual_server\",bytes_in=$req_length\r\n" }
ASM Logging to Splunk Anomoly
Hello, The ASM logs we're sending to Splunk have random (Splunk assigned?) field names. For example, violation_rating is named cn2 in Splunk, attack_type shows up as cs4, user_agent is called pm_fpua in Splunk, and so on. Does anyone know if this is a Splunk issue or a logging profile issue? The profile I inherited was configured with a logging format of Common Event Format (ArcSight) although we're talking to Splunk. I assumed changing it back to Key-Value Pairs (Splunk) might fix the issue but it's still jacked up. Thanks, Tone
F5 Splunk logging issue
Hi Team, We have a requirement where all of the F5s in our environment need to send logs to the Splunk VIP configured on a different F5. I want to use F5 Analytics iApp to simplify the deployment. The Splunk VIP (on a different F5) has 2 Splunk servers as the Pool members. When we are creating "HTTP Event Collector" on Spunk, it generates a Token which I need to use in the iApp. However, the issue is each Splunk server is generating a separate Token (so total 2 tokens) whereas the iApp is accepting only one token. How can we resolve this issue? Thanks in advance!! Regards Vijay7200 chassis logs to Splunk as 'localhost' rather than hostname
We have a couple of 7200 chassis with four VMs on each, arranged as four pairs of active/standby vcmp guests. I noticed today that in Splunk, the VMs are showing up (logging) using their host names, but both chassis are showing up as localhost. I've looked over the config and they only thing I've found different so far (comparing a VM to a chassis) is in the DNS setting for the VM it set with localhost and mydomain.com, whereas the chassis is just set with localhost. This is in System››Configuration : Device : DNS (DNS Search Domain List). I wouldn't think that would matter here, but I thought I would ask. Thanks!