Enabling AVR and creating Profiles
Hi All, I've recently provisioned the AVR module (nominal) with out too much trouble. However when I attempt to create a HTTP Analytics profile, I don't see all the my Virtual Servers. I don't see any Virtual Servers when I look at the Statistics->Analytics->HTTP. However I am able to access: Statistics->Analytics->Virtual Servers- Statistics->Analytics->Pools Not sure what I'm missing, that I'm unable to see all the relevant info. Appreciate all advice and suggestions. Thanks DeenaHow I did it - "Remote Logging with the F5 XC Global Log Receiver and Elastic"
Welcome to configuring remote logging to Elastic, where we take a look at the F5 Distributed Cloud’s global log receiver service and we can easily send event log data from the F5 distributed cloud services platform to Elastic stack.
APM/Analytics - Log click on webtop resources [serverside]
Problem this snippet solves: When providing VPN SSL portal to the client, you publish several applications like Portal access, Remote Desktop, Webtop links, SAML Resources, Citrix/Vmware VDI, etc. APM log access to the webtop, once connected, there is no more visibility on who click on which icon. We provide Visibility and Analytics capabilities through a simple irule. How to use this snippet: Installation You just need to put this irule in the Virtual Server configuration that handle your access profile. Logging information Clicks are logged in the local0 (ltm logs). You can see below examples : virtual=/Common/test, apm=1239853, user=testuser, resource_type=portal, resourcename=owa virtual=/Common/test, apm=1239853, user=testuser, resource_type=remote_desktop, resourcename=ActiveDirectory Features You can currently log the following application types : Portal access Remote Desktop access The irule provide the additional features : Decode Portal access uri logging of username, apm session and resource name Credits Inspired from an original irule in a reply from Kevin Stewart : Logging for Portal Access External links Github : github.com/e-XpertSolutions/f5 Code : when ACCESS_ACL_ALLOWED { switch -glob [HTTP::uri] { "*resourcetype=remote_desktop*" { # # Basic logging. Remote or local logging settings can be configured # # log local0. "virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resourcetype=remote_desktop, [findstr [HTTP::uri] "resourcename=" 0 "\%"]" # # ACCESS logging before v13.x # log -noname accesscontrol.local1.notice "$static::ACCESS_LOG_PREFIX /Common/ap-ad-auth:Common:$session: virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resourcetype=remote_desktop, [findstr [HTTP::uri] "resourcename=" 0 "\%"]" # # ACCESS::log is available in v13.x. Log saved in Access report too. # # ACCESS::log accesscontrol.notice "virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resourcetype=remote_desktop, [findstr [HTTP::uri] "resourcename=" 0 "\%"]" } "*f5-w-*" { catch { set resource [binary format H* [findstr [HTTP::uri] "/f5-w-" 6 "\$\$"]] if { [table lookup -subtable PORTALACCESS "[ACCESS::session sid]:[ACCESS::session data get session.logon.last.username]:$resource"] eq "" } { table set -subtable PORTALACCESS "[ACCESS::session sid]:[ACCESS::session data get session.logon.last.username]:$resource" [clock format [clock seconds] -format %Y%m%d-%H%M%S] 3600 # # Basic logging. Remote or local logging settings can be configured # # log local0. "virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resource_type=portal, resourcename=$resource" # # ACCESS logging before v13.x # log -noname accesscontrol.local1.notice "$static::ACCESS_LOG_PREFIX /Common/ap-ad-auth:Common:$session: virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resource_type=portal, resourcename=$resource" # # ACCESS::log is available in v13.x. Log saved in Access report too. # # ACCESS::log accesscontrol.notice "virtual=[virtual], apm=[string range [ACCESS::session sid] [expr [string length [ACCESS::session sid]] - 10] end], user=[ACCESS::session data get session.logon.last.username], resource_type=portal, resourcename=$resource" } } } } } Tested this on version: 11.5Google Analytics script injection
Problem this snippet solves: Add google analytics script in the html content of the HTTP response. Works also for other Analytics providers like Piwik. How to use this snippet: Installation Files The code below has to be imported as an ifile. By default, you must name this ifile google.js but you can change it in the irule if required. Google Analytics code : <!-- Google Analytics --> <script> window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date; ga('create', '$static::tracking_id', 'auto'); ga('send', 'pageview'); </script> <script async src='https://www.google-analytics.com/analytics.js'></script> <!-- End Google Analytics --> Piwik javascript code : <!-- Piwik --> <script type="text/javascript"> var _paq = _paq || []; _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//$static::piwik_url/"; _paq.push(['setTrackerUrl', u+'piwik.php']); _paq.push(['setSiteId', {$static::siteid}]); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.type='text/javascript'; g.async=true; g.defer=true; g.src=u+'piwik.js'; s.parentNode.insertBefore(g,s); })(); </script> <!-- End Piwik Code --> irule You need to install the irule on your Virtual Server. Variables set static::tracking_id "UA-XXXXX-Y" # replace the Google Tracking ID by your own set static::siteid "UA-XXXXX-Y" # replace the Piwik Site ID by your own set static::piwik_url "https://www.mypiwik.com/piwik/piwik" # replace the Piwik URL by your own Features Version 1.0 Insert Google Analytics JS code within html response support for Piwik JS insertion Manage Multiple TrackingID by hostname (see Multiple "hostname and TrackingID section") Backlog Add logging External links Github : https://github.com/e-XpertSolutions/f5 BONUS : Multiple hostname and TrackingID Prerequisite You need to add a string based Datagroup named HOST_TRACKING_MAPPING. ltm data-group internal HOST_TRACKING_MAPPING { records { blog.e-xpertsolutions.com { data UA-XXXXX-Z } www.e-xpertsolutions.com { data UA-XXXXX-Y } } type string } The google.js ifile need to be replaced by the following example : <!-- Google Analytics --> <script> window.ga=window.ga||function(){(ga.q=ga.q||[]).push(arguments)};ga.l=+new Date; ga('create', '$tracking_id', 'auto'); ga('send', 'pageview'); </script> <script async src='https://www.google-analytics.com/analytics.js'></script> <!-- End Google Analytics --> Irule when RULE_INIT { set static::default_trackingid "UA-XXXXX-Y" } when HTTP_REQUEST { HTTP::header remove "Accept-Encoding" set host [HTTP::host] } when HTTP_RESPONSE { if { [HTTP::header Content-Type] contains "text/html" } { if { [HTTP::header exists "Content-Length"] } { set content_length [HTTP::header "Content-Length"] } else { set content_length 1000000 } if { $content_length > 0 } { HTTP::collect $content_length } } } when HTTP_RESPONSE_DATA { set search "</head>" set tracking_id [class match -value -- $host equals HOST_TRACKING_MAPPING] if { $tracking_id eq "" } { set tracking_id $static::default_trackingid } HTTP::payload replace 0 $content_length [string map [list $search "[subst -nocommands -nobackslashes [ifile get google.js]]</head>"] [HTTP::payload]] HTTP::release } Code : when RULE_INIT { set static::tracking_id "UA-XXXXX-Y" set static::siteid "XXXXX" set static::piwik_url "https://www.piwik.url/piwik/piwik" } when HTTP_REQUEST { HTTP::header remove "Accept-Encoding" } when HTTP_RESPONSE { if { [HTTP::header Content-Type] contains "text/html" } { if { [HTTP::header exists "Content-Length"] } { set content_length [HTTP::header "Content-Length"] } else { set content_length 1000000 } if { $content_length > 0 } { HTTP::collect $content_length } } } when HTTP_RESPONSE_DATA { set search "" HTTP::payload replace 0 $content_length [string map [list $search "[subst -nocommands -nobackslashes [ifile get google.js]]"] [HTTP::payload]] HTTP::release } Tested this on version: 11.5
Applying Analytics profile to VS changes chunking behavior?
Hi, I was doing some test (v11.2.0HF7) and for some reason two almost identically configured server was delivering different responses to the client: One chunked Second not chunked Both VS were sending traffic to the same backend server, using same settings for Response Chunking: Selective After long trial&error it turned out that disconnecting Analytics profile from VS disables chunking of the reply send to client. Why so? Is that bug or expected behavior? In Analytics profile options enabled as below: ltm profile analytics my-analytics { alerts none app-service none captured-traffic-external-logging disabled captured-traffic-internal-logging enabled collect-geo disabled collect-http-throughput enabled collect-ip disabled collect-methods enabled collect-page-load-time enabled collect-response-codes enabled collect-server-latency enabled collect-url enabled collect-user-agent disabled collect-user-sessions disabled collected-stats-external-logging disabled collected-stats-internal-logging enabled defaults-from analytics description none notification-by-email disabled notification-by-snmp disabled notification-by-syslog disabled notification-email-addresses none partition Common remote-server-ip any6 remote-server-port 514 remote-server-syslog-facility local0 sampling-ratio 1 session-timeout-minutes 30 smtp-config none traffic-capture { capturing-for-my-analytics { app-service none captured-protocols all client-ips none methods none node-addresses none request-captured-parts headers request-content-filter-search-part none request-content-filter-search-string none response-captured-parts headers response-codes none response-content-filter-search-part none response-content-filter-search-string none url-path-prefixes none user-agent-substrings none virtual-servers none } } trust-xff enabled } Is any of above setting forcing VS to send chunked response to client? PiotrApplying Analytics profile to VS changes chunking behavior?
Hi, I was doing some test (v11.2.0HF7) and for some reason two almost identically configured server was delivering different responses to the client: One chunked Second not chunked Both VS were sending traffic to the same backend server, using same settings for Response Chunking: Selective After long trial&error it turned out that disconnecting Analytics profile from VS disables chunking of the reply send to client. Why so? Is that bug or expected behavior? In Analytics profile options enabled as below: ltm profile analytics my-analytics { alerts none app-service none captured-traffic-external-logging disabled captured-traffic-internal-logging enabled collect-geo disabled collect-http-throughput enabled collect-ip disabled collect-methods enabled collect-page-load-time enabled collect-response-codes enabled collect-server-latency enabled collect-url enabled collect-user-agent disabled collect-user-sessions disabled collected-stats-external-logging disabled collected-stats-internal-logging enabled defaults-from analytics description none notification-by-email disabled notification-by-snmp disabled notification-by-syslog disabled notification-email-addresses none partition Common remote-server-ip any6 remote-server-port 514 remote-server-syslog-facility local0 sampling-ratio 1 session-timeout-minutes 30 smtp-config none traffic-capture { capturing-for-my-analytics { app-service none captured-protocols all client-ips none methods none node-addresses none request-captured-parts headers request-content-filter-search-part none request-content-filter-search-string none response-captured-parts headers response-codes none response-content-filter-search-part none response-content-filter-search-string none url-path-prefixes none user-agent-substrings none virtual-servers none } } trust-xff enabled } Is any of above setting forcing VS to send chunked response to client? PiotrHow to generate ASM report from a pre-defined filter using TMSH
Is there a way to generate a report using a pre-defined report (filter) using CLI (TMSH)? Though I should be able to generate the same report by typing down the filter for the report in TMSH but TMSH does not understand the Drilldown for Security Policy for some reason. For instanace, when I try to run the following command it is not able to identify the Policy save analytics application-security report view-by attack-type measures { occurrences } drilldown {{ entity policy values { My_Security_Policy_Name } } } format pdf file foo.pdf range now-3w The error I get for above is : Data Input Error: Invalid entity value My_Security_Policy_Name This is strange because I am able to see and set Security Policies in the filter in GUI. I am using Big-IP 12.1.0.AVR: User-Agent Strings
I'm looking at the AVR metrics collected, and trying to figure out which user-agents are most common in our environment. When I look at the analytics page, I can see the top 10 user-agents, but I can't seem to get anywhere to copy the full user-agent string (so i can parse out the specifics). It always truncates it somehow, even when I export the PDF or CSV. I did find here that I can view the data I'm looking for through tmsh with something like show / analytics http report view-by user-agent range now-31d , but I'd like to find a way to grab it from the GUI instead if I could. Any thoughts or experience with this?F5 analytics iApp on v14.1
Hi everybody. I'm doing some testing with v14.1, and I've tried to install the F5 Analytics iApp. I've installed the f5.analytics.v3.7.2rc4 template and configured it with default values more or less. The iApp deploys without any problems, but is logging an error message when trying to run the iCall script: hostname="bigip1.home.local",errdefs_msgno="014f0013:3:",msg="Script (/Common/analytics-send_stats1) generated this Tcl error: (script did not successfully complete: (can't read \"splunkdestinationip_withrd\": no such variable\n while executing\n\"string compare \"$splunkdestinationip_withrd\" \"0\" \"\n (\"Splunk\" arm line 5)\n invoked from within\n\"switch -- $format {\n \"Splunk\" -\n \"F5 Analytics\" {\n catch {exec /bin/gzip $filename$i$ext -f}\n ...\"\n (\"for\" body line 4)\n invoked from within\n\"for {set i 0} {$i <= $currentfile} {incr i} {\n set curl_response \"fail\"\n set ext \"\"\n switch -- $format {\n ...\" line:2619))" I don't have any route domains or partition configured on my BigIP. Any ideas?
Why does "Page Load Time" in AVR show no sampled transactions?
Hi All, I've got AVR working fine other than this so far. It's not clear to me if Page Load Times, currently showing N/A to transactions, needs more in the analytics profile than just Statistics Logging Type enabling (set to internal at the moment). I've tried several options, I got it to work... Once... But pretty much all options were on. I don't actually want all options. Current settings I have: ltm profile analytics analytics { app-service none captured-traffic-external-logging disabled captured-traffic-internal-logging disabled collect-geo disabled collect-ip disabled collect-max-tps-and-throughput enabled collect-methods enabled collect-page-load-time enabled collect-response-codes enabled collect-subnets disabled collect-url disabled collect-user-agent disabled collect-user-sessions enabled collected-stats-external-logging disabled collected-stats-internal-logging enabled description none notification-by-email disabled notification-by-snmp disabled notification-by-syslog disabled partition Common publish-irule-statistics disabled sampling disabled session-cookie-security ssl-only session-timeout-minutes 5 smtp-config none traffic-capture { capturing-for-analytics { request-captured-parts headers response-captured-parts headers } } } The profile is attached to the appropriate virtual-server. Any and all input is much appreciated. JD
