FTP Session Logging
Problem this snippet solves: This iRule logs FTP connections and username information. By default connection mapping from client through BIG-IP to server is logged as well as the username entered by the client. Optionally you can log the entire FTP session by uncommenting the log message in CLIENT_DATA. Code : # This iRule logs FTP connections and username information. # By default connection mapping from client through BIG-IP to server is logged # as well as the username entered by the client. Optionally you can log the # entire FTP session by uncommenting the log message in CLIENT_DATA. when CLIENT_ACCEPTED { set vip [IP::local_addr]:[TCP::local_port] set user "unknown" } when CLIENT_DATA { # uncomment for full session logging #log local0. "[IP::client_addr]:[TCP::client_port]: collected payload ([TCP::payload length]): [TCP::payload]" # check if payload contains the string we want to replace if { [TCP::payload] contains "USER" } { # use a regular expression to save the user name ## regex modified by arkashik regexp "USER \(\[a-zA-Z0-9_-]+)" [TCP::payload] all user # log connection mapping from client through BIG-IP to server log local0. "FTP connection from $client. Mapped to $inside -> $node, user $user" TCP::release TCP::collect } else { TCP::release TCP::collect } } when SERVER_CONNECTED { set client "[IP::client_addr]:[TCP::client_port]" set node "[IP::server_addr]:[TCP::server_port]" set inside "[serverside {IP::local_addr}]:[serverside {TCP::local_port}]" TCP::collect } when SERVER_DATA { TCP::release clientside { TCP::collect } }Command Performance
Problem this snippet solves: The article Ten Steps to iRules Optimization illustrates some ways to optimize your iRules. I took a look at the control statements and built a little iRule that will test those assertions and generate performance graphs using Google Charts to present the findings. How to use this snippet: Dependencies This iRule relies on external Class files for the test on the "class match" command. The class names should be in the form of "class_xxx" where xxx is the list size you want to test. Include xxx number of entries with values from 0 to xxx-1. For a list size of 10, the class should look like this: # Snippet in bigip.conf class calc_10 { "0" "1" "2" "3" "4" "5" "6" "7" "8" "9" } I used perl to generate larger classes of size 100, 1000, 5000, and 10000 for my tests. Usage Assign the iRule to a virtual server and then browse to the url http://virtualserver/calccommands. I've included query string arguments to override the default test parameters as follows ls=nnn - List Size. You will need a class defined titled calc_10 for a value of ls=10. i=nnn - Number of iterations. This will be how many times the test is performed for each list size. gw=nnn - Graph Width (default value of 300) gh=nnn - Graph Height (default value of 200) ym=nnn - Graph Y Max value (default 500) An example usage is: http://virtualserver/calccommands?ls=1000&i=500. This will work on a list size of 1000 with 500 iterations per test. Code : when HTTP_REQUEST { #-------------------------------------------------------------------------- # read in parameters #-------------------------------------------------------------------------- set listsize [URI::query [HTTP::uri] "ls"]; set iterations [URI::query [HTTP::uri] "i"]; set graphwidth [URI::query [HTTP::uri] "gw"]; set graphheight [URI::query [HTTP::uri] "gh"]; set ymax [URI::query [HTTP::uri] "ym"]; #-------------------------------------------------------------------------- # set defaults #-------------------------------------------------------------------------- if { ("" == $iterations) || ($iterations > 10000) } { set iterations 500; } if { "" == $listsize } { set listsize 5000; } if { "" == $graphwidth } { set graphwidth 300; } if { "" == $graphheight } { set graphheight 200; } if { "" == $ymax } { set ymax 500; } set modulus [expr $listsize / 5]; set autosize 0; #-------------------------------------------------------------------------- # build lookup list #-------------------------------------------------------------------------- set matchlist "0"; for {set i 1} {$i < $listsize} {incr i} { lappend matchlist "$i"; } set luri [string tolower [HTTP::path]] switch -glob $luri { "/calccommands" { #---------------------------------------------------------------------- # check for existence of class file. If it doesn't exist # print out a nice error message. Otherwise, generate a page of # embedded graphs that route back to this iRule for processing #---------------------------------------------------------------------- if { [catch { class match "1" equals calc_$listsize } ] } { # error set content "<CENTER>BIG-IP Version $static::tcl_platform(tmmVersion)" append content "<H1 id="community-286142-toc-hId-1039316030"><FONT color="red">ERROR: class file 'calc_$listsize' not found</FONT></H1>"; append content ""; } else { # Build the html and send requests back in for the graphs... set content "<CENTER>BIG-IP Version $static::tcl_platform(tmmVersion)" append content "<P>List Size: ${listsize}</P><P></P><HR size="3" width="75%" /><P>" set c 0; foreach item $matchlist { set mod [expr $c % $modulus]; if { $mod == 0 } { append content "<IMG src="$luri/$item" append content "?ls=${listsize}&i=${iterations}&gw=${graphwidth}&gh=${graphheight}&ym=${ymax}" />"; } incr c; } append content "</P></CENTER>"; } HTTP::respond 200 content $content; } "/calccommands/*" { #---------------------------------------------------------------------- # Time various commands (switch, switch -glob, if/elseif, matchclass, # class match) and generate redirect to a Google Bar Chart #---------------------------------------------------------------------- set item [getfield $luri "/" 3] set labels "|" set values "" #---------------------------------------------------------------------- # Switch #---------------------------------------------------------------------- set expression "set t1 \[clock clicks -milliseconds\]; \n" append expression "for { set y 0 } { \$y < $iterations } { incr y } { " append expression "switch $item {" foreach i $matchlist { append expression "\"$i\" { } "; } append expression " } " append expression " } \n" append expression "set t2 \[clock clicks -milliseconds\]"; eval $expression; set duration [expr {$t2 - $t1}] if { [expr {$duration < 0}] } { log local0. "NEGATIVE TIME ($item, matchclass: $t1 -> $t2"; } append labels "s|"; if { $values ne "" } { append values ","; } append values "$duration"; if { $autosize && ($duration > $ymax) } { set ymax $duration } #---------------------------------------------------------------------- # Switch -glob #---------------------------------------------------------------------- set expression "set t1 \[clock clicks -milliseconds\]; \n" append expression "for { set y 0 } { \$y < $iterations } { incr y } { " append expression "switch -glob $item {" foreach i $matchlist { append expression "\"$i\" { } "; } append expression " } " append expression " } \n" append expression "set t2 \[clock clicks -milliseconds\]"; eval $expression; set duration [expr {$t2 - $t1}] if { [expr {$duration < 0}] } { log local0. "NEGATIVE TIME ($item, matchclass: $t1 -> $t2"; } append labels "s-g|"; if { $values ne "" } { append values ","; } append values "$duration"; if { $autosize && ($duration > $ymax) } { set ymax $duration } #---------------------------------------------------------------------- # If/Elseif #---------------------------------------------------------------------- set z 0; set y 0; set expression "set t1 \[clock clicks -milliseconds\]; \n" append expression "for { set y 0 } { \$y < $iterations } { incr y } { " foreach i $matchlist { if { $z > 0 } { append expression "else"; } append expression "if { $item eq \"$i\" } { } "; incr z; } append expression " } \n"; append expression "set t2 \[clock clicks -milliseconds\]"; eval $expression; set duration [expr {$t2 - $t1}] if { [expr {$duration < 0}] } { log local0. "NEGATIVE TIME ($item, matchclass: $t1 -> $t2"; } append labels "If|"; if { $values ne "" } { append values ","; } append values "$duration"; if { $autosize && ($duration > $ymax) } { set ymax $duration } #---------------------------------------------------------------------- # Matchclass on list #---------------------------------------------------------------------- set expression "set t1 \[clock clicks -milliseconds\]; \n" append expression "for { set y 0 } { \$y < $iterations } { incr y } { " append expression "if { \[matchclass $item equals \$matchlist \] } { }" append expression " } \n"; append expression "set t2 \[clock clicks -milliseconds\]"; eval $expression; set duration [expr {$t2 - $t1}] if { [expr {$duration < 0}] } { log local0. "NEGATIVE TIME ($item, matchclass: $t1 -> $t2"; } append labels "mc|"; if { $values ne "" } { append values ","; } append values "$duration"; if { $autosize && ($duration > $ymax) } { set ymax $duration } #---------------------------------------------------------------------- # class match (with class) #---------------------------------------------------------------------- set expression "set t1 \[clock clicks -milliseconds\]; \n" append expression "for { set y 0 } { \$y < $iterations } { incr y } { " append expression "if { \[class match $item equals calc_$listsize \] } { }" append expression " } \n"; append expression "set t2 \[clock clicks -milliseconds\]"; log local0. $expression; eval $expression; set duration [expr {$t2 - $t1}] if { [expr {$duration < 0}] } { log local0. "NEGATIVE TIME ($item, matchclass: $t1 -> $t2"; } append labels "c|"; if { $values ne "" } { append values ","; } append values "$duration"; if { $autosize && ($duration > $ymax) } { set ymax $duration } #---------------------------------------------------------------------- # build redirect for the google chart and issue a redirect #---------------------------------------------------------------------- set mod [expr $item % 10] set newuri "http://${mod}.chart.apis.google.com/chart?chxl=0:${labels}&chxr=1,0,${ymax}&chxt=x,y" append newuri "&chbh=a&chs=${graphwidth}x${graphheight}&cht=bvg&chco=A2C180&chds=0,${ymax}&chd=t:${values}" append newuri "&chdl=(in+ms)&chtt=Perf+(${iterations}-${item}/${listsize})&chg=0,2&chm=D,0000FF,0,0,3,1" HTTP::redirect $newuri; } } }
Log Tcp And Http Request Response Info
Problem this snippet solves: This iRule logs a line for the following events: when a new TCP connection is established with a client when the HTTP headers of an HTTP request are received from the client when the HTTP headers of an HTTP response are received from the pool member when the TCP connection with a client is closed Code : # Here is a sample of the log output for a single TCP connection with three HTTP requests: : New TCP connection from 192.168.99.210:2675 to 192.168.101.41:80 : Client 192.168.99.210:2675 -> test_http_vip/test0.html?parameter=val (request) : Client 192.168.99.210:2675 -> test_http_vip/test0.html?parameter=val (response) - pool info http_pool 192.168.101.45 80 - status: 200 (request/response delta: 0ms) : Client 192.168.99.210:2675 -> test_http_vip/test1.html?parameter=val (request) : Client 192.168.99.210:2675 -> test_http_vip/test1.html?parameter=val (response) - pool info http_pool 192.168.101.45 80 - status: 200 (request/response delta: 0ms) : Client 192.168.99.210:2675 -> test_http_vip/test2.html?parameter=val (request) : Client 192.168.99.210:2675 -> test_http_vip/test2.html?parameter=val (response) - pool info http_pool 192.168.101.45 80 - status: 200 (request/response delta: 1ms) : Closed TCP connection from 192.168.99.210:2675 to 192.168.101.41:80 (open for: 1078ms) when CLIENT_ACCEPTED { # Get time for start of TCP connection in milleseconds set tcp_start_time [clock clicks -milliseconds] # Log the start of a new TCP connection log local0. "New TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port]" } when HTTP_REQUEST { # Get time for start of HTTP request set http_request_time [clock clicks -milliseconds] # Log the start of a new HTTP request set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]" log local0. "$LogString (request)" } when LB_SELECTED { log local0. "Client [IP::client_addr]:[TCP::client_port]: Selected [LB::server]" } when LB_FAILED { log local0. "Client [IP::client_addr]:[TCP::client_port]: Failed to [LB::server]" } when SERVER_CONNECTED { log local0. "Client [IP::client_addr]:[TCP::client_port]: Connected to [IP::server_addr]:[TCP::server_port]" } when HTTP_RESPONSE { # Received the response headers from the server. Log the pool name, IP and port, status and time delta log local0. "$LogString (response) - pool info: [LB::server] - status: [HTTP::status] (request/response delta: [expr {[clock clicks -milliseconds] - $http_request_time}] ms)" } when CLIENT_CLOSED { # Log the end time of the TCP connection log local0. "Closed TCP connection from [IP::client_addr]:[TCP::client_port] to [IP::local_addr]:[TCP::local_port] (open for: [expr {[clock clicks -milliseconds] - $tcp_start_time}] ms)" }
iRule to decode WebSocket negotiation and frames
Problem this snippet solves: WebSocket establishes a socket via HTTP upgrade and once socket is established subsequent messages are non-HTTP, but WebSocket frame. There might be a situation where you want to dump WebSocket negotiation and frame into log for troubleshooting purpose. This iRule dumps negotiation and WebSocket frame header fields, and payload (only text data). WebSocket frame format looks as below. RFC 6455 - 5.2. Base Framing Protocol 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-------+-+-------------+-------------------------------+ |F|R|R|R| opcode|M| Payload len | Extended payload length | |I|S|S|S| (4) |A| (7) | (16/64) | |N|V|V|V| |S| | (if payload len==126/127) | | |1|2|3| |K| | | +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - + | Extended payload length continued, if payload len == 127 | + - - - - - - - - - - - - - - - +-------------------------------+ | |Masking-key, if MASK set to 1 | +-------------------------------+-------------------------------+ | Masking-key (continued) | Payload Data | +-------------------------------- - - - - - - - - - - - - - - - + : Payload Data continued ... : + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + | Payload Data continued ... | +---------------------------------------------------------------+ WebSocket frame is quite simple. The first 2 bytes are necessary and always present. Extended payload length exists only when Payload length is set to 126 (in this case Ext len is 16 bits) or 127 (in this case Ext len is 64 bits). Masking-key exists only if MASK bit is set to 1. FIN : If the frame is the last frame, set to 1. If payload is fragmented, the last frame should have FIN = 1. Other fragmented ones should have FIN = 0. RSV : If extension is not used, it is set to 0 opcode : This tells you if it is data frame (can be text or binary) or control frame. %x0:continuation frame %x1:text frame %x2:binary frame %x3-7:reserved for further %x8:connection close %x9:ping %xA:pong %xB-F:reserved for further MASK : When browser sends a data, this bit MUST be set to 1, which means data is masked using the Masking-key. When server sends a data, this bit MUST NOT set to 1 Payload len : If 0 - 125, this field represents the payload length If it is 126, then the Extended payload length (16bit) are used to tell the actual payload length (maximum data size is 65535 bytes) If it is 127, then the Extended payload length (64bit : MSB must be 0) are used to tell the actual payload length (maximum data size is 9223372036854775807 bytes) Extended payload length Only when Payload len is set to 126 Extended payload length_continued Only when Payload len is set to 127 Masking-key Used to mask data. Masking is to avoid proxy poisoning. Non-compliant HTTP proxy caches WebSocket data. If MASK bit is 1, this field is present. If MASK bit is 0, this field is not present. Client sets this key and it must be unpredictable. Payload Payload from client to server is masked using Masking-key. How to use this snippet: Here I am sending text data "DEAD BEEF" in WebSocket frame via WebSocket_cURL.py (https://github.com/jussmen/WebSocket_cURL). HTTP request and response (negotiation) look like below. $ python WebSocket_cURL.py 10.10.148.101 80 -s "DEAD BEEF" GET /ws HTTP/1.1 Host: 10.10.148.101 Connection: Upgrade Upgrade: websocket Sec-WebSocket-Version: 13 Sec-WebSocket-Key: n5twxG/tNPf8h3po+pNrPA== User-Agent: IE HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: y9WDs+d4zDl+qvQ7H17KpnP0EhI= This is how iRule dumps the negotiation and subsequent WebSocket frames in /var/log/ltm <ws_request>: ============================================= <ws_request>: Client 10.10.1.2:47427 -> 10.10.148.101/ws (request) <ws_request>: Host: 10.10.148.101 <ws_request>: Connection: Upgrade <ws_request>: Upgrade: websocket <ws_request>: Sec-WebSocket-Version: 13 <ws_request>: Sec-WebSocket-Key: n5twxG/tNPf8h3po+pNrPA== <ws_request>: User-Agent: IE <ws_request>: ============================================= <ws_response>: ============================================= <ws_response>: Client 10.10.1.2:47427 -> 10.10.148.101/ws (response) <ws_response>: Upgrade: websocket <ws_response>: Connection: Upgrade <ws_response>: Sec-WebSocket-Accept: y9WDs+d4zDl+qvQ7H17KpnP0EhI= <ws_response>: ============================================= <ws_server_frame>: ============================================= <ws_server_frame>: FIN bit : 1 <ws_server_frame>: MASK bit : 1 <ws_server_frame>: MASK : 0 <ws_server_frame>: Type : Text - 1 <ws_server_frame>: ============================================= <ws_server_data>: ============================================= <ws_server_data>: The server says: 'Hello'. Connection was accepted. <ws_server_data>: ============================================= <ws_client_frame>: ============================================= <ws_client_frame>: FIN bit : 1 <ws_client_frame>: MASK bit : 1 <ws_client_frame>: MASK : 1178944834 <ws_client_frame>: Type : Text - 1 <ws_client_frame>: ============================================= <ws_client_data>: ============================================= <ws_client_data>: DEAD BEEF <ws_client_data>: ============================================= <ws_server_frame>: ============================================= <ws_server_frame>: FIN bit : 1 <ws_server_frame>: MASK bit : 1 <ws_server_frame>: MASK : 0 <ws_server_frame>: Type : Text - 1 <ws_server_frame>: ============================================= <ws_server_data>: ============================================= <ws_server_data>: The server says: DEAD BEEF back at you <ws_server_data>: ============================================= <ws_client_frame>: ============================================= <ws_client_frame>: FIN bit : 1 <ws_client_frame>: MASK bit : 1 <ws_client_frame>: MASK : 1178944834 <ws_client_frame>: Type : Connection close - 8 <ws_client_frame>: ============================================= <ws_server_frame>: ============================================= <ws_server_frame>: FIN bit : 1 <ws_server_frame>: MASK bit : 1 <ws_server_frame>: MASK : 0 <ws_server_frame>: Type : Connection close - 8 <ws_server_frame>: ============================================= Code : when WS_REQUEST { # Copied from : https://devcentral.f5.com/s/articles/log-http-headers set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]" log local0. "=============================================" log local0. "$LogString (request)" foreach aHeader [HTTP::header names] { log local0. "$aHeader: [HTTP::header value $aHeader]" } log local0. "=============================================" } when WS_RESPONSE { # Copied from : https://devcentral.f5.com/s/articles/log-http-headers log local0. "=============================================" log local0. "$LogString (response)" foreach aHeader [HTTP::header names] { log local0. "$aHeader: [HTTP::header value $aHeader]" } log local0. "=============================================" } when WS_CLIENT_FRAME { log local0. "=============================================" log local0. "FIN bit : [WS::frame eom]" log local0. "MASK bit : [WS::frame orig_masked]" if { [WS::frame orig_masked] eq 0 } { log local0. "Not masked. Client frame MUST be masked." } if { [WS::frame orig_masked] eq 1 } { log local0. "MASK : [WS::frame mask]" } switch -glob [WS::frame type] { "0" { log local0. "Type : Continuatoin frame - 0" } "1" { log local0. "Type : Text - 1" WS::collect frame } "2" { log local0. "Type : Binary - 2" } "3" - "4" - "5" - "6" - "7" { log local0. "Type : Reserved type (3-7) - [WS::frame type]" } "8" { log local0. "Type : Connection close - 8" } "9" { log local0. "Type : ping - 9" } "10" { log local0. "Type : pong - 10" } "11" - "12" - "13" - "14" - "15" { log local0. "Type : Reserved type (11-15) - [WS::frame type]" } } log local0. "=============================================" } when WS_SERVER_FRAME { log local0. "=============================================" log local0. "FIN bit : [WS::frame eom]" log local0. "MASK bit : [WS::frame orig_masked]" if { [WS::frame orig_masked] eq 1 } { log local0. "MASK : [WS::frame mask]" } switch -glob [WS::frame type] { "0" { log local0. "Type : Continuatoin frame - 0" } "1" { log local0. "Type : Text - 1" WS::collect frame } "2" { log local0. "Type : Binary - 2" } "3" - "4" - "5" - "6" - "7" { log local0. "Type : Reserved type (3-7) - [WS::frame type]" } "8" { log local0. "Type : Connection close - 8" } "9" { log local0. "Type : ping - 9" } "10" { log local0. "Type : pong - 10" } "11" - "12" - "13" - "14" - "15" { log local0. "Type : Reserved type (11-15) - [WS::frame type]" } } log local0. "=============================================" } #when WS_CLIENT_FRAME_DONE { #log local0. "WS_CLIENT_FRAME_DONE" #} #when WS_SERVER_FRAME_DONE { #log local0. "WS_SERVER_FRAME_DONE" #} when WS_CLIENT_DATA { log local0. "=============================================" log local0. "[WS::payload]" log local0. "=============================================" WS::release } when WS_SERVER_DATA { log local0. "=============================================" log local0. "[WS::payload]" log local0. "=============================================" WS::release } Tested this on version: 12.0
Stream matched rewrite irule statistics
We have a rewrite irule that uses stream match to rewrite hostnames etc. For decomissioning purposes we woud like to generate a report showing which requests (urls) are using the irule and the results of rewrite. The report should have columns like matched, replacewith, fromURI, count when ACCESS_ACL_ALLOWED { #STREAM::disable #HTTP::header remove "Accept-Encoding" #log local0. "[IP::client_addr] is accessing and we are removing Accept-Encoding" } when CLIENT_ACCEPTED { ACCESS::restrict_irule_events enable } when HTTP_REQUEST { STREAM::disable HTTP::header remove "Accept-Encoding" set http_uri "https://[HTTP::host][HTTP::uri]" } when HTTP_RESPONSE { # Check if response type is text if {([HTTP::header value Content-Type] contains "text") || ([HTTP::header value Content-Type] contains "json")}{ STREAM::expression {@https?:\/\/([^\/\"]*\.)*city\.council\.com@replace_me@} # Enable the stream filter for this response only STREAM::enable } } when STREAM_MATCHED { #log local0. "Debug1: STREAM_MATCHED" set CHECK [class match -value [string tolower [STREAM::match]] starts_with URI_department.city.council.com_no_rewrite] if { $CHECK == "1" } { #log local0. "found exception for [STREAM::match]" STREAM::replace [STREAM::match] } else { set RE {https:\/\/([^\/\"]*\.)*city\.council\.com} set STRING "[string map {http:// https://} [STREAM::match]]" set SUBST "https://\\1department.city.council.com if {! ($STRING contains "intranett") } { # we need to insert intranett set STRING "[regsub -all $RE $STRING $SUBST ]" } #log local3. "[IP::client_addr]:[TCP::local_port]: matched: '[STREAM::match]', replaced with: '$STRING', from URI: $http_uri" STREAM::replace $STRING #if { ! ($STRING == [STREAM::match]) } { # } } }Link Tracking
Problem this snippet solves: This iRule application will track URI's that come in through your virtual server and store them in a table with the number of times they have been referenced. The data stored can then be viewed by passing in the "/linkadmin" url which will generate a table of the URI's and their view counts. Code : when HTTP_REQUEST { set TABLE_LINKDATA "LINK_TRACKING_[virtual name]" switch [string tolower [HTTP::uri]] { "/linkadmin" { set content { } foreach key [table keys -subtable $TABLE_LINKDATA] { append content ""; } append content " Clear Data URI Views $key [table lookup -subtable $TABLE_LINKDATA $key] "; HTTP::respond 200 Content $content; } "/linkcleardata" { table delete -subtable $TABLE_LINKDATA -all; HTTP::redirect "http://[HTTP::host]/linkadmin" } default { if { [table incr -subtable $TABLE_LINKDATA -mustexist [HTTP::uri]] eq ""} { table set -subtable $TABLE_LINKDATA [HTTP::uri] 1 indefinite indefinite; } } } }
Heatmaps Part1
Problem this snippet solves: I’ve been tinkering with a way to allow iRules to make that easier, and to allow those interested parties to see some usage statistics in a visually interesting, real-time manner, without adding much heavy lifting for you or your application. The idea is simple, create a heat map view of your HTTP requests, mapped to the locations around the United States (to start with). This will give you an idea which areas are most highly utilizing your application in a very easy on the eyes fashion. Best of all, of course, is that we’re going to generate this 100% with iRules. For the full write-up check out the article. (Note: There is an update to this iRule using the newer GeoCharts.) Code: (as gist) when HTTP_REQUEST { if {[HTTP::uri] starts_with "/heatmap"} { set chld "" set chd "" foreach state [table keys -subtable states] { append chld $state append chd "[table lookup -subtable states $state]," } set chd [string trimright $chd ","] HTTP::respond 200 content "<HTML><center><font size=5>Here is your site's usage by state:</font><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=usa&chd=t:$chd&chld=$chld&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'><a href='/resetmap'>Reset Map</a></center></HTML>" } elseif {[HTTP::uri] starts_with "/resetmap"} { foreach state [table keys -subtable states] { table delete -subtable states $state } HTTP::respond 200 Content "<HTML><center>Table Cleared. <a href='/heatmap'>Return to Map</a></HTML>" } else { set loc [whereis [IP::client_addr] abbrev] if {$loc eq ""} { set ip [expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(rand()*255) }] set loc [whereis $ip abbrev] } if {[table incr -subtable states -mustexist $loc] eq ""} { table set -subtable states $loc 1 indefinite indefinite } } }Heatmaps Part4
Problem this snippet solves: The final installment of the HeatMaps tech tip series, here is the completed project. A this stage you can view the entire world map, zoom to different regions, sort by URI, and you even get a readout of the actual numbers that are driving the maps being generated so you can tell exactly how many requests per region you're getting to each URI in question. Full tech tip here Code : when RULE_INIT { ## Configure static portions of the HTML response for the heatmap pages set static::resp1 "<HTML><table width='100%' border='0'><tr><td></td><td><center><font size=5>Here is your site's usage:\ </font></td></tr><tr><td align='center' rowspan='5'><b><u>Connections per Region:</u></b>" set static::resp2 "</td><td><center><img src='http://chart.apis.google.com/chart?cht=t&chd=&chs=440x220&chtm=" set static::resp3 "&chco=f5f5f5,edf0d4,6c9642,365e24,13390a' border='0'></center></td></tr><td><center>Zoom to region:\ <a href='/asia'>Asia</a> | <a href='/africa'>Africa</a> | <a href='/europe'>Europe</a> | <a href='/middle_east'>Middle East</a> | \ <a href='/south_america'>South America</a> | <a href='/usa'>United States</a> | <a href='/heatmap'>World</a></td></tr><tr><td><center>" set static::resp4 "<tr><td><center><a href='/resetmap'>Reset All Counters</a></center></td></tr><tr></tr></HTML>" } when HTTP_REQUEST timing on { switch -glob [string tolower [HTTP::uri]] { "/asia*" - "/africa*" - "/europe*" - "/middle_east*" - "/south_america*" - "/usa*" - "/world*" - "/heatmap*" { set chld "" set chd "" set zoom "" set zoomURL "" set regions "" set urlTotal 0 set regionTotal 0 ## Split apart the zoom region from the filter URL in the request set zoom [getfield [string map {"/" "" "heatmap" "world"} [HTTP::uri]] "?" 1] set zoomURL [getfield [string map {"/" "" "heatmap" "world"} [HTTP::uri]] "?" 2] ## Get a list of all states or countries, applying the URL filter where necessary ## and retrieve the associated count of requests from that area to that URL ## First step through the mytables table, which is a pointer table referencing all subtables with counter values in them foreach mysub [table keys -subtable mytables] { ## Next determine whether to search state or country tables if {$zoom eq "usa"} { if {$mysub starts_with "state:"} { ## For each state sub table step through each key, which will be a URL, and count the request to that URL. ## This is also where URL filtering is applied if applicable foreach myurl [table keys -subtable $mysub] { if {$zoomURL ne ""} { if {$myurl eq $zoomURL} { append chld "[getfield $mysub ":" 2]" append chd "[table lookup -subtable $mysub $myurl]," set urlTotal [table lookup -subtable $mysub $myurl] } } else { append chld "[getfield $mysub ":" 2]" append chd "[table lookup -subtable $mysub $myurl]," set urlTotal [table lookup -subtable $mysub $myurl] } set regionTotal [expr $regionTotal + $urlTotal] set urlTotal 0 } append regions "[getfield $mysub ":" 2] : $regionTotal" set regionTotal 0 } } } ## Send back the pre-formatted response, set in RULE_INIT, combined with the map zoom, list of areas, and requ est count set chd [string trimright $chd ","] ## First loop through the trackingurls class to get a list of all URLs to be tracked and format HTML around them for links set filters "" foreach mytrackingurl [class names trackingurls] { append filters "<a href='${zoom}?${mytrackingurl}'>${mytrackingurl}</a> | " } set filters [string trimright $filters " | "] ## Combine the above generated HTML with the static HTML in RULE INIT and respond to the client HTTP::respond 200 content "${static::resp1}${regions}${static::resp2}${zoom}&chd=t:${chd}&chld=${chld}${st atic::resp3} \ Filter by URL: <a href='/$zoom'>All URLs</a> | $filters\ $static::resp4" } "/resetmap" { foreach pointertable [table keys -subtable mytables] { foreach entry [table keys -subtable $pointertable] { table delete -subtable $pointertable $entry } } foreach pointerentry [table keys -subtable mytables] { table delete -subtable mytables $pointerentry } HTTP::respond 200 Content "<HTML><center>Table Cleared. <a href='/hea tmap'>Return to Map</a></HTML>" } default { ## Look up country & state locations set cloc [whereis [IP::client_addr] country] set sloc [whereis [IP::client_addr] abbrev] ## If the IP doesn't resolve to anything, pick a random IP (useful for testing on private networks) if {($cloc eq "") and ($sloc eq "")} { set ip [expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(rand()*255) }].[expr { int(ran d()*255) }] set cloc [whereis $ip country] set sloc [whereis $ip abbrev] if {($cloc eq "") or ($sloc eq "")} { set cloc "US" set sloc "WA" } } ## Strip slashes from URI to allow easy queries set friendlyURL [string map {/ ""} [HTTP::uri]] ## Create a new table named country:location or state:location if {[table incr -subtable country:$cloc -mustexist $friendlyURL] eq ""} { table set -subtable country:$cloc $friendlyURL 1 indefinite indefinite } ## Update the mytables pointer table with the new country or state table name if {[table incr -subtable mytables -mustexist country:$cloc] eq ""} { table set -subtable mytables country:$cloc 1 indefinite indefinite } ## Same as above for states, not countries. if {$cloc eq "US"} { if {[table incr -subtable state:$sloc -mustexist $friendlyURL] eq ""} { table set -subtable state:$sloc $friendlyURL 1 indefinite indefinite } if {[table incr -subtable mytables -mustexist state:$sloc] eq ""} { table set -subtable mytables state:$sloc 1 indefinite indefinite } } HTTP::respond 200 Content "Added - Country: $cloc State: $sloc" } } }What is BIG-IQ?
tl;dr - BIG-IQ centralizes management, licensing, monitoring, and analytics for your dispersed BIG-IP infrastructure. If you have more than a few F5 BIG-IP's within your organization, managing devices as separate entities will become an administrative bottleneck and slow application deployments. Deploying cloud applications, you're potentially managing thousands of systems and having to deal with traditionally monolithic administrative functions is a simple no-go. Enter BIG-IQ. BIG-IQ enables administrators to centrally manage BIG-IP infrastructure across the IT landscape. BIG-IQ discovers, tracks, manages, and monitors physical and virtual BIG-IP devices - in the cloud, on premise, or co-located at your preferred datacenter. BIG-IQ is a stand alone product available from F5 partners, or available through the AWS Marketplace. BIG-IQ consolidates common management requirements including but not limited to: Device discovery and monitoring: You can discovery, track, and monitor BIG-IP devices - including key metrics including CPU/memory, disk usage, and availability status Centralized Software Upgrades: Centrally manage BIG-IP upgrades (TMOS v10.20 and up) by uploading the release images to BIG-IQ and orchestrating the process for managed BIG-IPs. License Management: Manage BIG-IP virtual edition licenses, granting and revoking as you spin up/down resources. You can create license pools for applications or tenants for provisioning. BIG-IP Configuration Backup/Restore: Use BIG-IQ as a central repository of BIG-IP config files through ad-hoc or scheduled processes. Archive config to long term storage via automated SFTP/SCP. BIG-IP Device Cluster Support: Monitor high availability statuses and BIG-IP Device clusters. Integration to F5 iHealth Support Features: Upload and read detailed health reports of your BIG-IP's under management. Change Management: Evaluate, stage, and deploy configuration changes to BIG-IP. Create snapshots and config restore points and audit historical changes so you know who to blame. 😉 Certificate Management: Deploy, renew, or change SSL certs. Alerts allow you to plan ahead before certificates expire. Role-Based Access Control (RBAC): BIG-IQ controls access to it's managed services with role-based access controls (RBAC). You can create granular controls to create view, edit, and deploy provisioned services. Prebuilt roles within BIG-IQ easily allow multiple IT disciplines access to the areas of expertise they need without over provisioning permissions. Fig. 1 BIG-IQ 5.2 - Device Health Management BIG-IQ centralizes statistics and analytics visibility, extending BIG-IP's AVR engine. BIG-IQ collects and aggregates statistics from BIG-IP devices, locally and in the cloud. View metrics such as transactions per second, client latency, response throughput. You can create RBAC roles so security teams have private access to view DDoS attack mitigations, firewall rules triggered, or WebSafe and MobileSafe management dashboards. The reporting extends across all modules BIG-IQ manages, drastically easing the pane-of-glass view we all appreciate from management applications. For further reading on BIG-IQ please check out the following links: BIG-IQ Centralized Management @ F5.com Getting Started with BIG-IQ @ F5 University DevCentral BIG-IQ BIG-IQ @ Amazon Marketplace
