iRules Editor & Declarative Development with Visual Studio Code
The windows iRule Editor has had a very long life. But...it hasn't been updated in years and really should be sunsetted in your environment. There have been other attempts along the way, from a personal project with a Mac desktop app written in python and Qt that never made it past me, an Eclipse plugin several years back that gained a little traction, but the iRule Editor Joe Pruitt created lived on through all of that. However, there are a couple fantastic options now in the Visual Studio Code marketplace that combine to make for a great iRules development environment and also include the ability to pursue the automation toolchain development as well. Here are the tools you'll need: Visual Studio Code F5 Networks iRules (for iRules command completion and syntax highlighting) The F5 Extension (for session management and soooo much more) ACC Chariot (for converting config from UCS upload to AS3) John Wagnon and I had Ben Gordon on our DevCentral Connects live stream a couple times to highlight the functionality, which as mentioned goes far beyond just iRules.
CORS implementation
Problem this snippet solves: Pretty self-explanatory - we had to implement CORS (Cross-Origin-Resource-Sharing) where we had multiple domains, all of which had to be able to make AJAX calls to API's in our 'api.example.com' subdomain. Additionally, we had some partners who also need to be able to call our API's. In some cases, we had to pass cookies in the request. In the past, various developers had created backend Java code to return the CORS response headers, but almost invariably they did an incomplete job - either returning an invalid value or not returning all the required headers or writing the code such that it wasn't portable across applications. Therefore, I decided to write some 'common' CORS handling code, which would have the benefit of doing 'proper' origin checking and would also immediately return the OPTIONS preflight response directly from F5, thus improving performance. After much hacking around, here is what I came up with. We used a class to define multiple top-level domains as 'allowed' origins - this would contain both your domains and also those of any partners whom you want to allow to make CORS requests to your site. If you just have multiple subdomains on a single domain (e.g. www.example.com, api.example.com, code.example.com), you could simply use [HTTP::header Origin] ends_with ".example.com" - it's a little simpler. I'm always returning the Access-Control-Allow-Credentials: true response header for 'valid' non-OPTIONS (e.g. GET/POST) CORS requests, even if it's not required (i.e. even if the withCredentials flag was not set in the request) - unfortunately, there is no way to know whether it is needed simply by looking at the request, so it's the only way to ensure client errors don't occur. I'm passing the value of the Access-Control-Request-Method request header in the Access-Control-Allow-Methods response header (e.g. a single value of 'GET' or 'POST' or whatever) - in most implementations, you'll see people returning somethign like a string like Access-Control-Allow-Methods: GET, POST, PUT, but there's really no significant benefit to doing this - much simpler to only return what is passed. In either case, it will be cached by the browser because of the Access-Control-Max-Age response header. Note that because you will be returning a specific Access-Control-Allow-Origin value, rather than '*', you should also return the Vary: Origin response header. This may have issues with browser caching or if you use a CDN like Akamai or Cloudflare - you should consult any CDN product documentation. There are multiple good sources for explaining the Vary header - Google is your friend. If anyone has any comments, please add them, good or bad! I would love to know if someone finds this snippet useful... Code : # Domains that are allowed to make cross-domain calls to example.com class allowed_origins { ".example.com" ".example2.com" ".goodpartner.com" } when HTTP_REQUEST { unset -nocomplain cors_origin if { [class match [HTTP::header Origin] ends_with allowed_origins] } { if { ( [HTTP::method] equals "OPTIONS" ) and ( [HTTP::header exists "Access-Control-Request-Method"] ) } { # CORS preflight request - return response immediately HTTP::respond 200 "Access-Control-Allow-Origin" [HTTP::header "Origin"] \ "Access-Control-Allow-Methods" [HTTP::header "Access-Control-Request-Method"] \ "Access-Control-Allow-Headers" [HTTP::header "Access-Control-Request-Headers"] \ "Access-Control-Max-Age" "86400" \ "Vary" "Origin" } else { # CORS GET/POST requests - set cors_origin variable set cors_origin [HTTP::header "Origin"] } } } when HTTP_RESPONSE { # CORS GET/POST response - check cors_origin variable set in request if { [info exists cors_origin] } { HTTP::header insert "Access-Control-Allow-Origin" $cors_origin HTTP::header insert "Access-Control-Allow-Credentials" "true" HTTP::header insert "Vary" "Origin" } } Tested this on version: 11.0
Proxy Protocol Initiator
Problem this snippet solves: iRule Support for BIG-IP sending Proxy header to serverside pool member. (BIG-IP as Proxy Protocol Initiator) Implements v1 of PROXY protocol at: http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt How to use this snippet: Add iRule to Virtual Server. Back-end server should accept Proxy header. Code : when CLIENT_ACCEPTED { set proxyheader "PROXY " if {[IP::version] eq 4} { append proxyheader "TCP4 " } else { append proxyheader "TCP6 " } append proxyheader "[IP::remote_addr] [IP::local_addr] [TCP::remote_port] [TCP::local_port]\r\n" } when SERVER_CONNECTED { TCP::respond $proxyheader } ### Alternate Optimized Version ### when CLIENT_ACCEPTED { set proxyheader "PROXY TCP[IP::version] [IP::remote_addr] [IP::local_addr] [TCP::remote_port] [TCP::local_port]\r\n" } when SERVER_CONNECTED { TCP::respond $proxyheader } Tested this on version: 11.6
ProxyPass v10/v11
Problem this snippet solves: iRule to replace the functionality of Apache Webserver ProxyPass and ProxyPassReverse functions. Allows you to do hostname and path name modifications as HTTP traffic passes through the LTM. This optimized version requires TMOS v10 or higher. If you are using APM for authentication on the virtual server, please use the ProxyPass_for_use_with_APM iRule instead. (Full documentation follows the iRule.) For use with TMOS v9, see ProxyPass v8.2 Note: In 10.1+ you can use an internal data group to store the ProxyPass configuration. Ignore the comments to the contrary in the instructions as these were for pre-10.1 releases. Please post questions or fixes for this iRule in the iRules forum to get the fastest response. Thanks Aaron. Introduction Sometimes it is desirable to have a different look and feel to your website on the outside than you have in the inside. You may want www.company.com/usa/ to internally go to the server usa.company.com. You may want support.company.com to go internally to abc123.company.com/web/support. This can create a few issues - sometimes the web server expects to see a certain hostname (i.e. for name-based virtual hosting) or will use the internal hostname and/or path when sending a redirect to the clients. These problems can be overcome with the Apache webserver using the ProxyPass module which can translate the URL on the way into the server, and the ProxyPassReverse module which can un-translate header fields such as Location in the case of a redirect. Now you can accomplish this with an iRule. The ProxyPass iRule translates incoming requests in a flexible manner and untranslates the Location, Content-Location, and/or URI headers in the response back to the client. ProxyPass also rewrites the domain and path in cookies set by the server. Page Content Modification In addition, this rule will perform basic page modification as needed (this feature is disabled by default but can be enabled in the RULE_INIT event). Using the example from the introduction, if the page content contains a link to http://abc123.company.com/web/support/viewticket.html, the iRule will modify that to be http://support.company.com/viewticket.html. Here are some examples: < a href=http://www.domain.com/path/file.html> will be modified assuming the incoming request was matched by the ProxyPass iRule and the inside hostname was www.domain.com and the inside path was /path. < a href="page2.html"> will not need to be modified as this is a URL already relative to the path. Even relative URLs such as < a href="../page2.html"> will work as long as it does not try to go above the top-level directory defined by the ProxyPass rules. You must have a stream profile defined on any virtual servers the rule is applied to in order to enable the page modification feature. Virtual Server The first step to using the ProxyPass iRule is to define the rule on your BIG-IP and associate it with one or more Virtual Servers. Note that each virtual server MUST have an HTTP profile defined (doesn't matter which one). I also highly recommend applying a OneConnect profile, especially if you will be choosing pools with ProxyPass. It also must have a stream profile associated with it if you want to uncomment the page modification code. The rule will work on HTTP sites as well as HTTPS sites where the SSL is terminated on the BIG-IP (i.e. a client-side SSL profile is defined). Data Groups You can apply the ProxyPass rule to any virtual server that you want to do translations on. But just applying the rule will have no effect unless you define the translations you want done. This is done by defining specific Data Groups. The ProxyPass iRule uses Data Groups which are created and managed by going to Local Traffic / Virtual Servers / iRules on the left menu bar in the BIG-IP GUI. Then choose the "Data Group List" tab at the top of the screen. Here you can create the data groups used by this rule. For 10.0.x, the data groups must be External, type "String", and Read-only. For 10.1 and higher, the data group can be internal or external with name=value pairings. In order to use this rule on a virtual server you must apply the rule to the virtual server and create a data group named ProxyPassXYZ where "XYZ" needs to be the name of the virtual server. If both of these conditions are not met then the rule will not work for that virtual server. ProxyPassXYZ Data Groups If your virtual server is named XYZ and has the ProxyPass iRule associated with it, it will look for a data group named ProxyPassXYZ. Assuming that class is found, for each new HTTP request, the rule will find the one row that matches the hostname/path used in the request. For example, the data group may contain 4 entries (each line below is one string in the data group): "www.usa.company.com/support" := "support.company-internal.com:8080/usa", "www.usa.company.com/" := "www.company-internal.com:8080/usa", "www.japan.company.com/" := "www.company-internal.com:8080/japan", "/" := "www.company-internal.com:8080/others", A request need not match any entries - if no entries match then the iRule will have no effect. But each request will only match at most one entry and that will be the entry with the most specific left-hand-side. Entries with hostnames specified on the left-hand-side will be matched before entries without hostnames. If multiple entries match then the entry with the longest path name on the left-hand-side will be used. The example above lists entries from most-specific to least-specific, just as the rule will process them, but in your actual data group the order of the entries does not matter. In the example above, requests to http://www.usa.company.com/support will match the first entry and have the host header changed to "support.company-internal.com:8080" and the URI will be rewritten so that the string /support at the beginning of the URI will be changed to /usa. Furthermore, requests to http://www.usa.company.com/ will match the second entry as long as the URI does not begin with "/support" in which case it would match the first entry. In that case the Host header will be changed to "www.company-internal.com:8080" and whatever URI the client sends in will be prepended with "/usa". Likewise all requests to http://www.japan.company.com/ will match the third entry and have the Host header changed to "www.company-internal.com:8080" and the URIs would be prepended with "/japan". Finally, all other requests that hit this example virtual server would match the least-specific rule which is simply "/" -- all URIs begin with "/" and thus all requests will match the fourth entry if they did not match any others. Remember that a catch-all entry is not required, but in this example we want to prepend the URI of all other requests with "/others". Note that the ProxyPass iRule does not actually alter the destination of the requests by default. In the examples above all of the requests would go to the default pool regardless of the entries they match. The hostnames and ports specified in the right-hand entry is only used to modify the Host header. To alter the destination pool see the next section. Dynamic Pool Selection You may also specify an alternate pool as a second item in the right-hand value of the entry. This is optional any items in the list without a pool name will just use the default pool associated with the virtual server. For example: "/support" := "support.company.com/ SupportPool", "/downloads" := "downloads.company.com/ DownloadPool", If the pool name is not valid the user will get an error and you should see an error message in /var/log/ltm. Dynamic SNAT You can optionally define a ProxyPassSNATs data group. This will allow you to use different SNAT IP based on which pool you send traffic to. The ProxyPassSNATs data group is shared by all virtual servers but will only have an effect if the selected pool is listed in the data group. The format of this data group (internal string type) is: Pool1 W.X.Y.Z Pool2 automap Dynamic ServerSSL Profiles You can optionally define a ProxyPassSSLProfiles data group and apply a generic serverssl profile to the virtual server. This will allow you to use different serverssl profiles based on which pool you send traffic to. The ProxyPassSSLProfiles data group is shared by all virtual servers but will only have an effect if the selected pool is listed in the data group and a generic serverssl is applied to the virtual server. The format of this data group (internal string type) is: Pool1 ServerSSLProfile1 Pool2 ServerSSLProfile2 Regular Expressions New in v10: you may also use regular expressions and backreferences when building your rule set. "/" := "=www.company.com/(.*?)/=$1.company.com/=", As you see, instead of a regular server-side entry, we have "=regex=replace=". Basically, in order for the regex to be run, the left-hand side must match the client host/path (just "/" in this case, which will always match unless something more specific matches). Once this happens, the client host and URI are combined into a form similar to www.domain.com/path and the regular expression is run a against it. If the regular expression does not match, ProxyPass does not alter the request. If it does match, the clientside path becomes the match string and the serverside path becomes the replace string. Within this replace string you can use $1 through $9 as back-references to grouped items in the original regular expression. So, the example above, entry is equivalent to all of these entries: "www.company.com/sales" := "sales.company.com/", "www.company.com/support" := "support.company.com/", "www.company.com/employment" := "employment.company.com/", Debugging You can debug your ProxyPass rules by setting the static::ProxyPassDebug variable at the top of the rule to 1 (or 2 for more verbose debugging). Once you do this you can SSH to the BIG-IP and run the command "tail -f /var/log/ltm" to see what ProxyPass is doing to your requests. Code : # ProxyPass iRule, Version 10.9 # Nov 26 2012. Date # THIS VERSION REQUIRES TMOS v10 or higher. Use ProxyPass v8.2 for TMOS 9.x. # This version does not work with APM-enabled virtual servers, please # download ProxyPass 10.2APM for this use case. # Created by Kirk Bauer # https://devcentral.f5.com/s/wiki/default.aspx/iRules/ProxyPassV10.html # (please see end of iRule for additional credits) # Purpose: # iRule to replace the functionality of Apache Webserver ProxyPass and # ProxyPassReverse functions. It allows you to perform host name and path name # modifications as HTTP traffic passes through the LTM. In other words, you # can have different hostnames and directory names on the client side as you # do on the server side and ProxyPass handles the necessary translations. # NOTE: You should not need to modify this iRule in any way except the settings # in the RULE_INIT event. Just apply the iRule to the virtual server and # define the appropriate Data Group and you are done. If you do make any # changes to this iRule, please send your changes and reasons to me so that # I may understand how ProxyPass is being used and possibly incorporate your # changes into the core release. # Configuration Requirements # 1) The ProxyPass iRule needs to be applied to an HTTP virtual server or # an HTTPS virtual server with a clientssl profile applied to it. # 2) A data group (LTM -> iRules -> Data Groups tab) must be defined with # the name "ProxyPassVIRTUAL" where VIRTUAL is the name of the virtual server # (case-sensitive!). See below for the format of this data group (class). # For 10.0.x, you must use an EXTERNAL data group. # 3) You must define a default pool on the virtual server unless you specify # a pool in every entry in the data group. # 4) If you are using ProxyPass to select alternate pools, you must define # a OneConnect profile in most cases! # 5) ProxyPass does not rewrite links embedded within pages by default, just # headers. If you want to change this, edit the $static::RewriteResponsePayload variable in RULE_INIT # and apply the default stream profile to the virtual server. # Data Group Information # For 10.0.x, you must define an external data group (type=String, read-only) which loads # from a file on your BIG-IP. For 10.1 and higher you can use an internal string data group with name=value pairings. # The format of the file is as follows: # "clientside" := "serverside", # or # "clientside" := "serverside poolname", # The clientside and serverside fields must contain a URI (at least a "/") and # may also contain a hostname. Here are some examples: # "/clientdir" := "/serverdir", # "www.host.com/clientdir" := "internal.company.com/serverdir", # "www.host.com/" := "internal.company.com/serverdir/", # Notes: # 1) You can optionally define a ProxyPassSNATs data group to SNAT based # on the pool selected. # 2) You can optionally define a ProxyPassSSLProfiles data group to select # a serverssl profile based on the pool selected. # 3) You can also use regular expressions which is documented on DevCentral. when RULE_INIT { # Enable to debug ProxyPass translations via log messages in /var/log/ltm # (2 = verbose, 1 = essential, 0 = none) set static::ProxyPassDebug 0 # Enable to rewrite page content (try a setting of 1 first) # (2 = attempt to rewrite host/path and just /path, 1 = attempt to rewrite host/path) set static::RewriteResponsePayload 0 } when CLIENT_ACCEPTED { # Get the default pool name. This is used later to explicitly select # the default pool for requests which don't have a pool specified in # the class. set default_pool [LB::server pool] # The name of the Data Group (aka class) we are going to use. # Parse just the virtual server name by stripping off the folders (if present) set clname "ProxyPass[URI::basename [virtual name]]" if { $static::ProxyPassDebug > 1 } { log local0. "[virtual name]: [IP::client_addr]:[TCP::client_port] -> [IP::local_addr]:[TCP::local_port]" } } when HTTP_REQUEST { # "bypass" tracks whether or not we made any changes inbound so we # can skip changes on the outbound traffic for greater efficiency. set bypass 1 # Initialize other local variables used in this rule set orig_uri "[HTTP::uri]" set orig_host "[HTTP::host]" set log_prefix "VS=[virtual name], Host=$orig_host, URI=$orig_uri" set clientside "" set serverside "" set newpool "" set ppass "" if {! [class exists $clname]} { log local0. "$log_prefix: Data group $clname not found, exiting." pool $default_pool return } else { set ppass [class match -element "$orig_host$orig_uri" starts_with $clname] if {$ppass eq ""} { # Did not find with hostname, look for just path set ppass [class match -element "$orig_uri" starts_with $clname] } if {$ppass eq ""} { # No entries found if { $static::ProxyPassDebug > 0 } { log local0. "$log_prefix: No rule found, using default pool $default_pool and exiting" } pool $default_pool return } } # Store each entry in the data group line into a local variable set clientside [getfield $ppass " " 1] set serverside [string trimleft [getfield $ppass " " 2 ] "{" ] set newpool [string trimright [getfield $ppass " " 3 ] "}" ] # If serverside is in the form =match=replace=, apply regex if {$serverside starts_with "="} { set regex [getfield $serverside "=" 2] set rewrite [getfield $serverside "=" 3] if {[regexp -nocase $regex "$orig_host$orig_uri" 0 1 2 3 4 5 6 7 8 9]}{ # The clientside becomes the matched string and the serverside the substitution set clientside $0 set serverside [eval set X $rewrite] } else { pool $default_pool return } } if {$clientside starts_with "/"} { # No virtual hostname specified, so use the Host header instead set host_clientside $orig_host set path_clientside $clientside } else { # Virtual host specified in entry, split the host and path set host_clientside [getfield $clientside "/" 1] set path_clientside [substr $clientside [string length $host_clientside]] } # At this point $host_clientside is the client hostname, and $path_clientside # is the client-side path as specified in the data group set host_serverside [getfield $serverside "/" 1] set path_serverside [substr $serverside [string length $host_serverside]] if {$host_serverside eq ""} { set host_serverside $host_clientside } # At this point $host_serverside is the server hostname, and $path_serverside # is the server-side path as specified in the data group # In order for directory redirects to work properly we have to be careful with slashes if {$path_clientside equals "/"} { # Make sure serverside path ends with / if clientside path is "/" if {!($path_serverside ends_with "/")} { append path_serverside "/" } } else { # Otherwise, neither can end in a / (unless serverside path is just "/") if {!($path_serverside equals "/")} { if {$path_serverside ends_with "/"} { set path_serverside [string trimright $path_serverside "/"] } if {$path_clientside ends_with "/"} { set path_clientside [string trimright $path_clientside "/"] } } } if { $static::ProxyPassDebug } { log local0. "$log_prefix: Found Rule, Client Host=$host_clientside, Client Path=$path_clientside, Server Host=$host_serverside, Server Path=$path_serverside" } # If you go to http://www.domain.com/dir, and /dir is a directory, the web # server will redirect you to http://www.domain.com/dir/. The problem is, with ProxyPass, if the client-side # path is http://www.domain.com/dir, but the server-side path is http://www.domain.com/, the server will NOT # redirect the client (it isn't going to redirect you to http://www.domain.com//!). Here is the problem with # that. If there is an image referenced on the page, say logo.jpg, the client doesn't realize /dir is a directory # and as such it will try to load http://www.domain.com/logo.jpg and not http://www.domain.com/dir/logo.jpg. So # ProxyPass has to handle the redirect in this case. This only really matters if the server-side path is "/", # but since we have the code here we might as well offload all of the redirects that we can (that is whenever # the client path is exactly the client path specified in the data group but not "/"). if {$orig_uri eq $path_clientside} { if {([string index $path_clientside end] ne "/") and not ($path_clientside contains ".") } { set is_https 0 if {[PROFILE::exists clientssl] == 1} { set is_https 1 } # Assumption here is that the browser is hitting http://host/path which is a virtual path and we need to do the redirect for them if {$is_https == 1} { HTTP::redirect "https://$orig_host$orig_uri/" if { $static::ProxyPassDebug } { log local0. "$log_prefix: Redirecting to https://$orig_host$orig_uri/" } } else { HTTP::redirect "http://$orig_host$orig_uri/" if { $static::ProxyPassDebug } { log local0. "$log_prefix: Redirecting to http://$orig_host$orig_uri/" } } return } } if {$host_clientside eq $orig_host} { if {$orig_uri starts_with $path_clientside} { set bypass 0 # Take care of pool selection if {$newpool eq ""} { pool $default_pool if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: Using default pool $default_pool" } set newpool $default_pool } else { pool $newpool if { $static::ProxyPassDebug > 0 } { log local0. "$log_prefix: Using parsed pool $newpool (make sure you have OneConnect enabled)" } } } } # If we did not match anything, skip the rest of this event if {$bypass} { return } # The following code will look up SNAT addresses from # the data group "ProxyPassSNATs" and apply them. # # The format of the entries in this list is as follows: # # # # All entries are separated by spaces, and both items # are required. set class_exists_cmd "class exists ProxyPassSNATs" if {! [eval $class_exists_cmd]} { return } set snat [findclass $newpool ProxyPassSNATs " "] if {$snat eq ""} { # No snat found, skip rest of this event return } if { $static::ProxyPassDebug > 0 } { log local0. "$log_prefix: SNAT address $snat assigned for pool $newpool" } snat $snat } when HTTP_REQUEST_SEND { # If we didn't match anything, skip the rest of this event if {$bypass} { return } # The following code does the actual rewrite on its way TO # the backend server. It replaces the URI with the newly # constructed one and masks the "Host" header with the FQDN # the backend pool server wants to see. # # If a new pool or custom SNAT are to be applied, these are # done here as well. If a SNAT is used, an X-Forwarded-For # header is attached to send the original requesting IP # through to the server. if {$host_clientside eq $orig_host} { if {$orig_uri starts_with $path_clientside} { if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: New Host=$host_serverside, New Path=$path_serverside[substr $orig_uri [string length $path_clientside]]" } clientside { # Rewrite the URI HTTP::uri $path_serverside[substr $orig_uri [string length $path_clientside]] # Rewrite the Host header HTTP::header replace Host $host_serverside # Now alter the Referer header if necessary if { [HTTP::header exists "Referer"] } { set protocol [URI::protocol [HTTP::header Referer]] if {$protocol ne ""} { set client_path [findstr [HTTP::header "Referer"] $host_clientside [string length $host_clientside]] if {$client_path starts_with $path_clientside} { if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: Changing Referer header: [HTTP::header Referer] to $protocol://$host_serverside$path_serverside[substr $client_path [string length $path_clientside]]" } HTTP::header replace "Referer" "$protocol://$host_serverside$path_serverside[substr $client_path [string length $path_clientside]]" } } } } } } # If we're rewriting the response content, prevent the server from using #compression in its response by removing the Accept-Encoding header #from the request. LTM does not decompress response content before #applying the stream profile. This header is only removed if we're #rewriting response content. clientside { if { $static::RewriteResponsePayload } { if { [HTTP::header exists "Accept-Encoding"] } { HTTP::header remove "Accept-Encoding" if { $static::ProxyPassDebug > 1} { log local0. "$log_prefix: Removed Accept-Encoding header" } } } HTTP::header insert "X-Forwarded-For" "[IP::remote_addr]" } } when HTTP_RESPONSE { if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: [HTTP::status] response from [LB::server]" } if {$bypass} { # No modification is necessary if we didn't change anything inbound so disable the stream filter if it was enabled # Check if we're rewriting the response if {$static::RewriteResponsePayload} { if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: Rewriting response content enabled, but disabled on this response." } # Need to explicity disable the stream filter if it's not needed for this response # Hide the command from the iRule parser so it won't generate a validation error #when not using a stream profile set stream_disable_cmd "STREAM::disable" # Execute the STREAM::disable command. Use catch to handle any errors. Save the result to $result if { [catch {eval $stream_disable_cmd} result] } { # There was an error trying to disable the stream profile. log local0. "$log_prefix: Error disabling stream filter ($result). If you enable static::RewriteResponsePayload, then you should add a stream profile to the VIP. Else, set static::RewriteResponsePayload to 0 in this iRule." } } # Exit from this event. return } # Check if we're rewriting the response if {$static::RewriteResponsePayload} { # Configure and enable the stream filter to rewrite the response payload # Hide the command from the iRule parser so it won't generate a validation error #when not using a stream profile if {$static::RewriteResponsePayload > 1} { set stream_expression_cmd "STREAM::expression \"@$host_serverside$path_serverside@$host_clientside$path_clientside@ @$path_serverside@$path_clientside@\"" } else { set stream_expression_cmd "STREAM::expression \"@$host_serverside$path_serverside@$host_clientside$path_clientside@\"" } set stream_enable_cmd "STREAM::enable" if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: \$stream_expression_cmd: $stream_expression_cmd, \$stream_enable_cmd: $stream_enable_cmd" } # Execute the STREAM::expression command. Use catch to handle any errors. Save the result to $result if { [catch {eval $stream_expression_cmd} result] } { # There was an error trying to set the stream expression. log local0. "$log_prefix: Error setting stream expression ($result). If you enable static::RewriteResponsePayload, then you should add a stream profile to the VIP. Else, set static::RewriteResponsePayload to 0 in this iRule." } else { # No error setting the stream expression, so try to enable the stream filter # Execute the STREAM::enable command. Use catch to handle any errors. Save the result to $result if { [catch {eval $stream_enable_cmd} result] } { # There was an error trying to enable the stream filter. log local0. "$log_prefix: error enabling stream filter ($result)" } else { if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: Successfully configured and enabled stream filter" } } } } # Fix Location, Content-Location, and URI headers foreach header {"Location" "Content-Location" "URI"} { set protocol [URI::protocol [HTTP::header $header]] if { $static::ProxyPassDebug > 1 } { log local0. "$log_prefix: Checking $header=[HTTP::header $header], \$protocol=$protocol" } if {$protocol ne ""} { set server_path [findstr [HTTP::header $header] $host_serverside [string length $host_serverside]] if {$server_path starts_with $path_serverside} { if { $static::ProxyPassDebug } { log local0. "$log_prefix: Changing response header $header: [HTTP::header $header] with $protocol://$host_clientside$path_clientside[substr $server_path [string length $path_serverside]]" } HTTP::header replace $header $protocol://$host_clientside$path_clientside[substr $server_path [string length $path_serverside]] } } } # Rewrite any domains/paths in Set-Cookie headers if {[HTTP::header exists "Set-Cookie"]}{ array unset cookielist foreach cookievalue [HTTP::header values "Set-Cookie"] { set cookiename [getfield $cookievalue "=" 1] set namevalue "" set newcookievalue "" foreach element [split $cookievalue ";"] { set element [string trim $element] if {$namevalue equals ""} { set namevalue $element } else { if {$element contains "="} { set elementname [getfield $element "=" 1] set elementvalue [getfield $element "=" 2] if {[string tolower $elementname] eq "domain"} { set elementvalue [string trimright $elementvalue "."] if {$host_serverside ends_with $elementvalue} { if {$static::ProxyPassDebug > 1} { log local0. "$log_prefix: Modifying cookie $cookiename domain from $elementvalue to $host_clientside" } set elementvalue $host_clientside } append elementvalue "." } if {[string tolower $elementname] eq "path"} { if {$elementvalue starts_with $path_serverside} { if {$static::ProxyPassDebug > 1} { log local0. "$log_prefix: Modifying cookie $cookiename path from $elementvalue to $path_clientside[substr $elementvalue [string length $path_serverside]]" } set elementvalue $path_clientside[substr $elementvalue [string length $path_serverside]] } } append newcookievalue "; $elementname=$elementvalue" } else { append newcookievalue "; $element" } } } set cookielist($cookiename) "$namevalue$newcookievalue" } HTTP::header remove "Set-Cookie" foreach cookiename [array names cookielist] { HTTP::header insert "Set-Cookie" $cookielist($cookiename) if {$static::ProxyPassDebug > 1} { log local0. "$log_prefix: Inserting cookie: $cookielist($cookiename)" } } } } # Only uncomment this event if you need extra debugging for content rewriting. # This event can only be uncommented if the iRule is used with a stream profile. #when STREAM_MATCHED { #if { $static::ProxyPassDebug } { #log local0. "$log_prefix: Rewriting match: [STREAM::match]" #} #} # The following code will look up SSL profile rules from # the Data Group ProxyPassSSLProfiles" and apply # them. # # The format of the entries in this list is as follows: # # # # All entries are separated by spaces, and both items # are required. The virtual server also will need to # have any serverssl profile applied to it for this to work. when SERVER_CONNECTED { if {$bypass} { return } set class_exists_cmd "class exists ProxyPassSSLProfiles" if {! [eval $class_exists_cmd]} { return } set pool [LB::server pool] set profilename [findclass $pool ProxyPassSSLProfiles " "] if {$profilename eq ""} { if { [PROFILE::exists serverssl] == 1} { # Hide this command from the iRule parser (in case no serverssl profile is applied) set disable "SSL::disable serverside" catch {eval $disable} } return } if { $static::ProxyPassDebug > 0 } { log local0. "$log_prefix: ServerSSL profile $profilename assigned for pool $pool" } if { [PROFILE::exists serverssl] == 1} { # Hide these commands from the iRule parser (in case no serverssl profile is applied) set profile "SSL::profile $profilename" catch {eval $profile} set enable "SSL::enable serverside" catch {eval $enable} } else { log local0. "$log_prefix: ServerSSL profile must be defined on virtual server to enable server-side encryption!" } } # ProxyPass Release History #v10.9: Nov 26, 2012: Used URI::basename to get the virtual server name. Thanks to Opher Shachar for the suggestion. #Replaced indentations with tabs intead of spaces to save on characters #v10.8: Oct 25, 2012: Updated the class name to remove the folder(s) (if present) from the virtual server name. # This assumes the ProxyPass data group is in the same partition as the iRule. #v10.7: Oct 24, 2012: Changed array set cookielist {} to array unset cookielist as the former does not clear the array. # Thanks to rhuyerman@schubergphilis.com and Simon Kowallik for pointing out the issue and this wiki page with details: http://wiki.tcl.tk/724 #v10.6: Oct 14, 2012: Updated how the protocol is parsed from URLs in request and response headers to fix errant matches #v10.5: Feb 2, 2012: Removed extra stream profile $result reference for debug logging. #v10.4: Nov 23, 2011: Removed an extra colon in sever HTTP::header replace commands to prevent duplicate headers from being inserted #v10.3: Sep 27, 2010: Moved rewrite code to HTTP_REQUEST_SEND to work with WebAccelerator # Fixed bug with cookie rewrites when cookie value contained an "=" #v10.2: Jun 04, 2010: Can handle individual file mappings thanks to Michael Holmes from AZDOE # Also fixed bug with directory slash logic #v10.1: Oct 24, 2009: Now CMP-friendly! (NOTE: use ProxyPass v8.2 for TMOS v9.x) #v10.0: May 15, 2009: Optimized for external classes in v10 only (use v8.2 for TMOS v9.x) # Added support for regular expressions and backreferences for the translations. # v8.2: Jun 04, 2010: Fixed bug with directory slash logic # v8.1: May 15, 2009: Added internal redirects back in (removing them was a mistake) # v8.0: May 13, 2009: pulled in changes submitted by Aaron Hooley (hooleylists gmail com) # TMOS v10 support added. Cookie domain/path rewriting added. # v7.0: May 6, 2008: added optional serverssl contributed by Joel Moses # v6.0: Jan 15, 2008: Small efficiency change # v5.0: Jul 27, 2007: Added Referer header conversions # v4.0: Jul 27, 2007: Added optional debugging flag # v3.0: Jul 20, 2007: Added SNAT support contributed by Adam Auerbach # v2.0: May 28, 2007: Added internal directory redirects and optional stream profile # v1.0: Feb 20, 2007: Initial Release Tested this on version: 10.0
Log Http Headers
Problem this snippet solves: This simple rule logs all HTTP headers in requests and responses to /var/log/ltm. This can be helpful in troubleshooting. Code : when HTTP_REQUEST { set LogString "Client [IP::client_addr]:[TCP::client_port] -> [HTTP::host][HTTP::uri]" log local0. "=============================================" log local0. "$LogString (request)" foreach aHeader [HTTP::header names] { log local0. "$aHeader: [HTTP::header value $aHeader]" } log local0. "=============================================" } when HTTP_RESPONSE { log local0. "=============================================" log local0. "$LogString (response) - status: [HTTP::status]" foreach aHeader [HTTP::header names] { log local0. "$aHeader: [HTTP::header value $aHeader]" } log local0. "=============================================" } # Sample output: Rule log_http_headers_rule : ============================================= Rule log_http_headers_rule : Client 192.168.99.32:2950 -> webmail.example.com/exchange/Aaron/Inbox/?Cmd=contents (request) Rule log_http_headers_rule : Host: webmail Rule log_http_headers_rule : User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.9) Rule log_http_headers_rule : Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,im Rule log_http_headers_rule : Accept-Language: en-us,en;q=0.5 Rule log_http_headers_rule : Accept-Encoding: gzip,deflate Rule log_http_headers_rule : Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Rule log_http_headers_rule : Keep-Alive: 300 Rule log_http_headers_rule : Connection: keep-alive Rule log_http_headers_rule : Referer: https://webmail.example.com/exchange/ Rule log_http_headers_rule : X-Forwarded-For: 192.168.99.32 Rule log_http_headers_rule : Front-End-Https: On Rule log_http_headers_rule : ============================================= Rule log_http_headers_rule : ============================================= Rule log_http_headers_rule : Client 192.168.99.32:2950 -> webmail.example.com/exchange/Aaron/Inbox/?Cmd=contents (response) - status: 200 Rule log_http_headers_rule : Date: Tue, 06 Nov 2007 16 Rule log_http_headers_rule : Server: Microsoft-IIS/6.0 Rule log_http_headers_rule : X-Powered-By: ASP.NET Rule log_http_headers_rule : Content-Type: text/html Rule log_http_headers_rule : Content-Length: 55446 Rule log_http_headers_rule : MS-WebStorage: 6.5.7638 Rule log_http_headers_rule : Cache-Control: no-cache Rule log_http_headers_rule : =============================================iRule Event Order Flowchart
Problem this snippet solves The original contributions are from HTTP Event Order -- Access Policy Manager - DevCentral (f5.com) I have updated this code example to use draw.io so that anyone can update diagram for required changes. Attempt to diagram a logic path for iRule event processing visually. There is special emphasis with ACCESS events as their lifecycle is a bit more complicated. These events should cover the most common use cases to customize ACCESS functionality. How to use this snippet This diagram can be updated via the free draw.io at https://app.diagrams.net/ Tested this on version No Version FoundProxy Protocol Receiver
Problem this snippet solves: iRule for BIG-IP to receive PROXY protocol (v1 and v2) header in TCP Payload and remove it before forwarding remaining TCP Payload to server side pool member. How to use this snippet: Enable iRule on virtual server where upstream proxy will be sending PROXY header. Testing done using proxied IPv6 and IPv4 HTTP connections from HAProxy using Proxy Protocol v1 and v2. Use of client or server SSL profiles slated for testing and validation since ELB promotes use of Proxy Protocol as a solution for customers that don't want to have ELB terminate HTTPS traffic but do want servers to see original IP addresses. Code : #PROXY Protocol Receiver iRule # c.jenison at f5.com (Chad Jenison) # v2.0 - Added support for PROXY Protocol v2, control for v1,v2 or lack of proxy via static:: variables set in RULE_INIT # v2.1 - Fix for skipping bytes in v2 code when RULE_INIT { set static::allowProxyV1 0 set static::allowProxyV2 1 set static::allowNoProxy 0 } when CLIENT_ACCEPTED { TCP::collect } when CLIENT_DATA { binary scan [TCP::payload 12] H* v2_protocol_sig if {$static::allowProxyV1 && [TCP::payload 0 5] eq "PROXY"} { set proxy_string [TCP::payload] set proxy_string_length [expr {[string first "\r" [TCP::payload]] + 2}] scan $proxy_string {PROXY TCP%s%s%s%s%s} tcpver srcaddr dstaddr srcport dstport log "Proxy Protocol v1 conn from [IP::client_addr]:[TCP::client_port] for an IPv$tcpver stream from Src: $srcaddr:$srcport to Dst: $dstaddr:$dstport" TCP::payload replace 0 $proxy_string_length "" } elseif {$static::allowProxyV2 && $v2_protocol_sig eq "0d0a0d0a000d0a515549540a"}{ binary scan [TCP::payload] @12H* v2_proxyheaderremainder binary scan [TCP::payload] @12H2H* v2_verCommand v2_remainder if {$v2_verCommand == 21}{ binary scan [TCP::payload] @13H2 v2_addressFamilyTransportProtocol if {$v2_addressFamilyTransportProtocol == 11} { binary scan [TCP::payload] @16ccccccccSS v2_sourceAddress1 v2_sourceAddress2 v2_sourceAddress3 v2_sourceAddress4 v2_destAddress1 v2_destAddress2 v2_destAddress3 v2_destAddress4 v2_sourcePort1 v2_destPort1 set v2_sourceAddress "[expr {$v2_sourceAddress1 & 0xff}].[expr {$v2_sourceAddress2 & 0xff}].[expr {$v2_sourceAddress3 & 0xff}].[expr {$v2_sourceAddress4 & 0xff}]" set v2_destAddress "[expr {$v2_destAddress1 & 0xff}].[expr {$v2_destAddress2 & 0xff}].[expr {$v2_destAddress3 & 0xff}].[expr {$v2_destAddress4 & 0xff}]" set v2_sourcePort [expr {$v2_sourcePort1 & 0xffff}] set v2_destPort [expr {$v2_destPort1 & 0xffff}] log "Proxy Protocol v2 conn from [IP::client_addr]:[TCP::client_port] for an IPv4 Stream from Src: $v2_sourceAddress:$v2_sourcePort to Dst: $v2_destAddress:$v2_destPort" binary scan [TCP::payload] @14S address_size set skip_bytes [expr 16 + $address_size] TCP::payload replace 0 $skip_bytes "" } elseif {$v2_addressFamilyTransportProtocol == 21} { binary scan [TCP::payload] @16H4H4H4H4H4H4H4H4 v2_v6sourceAddress1 v2_v6sourceAddress2 v2_v6sourceAddress3 v2_v6sourceAddress4 v2_v6sourceAddress5 v2_v6sourceAddress6 v2_v6sourceAddress7 v2_v6sourceAddress8 binary scan [TCP::payload] @32H4H4H4H4H4H4H4H4 v2_v6destAddress1 v2_v6destAddress2 v2_v6destAddress3 v2_v6destAddress4 v2_v6destAddress5 v2_v6destAddress6 v2_v6destAddress7 v2_v6destAddress8 binary scan [TCP::payload] @48SS v2_v6sourcePort1 v2_v6destPort1 set v2_v6sourcePort [expr {$v2_v6sourcePort1 & 0xffff}] set v2_v6destPort [expr {$v2_v6destPort1 & 0xffff}] set v2_v6sourceAddress "$v2_v6sourceAddress1:$v2_v6sourceAddress2:$v2_v6sourceAddress3:$v2_v6sourceAddress4:$v2_v6sourceAddress5:$v2_v6sourceAddress6:$v2_v6sourceAddress7:$v2_v6sourceAddress8" set v2_v6destAddress "$v2_v6destAddress1:$v2_v6destAddress2:$v2_v6destAddress3:$v2_v6destAddress4:$v2_v6destAddress5:$v2_v6destAddress6:$v2_v6destAddress7:$v2_v6destAddress8" log "Proxy Protocol v2 conn from from [IP::client_addr]:[TCP::client_port] for an IPv6 Stream from Src: $v2_v6sourceAddress:$v2_v6sourcePort to Dst: $v2_v6destAddress:$v2_v6destPort" binary scan [TCP::payload] @14S address_size set skip_bytes [expr 16 + $address_size] TCP::payload replace 0 $skip_bytes "" } else { log "v2_proxy conn from [IP::client_addr]:[TCP::client_port] - possible unknown/malformed transportProtocol or addressFamily" reject } } elseif {$v2_verCommand == 20}{ log "Proxy Protocol v2 and LOCAL command from [IP::client_addr]:[TCP::client_port]; skipping" binary scan [TCP::payload] @14S address_size set skip_bytes [expr 16 + $address_size] TCP::payload replace 0 $skip_bytes "" binary scan [TCP::payload] H* local_remainder } else { log "Proxy Protocol Protocol Signature Detected from [IP::client_addr]:[TCP::client_port] but protocol version and command not legal; connection reset" reject } } elseif {$static::allowNoProxy} { log "Connection from [IP::client_addr]:[TCP::client_port] allowed despite lack of PROXY protocol header" } else { reject log "Connection rejected from [IP::client_addr]:[TCP::client_port] due to lack of PROXY protocol header" } TCP::release } Tested this on version: 12.0
HTTP To HTTPS Redirect 301
Problem this snippet solves: Redirects all traffic to same hostname, same URI over https by issuing a redirect with status 301 (Moved Permanently). You can change the status code to a 302 to issue a non-cacheable redirect. Apply to HTTP virtual server to redirect all traffic to same hostname (stripping port if it exists), same URI over HTTPS. (Do not apply to shared/wildcard virtual server responding to HTTPS traffic, or infinite redirect will occur. Create separate virtual servers on port 80 and port 443, and apply this iRule ONLY to the port 80 HTTP-only virtual server. No iRule is needed on the port 443 HTTPS virtual server.) How to use this snippet: when HTTP_REQUEST { HTTP::respond 301 Location "https://[getfield [HTTP::host] : 1][HTTP::uri]" } The above rule may be modified to function on a shared virtual server by testing TCP::local_port and redirecting only if the request came in over port 80: when HTTP_REQUEST { if { [TCP::local_port] == 80 }{ HTTP::respond 301 Location "https://[getfield [HTTP::host] : 1][HTTP::uri]" } } Here is another option which handles HTTP requests which don't have a Host header value. In such a case, the VIP's IP address is used for the host in the redirect. Code : when HTTP_REQUEST { # Check if Host header has a value if {[HTTP::host] ne ""}{ # Redirect to the requested host and URI (minus the port if specified) HTTP::respond 301 Location "https://[getfield [HTTP::host] ":" 1][HTTP::uri]" } else { # Redirect to VIP's IP address HTTP::respond 301 Location "https://[IP::local_addr][HTTP::uri]" } }Ultimate irule debug - Capture and investigate
Problem this snippet solves: I decided to share this Irule for different reasons. When I help our community on devcentral, I regularly see people making recurring requests: How do I do to capture the queries header. How do I do to capture the response header. How do I check the information in the POST Request. How do I check response data (body). What cypher/protocol I use (SSL/TLS). I set up client certificate authentication but I do not know if it works and if I pass my certificate auth. I want to retrieve information from my authentication certificate (subject, issuer, …). My authenticating by certificate does not work and I get an error of what I have to do. I have latencies when dealing with my request. where does the latency come from (F5, server,..). I set up sso (kerberos delegation, json post, Form sso). I do not feel that my request is sent to the backend (or the kerberos token). Does F5 add information or modify the request/response. Which pool member has been selected My VS don’t answer (where does the problem come from) … instead of having an Irule for each request why not consolidate everything and provide a compact Irule. this Irule can help you greatly during your investigations and allows you to capture these different items: How to use this snippet: you have a function that allows you to activate the desired logs (1 to activate and 0 to disable) as describe below: array set app_arrway_referer { client_dest_ip_port 1 client_cert 1 http_request 1 http_request_release 1 http_request_payload 0 http_lb_selected 1 http_response 0 http_response_release 0 http_response_payload 0 http_time_process 0 } the posted logs will be preceded by a UID which will allow you to follow from the beginning to the end of the process of your request / answer. you can for example make a grep on the log to follow the complete process (request / answer). the UID is generated in the following way: `set uid [string range [AES::key 256] 15 23] client_dest_ip_port: this section will allow you to see source IP/Port and destination IP/Port. <CLIENT_ACCEPTED>: ----------- client_dest_ip_port ----------- <CLIENT_ACCEPTED>: uid: 382951fe9 - Client IP Src: 10.20.30.4:60419 <CLIENT_ACCEPTED>: uid: 382951fe9 - Client IP Dest:192.168.30.45:443 <CLIENT_ACCEPTED>: ----------- client_dest_ip_port ----------- client_cert: this section will allow you to check the result code for peer certificate verification ( and also if you have provide a certificate auth). moreover you will be able to recover the information of your authentication certficat (issuer, subject, …). if your authentication certificate that you provid is not valid an error message will be returned (ex: certificate chain too long, invalid CA certificate, …). all errors are listed in the link below: https://devcentral.f5.com/wiki/iRules.SSL__verify_result.ashx <HTTP_REQUEST>: ----------- client_cert ----------- <HTTP_REQUEST>: uid: 382951fe9 - cert number: 0 <HTTP_REQUEST>: uid: 382951fe9 - subject: OU=myOu, CN=youssef <HTTP_REQUEST>: uid: 382951fe9 - Issuer Info: DC=com, DC=domain, CN=MobIssuer <HTTP_REQUEST>: uid: 382951fe9 - cert serial: 22:00:30:5c:de:dd:ec:23:6e:b5:e6:77:bj:01:00:00:22:3c:dc <HTTP_REQUEST>: ----------- client_cert ----------- OR <HTTP_REQUEST>: ----------- client_cert ----------- <HTTP_REQUEST>: uid: 382951fe9 - No client certificate provided <HTTP_REQUEST>: ----------- client_cert ----------- http_request: This section allow you to retrieve the complete client HTTP request headers (that is, the method, URI, version, and all headers). I also added the protocol, the ciphers and the name of the vs used. <HTTP_REQUEST>: ----------- http_request ----------- <HTTP_REQUEST>: uid: 382951fe9 - protocol: https <HTTP_REQUEST>: uid: 382951fe9 - cipher name: ECDHE-RSA-AES128-GCM-SHA256 <HTTP_REQUEST>: uid: 382951fe9 - cipher version: TLSv1.2 <HTTP_REQUEST>: uid: 382951fe9 - VS Name: /Common/vs-myapp-443 <HTTP_REQUEST>: uid: 382951fe9 - Request: POST myapp.mydomain.com/browser-management/users/552462/playlist/play/api <HTTP_REQUEST>: uid: 382951fe9 - Host: myapp.mydomain.com <HTTP_REQUEST>: uid: 382951fe9 - Connection: keep-alive <HTTP_REQUEST>: uid: 382951fe9 - Content-Length: 290 <HTTP_REQUEST>: uid: 382951fe9 - Accept: application/json, text/javascript, */*; q=0.01 <HTTP_REQUEST>: uid: 382951fe9 - X-Requested-With: XMLHttpRequest <HTTP_REQUEST>: uid: 382951fe9 - User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36 <HTTP_REQUEST>: uid: 382951fe9 - Referer: https://myapp.mydomain.com/ <HTTP_REQUEST>: uid: 382951fe9 - Accept-Encoding: gzip, deflate, sdch, br <HTTP_REQUEST>: uid: 382951fe9 - Accept-Language: en-US,en;q=0.8 <HTTP_REQUEST>: uid: 382951fe9 - Cookie: RLT=SKjpfdkFDKjkufd976HJhldds=; secureauth=true; STT="LKJSDKJpjslkdjslkjKJSHjfdskjhoLHkjh78dshjhd980szKJH"; ASP.SessionId=dsliulpoiukj908798dsjkh <HTTP_REQUEST>: uid: 382951fe9 - X-Forwarded-For: 10.10.10.22 <HTTP_REQUEST>: ----------- http_request ----------- http_request_release: This section triggered when the system is about to release HTTP data on the serverside of the connection. This event is triggered after modules process the HTTP request. So it will allow you to check request after F5 process. suppose that you have put APM with SSO kerberos, you will be able to see the kerberos token insert by F5. Or XFF insert by HTTP profile… <HTTP_REQUEST_RELEASE>: ----------- http_request_release ----------- <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - VS Name: /Common/vs-myapp-443 <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Request: GET myapp.mydomain.com/browser-management/users/552462/playlist/play/api <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Host: myapp.mydomain.com <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Connection: keep-alive <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Accept: application/json, text/javascript, */*; q=0.01 <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - X-Requested-With: XMLHttpRequest <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36 <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Referer: https://myapp.mydomain.com/ <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Accept-Encoding: gzip, deflate, sdch, br <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Accept-Language: en-US,en;q=0.8 <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - Cookie: RLT=SKjpfdkFDKjkufd976HJhldds=; secureauth=true; STT="LKJSDKJpjslkdjslkjKJSHjfdskjhoLHkjh78dshjhd980szKJH"; ASP.SessionId=dsliulpoiukj908798dsjkh <HTTP_REQUEST_RELEASE>: uid: 382951fe9 - X-Forwarded-For: 10.10.10.22 <HTTP_REQUEST_RELEASE>: ----------- http_request_release ----------- http_request_payload: This section will allow you to retrieve the HTTP request body. <HTTP_REQUEST>: ----------- http_request_payload ----------- <HTTP_REQUEST>: uid: 382951fe9 - Content-Length header null in request If GET or POST withtout content) <HTTP_REQUEST>: ----------- http_request_payload ----------- or <HTTP_REQUEST>: ----------- http_request_payload ----------- <HTTP_REQUEST>: uid: 382951fe9 - post payload: { id: 24, retrive: 'identity', service: 'IT'} <HTTP_REQUEST>: ----------- http_request_payload ----------- http_lb_selected This section will allow you to you to see which pool member has been selected. Once the pool memeber has been selected, you will not see this logs again until another load balancing decision will be made. If you want to see the selected pool memeber for each request you can see this information in "http_response". <HTTP_REQUEST>: ----------- http_lb_selected ----------- <LB_SELECTED>: uid: 382951fe9 - pool member IP: /Common/pool-name 10.22.33.54 443 <HTTP_REQUEST>: ----------- http_lb_selected ----------- http_response: This section will allow you to retrieve the response status and header lines from the server response. You can also see which pool member has been selected. <HTTP_RESPONSE>: ----------- http_response ----------- <HTTP_RESPONSE>: uid: 382951fe9 - status: 200 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - pool member IP: /Common/pool-name 10.22.33.54 443 <HTTP_RESPONSE>: uid: 382951fe9 - Cache-Control: no-cache <HTTP_RESPONSE>: uid: 382951fe9 - Pragma: no-cache <HTTP_RESPONSE>: uid: 382951fe9 - Content-Type: application/json; charset=utf-8 <HTTP_RESPONSE>: uid: 382951fe9 - Expires: -1 <HTTP_RESPONSE>: uid: 382951fe9 - Server: Microsoft-IIS/8.5 <HTTP_RESPONSE>: uid: 382951fe9 - X-Powered-By: ASP.NET <HTTP_RESPONSE>: uid: 382951fe9 - Date: Fri, 28 Oct 2018 06:46:59 GMT <HTTP_RESPONSE>: uid: 382951fe9 - Content-Length: 302 <HTTP_RESPONSE>: ----------- http_response ----------- http_response_release: This section triggered when the system is about to release HTTP data on the clientside of the connection. This event is triggered after modules process the HTTP response. you can make sure that the answer has not been altering after the f5 process. You can also see which pool member has been selected. <HTTP_RESPONSE_RELEASE>: ----------- http_response_release ----------- <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - status: 200 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - pool member IP: /Common/pool-name 10.22.33.54 443 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Cache-Control: no-cache <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Pragma: no-cache <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Content-Type: application/json; charset=utf-8 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Expires: -1 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Server: Microsoft-IIS/8.5 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - X-Powered-By: ASP.NET <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Date: Fri, 28 Oct 2018 06:46:59 GMT <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Content-Length: 302 <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Strict-Transport-Security: max-age=16070400; includeSubDomains <HTTP_RESPONSE_RELEASE>: ----------- http_response_release ----------- http_response_payload: This section will allow you to Collects an amount of HTTP body data that you specify. <HTTP_RESPONSE_DATA>: ----------- http_response_payload ----------- <HTTP_RESPONSE_DATA>: uid: 382951fe9 - Response (Body) payload: { "username" : "youssef", "genre" : "unknown", "validation-factors" : { "validationFactors" : [ { "name" : "remote_address", "value" : "127.0.0.1" } ] }} <HTTP_RESPONSE_DATA>: ----------- http_response_payload ----------- http_time_process: this part will allow you to put back information which can be useful to you to target the latency problematic. it is clear that it is not precise and that f5 offers other tools for that. but you will be able to quickly see which elements take the most time to be processed. you will be able to see how long f5 takes to process the request, the response and how long the backend server takes time to respond. <HTTP_RESPONSE_RELEASE>: ----------- http_time_process ----------- <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Time to request (F5 request time) = 5 (ms) <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Time to response (F5 response time) = 0 (ms) <HTTP_RESPONSE_RELEASE>: uid: 382951fe9 - Time to server (server backend process time) = 4 (ms) <HTTP_RESPONSE_RELEASE>: ----------- http_time_process ----------- Code : when CLIENT_ACCEPTED { # set a unique id for transaction set uid [string range [AES::key 256] 15 23] # set what's you want to retrieve 0 or 1 array set app_arrway_referer { client_dest_ip_port 1 client_cert 1 http_request 1 http_request_release 1 http_request_payload 1 http_lb_selected 1 http_response 1 http_response_release 1 http_response_payload 1 http_time_process 1 } if {$app_arrway_referer(client_dest_ip_port)} { log local0. " ----------- client_dest_ip_port ----------- " clientside { log local0. "uid: $uid - Client IP Src: [IP::client_addr]:[TCP::client_port]" } log local0. "uid: $uid - Client IP Dest:[IP::local_addr]:[TCP::local_port]" log local0. " ----------- client_dest_ip_port ----------- " log local0. " " } } when HTTP_REQUEST { set http_request_time [clock clicks -milliseconds] # Triggered when the system receives a certificate message from the client. The message may contain zero or more certificates. if {$app_arrway_referer(client_cert)} { log local0. " ----------- client_cert ----------- " # SSL::cert count - Returns the total number of certificates that the peer has offered. if {[SSL::cert count] > 0}{ # Check if there was no error in validating the client cert against LTM's server cert if { [SSL::verify_result] == 0 }{ for {set i 0} {$i < [SSL::cert count]} {incr i}{ log local0. "uid: $uid - cert number: $i" log local0. "uid: $uid - subject: [X509::subject [SSL::cert $i]]" log local0. "uid: $uid - Issuer Info: [X509::issuer [SSL::cert $i]]" log local0. "uid: $uid - cert serial: [X509::serial_number [SSL::cert $i]]" } } else { # https://devcentral.f5.com/s/wiki/iRules.SSL__verify_result.ashx (OpenSSL verify result codes) log local0. "uid: $uid - Cert Info: [X509::verify_cert_error_string [SSL::verify_result]]" } } else { log local0. "uid: $uid - No client certificate provided" } log local0. " ----------- client_cert ----------- " log local0. " " } if {$app_arrway_referer(http_request)} { log local0. " ----------- http_request ----------- " if { [PROFILE::exists clientssl] == 1 } { log local0. "uid: $uid - protocol: https" log local0. "uid: $uid - cipher name: [SSL::cipher name]" log local0. "uid: $uid - cipher version: [SSL::cipher version]" } log local0. "uid: $uid - VS Name: [virtual]" log local0. "uid: $uid - Request: [HTTP::method] [HTTP::host][HTTP::uri]" foreach aHeader [HTTP::header names] { log local0. "uid: $uid - $aHeader: [HTTP::header value $aHeader]" } log local0. " ----------- http_request ----------- " log local0. " " } set collect_length_request [HTTP::header value "Content-Length"] set contentlength 1 if {$app_arrway_referer(http_request_payload)} { if { [catch { if { $collect_length_request > 0 && $collect_length_request < 1048577 } { set collect_length $collect_length_request } else { set collect_length 1048576 } if { $collect_length > 0 } { HTTP::collect $collect_length_request set contentlength 1 } }] } { # no DATA in POST Request log local0. " ----------- http_request_payload ----------- " log local0. "uid: $uid - Content-Length header null in request" log local0. " ----------- http_request_payload ----------- " log local0. " " set contentlength 0 } } } when HTTP_REQUEST_DATA { if {$app_arrway_referer(http_request_payload)} { log local0. " ----------- http_request_payload ----------- " if {$contentlength} { set postpayload [HTTP::payload] log local0. "uid: $uid - post payload: $postpayload" #HTTP::release } log local0. " ----------- http_request_payload ----------- " log local0. " " } } when HTTP_REQUEST_RELEASE { if {$app_arrway_referer(http_request_release)} { log local0. " ----------- http_request_release ----------- " if { [PROFILE::exists clientssl] == 1 } { log local0. "uid: $uid - cipher protocol: https" log local0. "uid: $uid - cipher name: [SSL::cipher name]" log local0. "uid: $uid - cipher version: [SSL::cipher version]" } log local0. "uid: $uid - VS Name: [virtual]" log local0. "uid: $uid - Request: [HTTP::method] [HTTP::host][HTTP::uri]" foreach aHeader [HTTP::header names] { log local0. "uid: $uid - $aHeader: [HTTP::header value $aHeader]" } log local0. " ----------- http_request_release ----------- " log local0. " " } set http_request_time_release [clock clicks -milliseconds] } when LB_SELECTED { if {$app_arrway_referer(http_lb_selected)} { log local0. " ----------- http_lb_selected ----------- " log local0. "uid: $uid - pool member IP: [LB::server]" log local0. " ----------- http_lb_selected ----------- " log local0. " " } } when HTTP_RESPONSE { set http_response_time [clock clicks -milliseconds] set content_length [HTTP::header "Content-Length"] if {$app_arrway_referer(http_response)} { log local0. " ----------- http_response ----------- " log local0. "uid: $uid - status: [HTTP::status]" log local0. "uid: $uid - pool member IP: [LB::server]" foreach aHeader [HTTP::header names] { log local0. "uid: $uid - $aHeader: [HTTP::header value $aHeader]" } log local0. " ----------- http_response ----------- " log local0. " " } if {$app_arrway_referer(http_response_payload)} { if { $content_length > 0 && $content_length < 1048577 } { set collect_length $content_length } else { set collect_length 1048576 } if { $collect_length > 0 } { HTTP::collect $collect_length } } } when HTTP_RESPONSE_DATA { if {$app_arrway_referer(http_response_payload)} { log local0. " ----------- http_response_payload ----------- " set payload [HTTP::payload] log local0. "uid: $uid - Response (Body) payload: $payload" log local0. " ----------- http_response_payload ----------- " log local0. " " } } when HTTP_RESPONSE_RELEASE { set http_response_time_release [clock clicks -milliseconds] if {$app_arrway_referer(http_response_release)} { log local0. " ----------- http_response_release ----------- " log local0. "uid: $uid - status: [HTTP::status]" log local0. "uid: $uid - pool member IP: [LB::server]" foreach aHeader [HTTP::header names] { log local0. "uid: $uid - $aHeader: [HTTP::header value $aHeader]" } log local0. " ----------- http_response_release ----------- " log local0. " " } if {$app_arrway_referer(http_time_process)} { log local0. " ----------- http_time_process ----------- " log local0.info "uid: $uid - Time to request (F5 request time) = [expr $http_request_time - $http_request_time_release] (ms)" log local0.info "uid: $uid - Time to response (F5 response time) = [expr $http_response_time - $http_response_time_release] (ms)" log local0.info "uid: $uid - Time to server (server backend process time) = [expr $http_request_time_release - $http_response_time] (ms)" log local0. " ----------- http_time_process ----------- " log local0. " " } } Tested this on version: 13.0
Google Authenticator Token Verification iRule For APM
Problem this snippet solves: This iRule adds token authentication capabilities for Google Authenticator to APM. The implementation is described in George Watkins' article: Two Factor Authentication With Google Authenticator And APM The iRule should be applied to an access policy-enabled virtual server. In order to provide two-factor authentication, a AAA server must be defined to verify user credentials. The users' Google Authenticator secrets can be mapped to individual users using a data group, an LDAP schema attribute, or an Active Directory attribute. The storage method can be defined in the beginning section of the iRule. Here are a list of all the configurable options: lockout_attempts - number of attempts a user is allowed to make prior to being locked out temporarily lockout_period - duration of lockout period ga_code_form_field - name of HTML form field used in the APM logon page, this field is define in the "Logon Page" access policy object ga_key_storage - key storage method for users' Google Authenticator shared keys, valid options include: datagroup, ldap, or ad ga_key_ldap_attr - name of LDAP schema attribute containing users' key ga_key_ad_attr - name of Active Directory schema attribute containing users' key ga_key_dg - data group containing user := key mappings Code : when ACCESS_POLICY_AGENT_EVENT { if { [ACCESS::policy agent_id] eq "ga_code_verify" } { ### Google Authenticator verification settings ### # lock the user out after x attempts for a period of x seconds set static::lockout_attempts 3 set static::lockout_period 30 # logon page session variable name for code attempt form field set static::ga_code_form_field "ga_code_attempt" # key (shared secret) storage method: ldap, ad, or datagroup set static::ga_key_storage "datagroup" # LDAP attribute for key if storing in LDAP (optional) set static::ga_key_ldap_attr "google_auth_key" # Active Directory attribute for key if storing in AD (optional) set static::ga_key_ad_attr "google_auth_key" # datagroup name if storing key in a datagroup (optional) set static::ga_key_dg "google_auth_keys" ##################################### ### DO NOT MODIFY BELOW THIS LINE ### ##################################### # set lockout table set static::lockout_state_table "[virtual name]_lockout_status" # set variables from APM logon page set username [ACCESS::session data get session.logon.last.username] set ga_code_attempt [ACCESS::session data get session.logon.last.$static::ga_code_form_field] # retrieve key from specified storage set ga_key "" switch $static::ga_key_storage { ldap { set ga_key [ACCESS::session data get session.ldap.last.attr.$static::ga_key_ldap_attr] } ad { set ga_key [ACCESS::session data get session.ad.last.attr.$static::ga_key_ad_attr] } datagroup { set ga_key [class lookup $username $static::ga_key_dg] } } # increment the number of login attempts for the user set prev_attempts [table incr -notouch -subtable $static::lockout_state_table $username] table timeout -subtable $static::lockout_state_table $username $static::lockout_period # verification result value: # 0 = successful # 1 = failed # 2 = no key found # 3 = invalid key length # 4 = user locked out # make sure that the user isn't locked out before calculating GA code if { $prev_attempts <= $static::lockout_attempts } { # check that a valid key was retrieved, then proceed if { [string length $ga_key] == 16 } { # begin - Base32 decode to binary # Base32 alphabet (see RFC 4648) array set static::b32_alphabet { A 0 B 1 C 2 D 3 E 4 F 5 G 6 H 7 I 8 J 9 K 10 L 11 M 12 N 13 O 14 P 15 Q 16 R 17 S 18 T 19 U 20 V 21 W 22 X 23 Y 24 Z 25 2 26 3 27 4 28 5 29 6 30 7 31 } set ga_key [string toupper $ga_key] set l [string length $ga_key] set n 0 set j 0 set ga_key_bin "" for { set i 0 } { $i < $l } { incr i } { set n [expr $n << 5] set n [expr $n + $static::b32_alphabet([string index $ga_key $i])] set j [incr j 5] if { $j >= 8 } { set j [incr j -8] append ga_key_bin [format %c [expr ($n & (0xFF << $j)) >> $j]] } } # end - Base32 decode to binary # begin - HMAC-SHA1 calculation of Google Auth token set time [binary format W* [expr [clock seconds] / 30]] set ipad "" set opad "" for { set j 0 } { $j < [string length $ga_key_bin] } { incr j } { binary scan $ga_key_bin @${j}H2 k set o [expr 0x$k ^ 0x5C] set i [expr 0x$k ^ 0x36] append ipad [format %c $i] append opad [format %c $o] } while { $j < 64 } { append ipad 6 append opad \\ incr j } binary scan [sha1 $opad[sha1 ${ipad}${time}]] H* token # end - HMAC-SHA1 calculation of Google Auth hex token # begin - extract code from Google Auth hex token set offset [expr ([scan [string index $token end] %x] & 0x0F) << 1] set ga_code [expr (0x[string range $token $offset [expr $offset + 7]] & 0x7FFFFFFF) % 1000000] set ga_code [format %06d $ga_code] # end - extract code from Google Auth hex token if { $ga_code_attempt eq $ga_code } { # code verification successful set ga_result 0 } else { # code verification failed set ga_result 1 } } elseif { [string length $ga_key] > 0 } { # invalid key length, greater than 0, but not length not equal to 16 chars set ga_result 3 } else { # could not retrieve user's key set ga_result 2 } } else { # user locked out due to too many failed attempts set ga_result 4 } # set code verification result in session variable ACCESS::session data set session.custom.ga_result $ga_result } }
