Generic iRule based on datagroup parsing

User Groups Community Contests Contest Entries
,
1

Problem

This iRule is derived from a customer’s migration project (anonymized and cut from some content). This customer needed to migrate hundreds of Apache virtual hosts to BIG-IP. From this big project, with many other intrications and integrations, we can get the following requirements:

  • Integrates with all Apache features used before the migration (that were not necessarily all assessed, and should easily be adaptable in case)
  • Should seamlessly allow for modifying, adding or removing services, with minimum overhead in object creation (control plane very solicited for other purposes and non-F5 friendly persons contributing, such as app owners)
  • Should prevent explosion of VIP and virtual servers and perform most routing on a single virtual server.
  • Should allow application owners involvement in complex behaviours setup without having them modifying code and limiting their impact on the device.

This iRule therefore aims to control most of the customer’s ingress traffic routing strategy based on a generic iRule actionable through entries in a datagroup that could be contributed to by application owners or non-F5 friendly persons, and even now contributed to through a simple form outputting this datagroup entry.

Solution

As you will see, this iRule is quite long as it delves into many details, verbosely logs and deals with many edge cases. However, we can easily summarize its principle:

When receiving a request, this iRule:

  1. Checks if the <FQDN/URI> matches a key in a redirect datagroup
    1. If it does, parses the value as a comma separated entry to fetch the redirection parameter
    2. Executes redirect and returns
  2. Checks if the <FQDN/URI> matches a key in a pool datagroup
    1. If it does, parses the value as a comma separated entry and handles various functions, based on pool attribution
    2. You can see details of the handling of the different under the specified functions (e.g. see section from line 956 to know more about the error pages handling)

Here are comments found in the code, describing this behaviour:

########## REDIRECT DG EXPLANATIONS ##############
        # 0 key: hostname/path (stars_with)  ou EQ:hostname/path (equals)
        ##################################################
        # 1 comment
        # 2 s=static routing , d=dynamic routing
        # 3 host and path to route to 
        # 4 redirect code ( 301,302,... )
        #
        # examples
        # static redirect :     key:site1.customer.ch/redir1/    value TF,s,www.customer.ch/r1/,301
        # static redirect :     key:site1.customer.ch/redir1/     value TEMPO,s,www.customer.ch/r2/,302
        # dynamic redirect :    key:site2.customer.ch/dynamic1/   value tf,d,www.customer.ch/d1/,302 ( /dynamic1/toto... -> /d1/toto...
        ##################################################


########## POOL DG EXPLANATIONS ##############
            # 0 key= /path
            ##################################################
            # comma separated entries :
            # 1 comment
            # 2 poolname or vs=vsname
            # 3 1->ssl, 0->nossl
            #
            # 4 s=rewrite host (static) , d=rewrite host (dynamic ), 0=dont rewrite host
            #   examples :
            #   s=newhost
            #   d=;ip1:newhost1;ip2:newhost2
            #   0 ou norewrite ou autre -> no rewrite
            #
            # 5 dg=specific rewrite -> check specific rewrite in DG ,  0 ou nodg -> no rewrtie
            #   examples
            #   dg=dg-COMPANY-specific-application1
            #   nodg ou autre
            #   example of specific DataGroup (4 lines )
            #   Attention : 
            #       request format is a pair of uri , space separated ( clientsenturi srvreceiveduri clientsenturi2 srvreceiveduri2 )
            #       response format is pair of uri, @ separated (@srvsendthisuri@clientreceivethis@ @srv2@client2@ ...)
            #       the the orider is reversed, and the syntax is different
            #   ContentTypeRewriteExclude xml
            #   ContentTypeRewriteInclude json
            #   request /rewrite1 /rewritten1 /torewrite2 /rewrittent2
            #   response @/srv1@/client1@ @/srv2@/client22@ @/rewritten2@/torewrite2@ 
            #
            # 6 various headers=ATTENTION ^ separated
            #   headerName=<action direction>
            #   action = i(insert), d(delete) ou r(replace)
            #   direction = q(reQuest),s(reSponse) ou b(Both side)
            #   :value(ATTENTION ; can be part of the value)
            #   examples of 3 headers : X-TEST1=iq:myvalue1^X-TEST2=rs:myvalue2;myvalue2bis^X-SRV=db
            #   Each ',', '^', '=' and ':' char should be replaced by '#commaCOMPANY#', '#HatCOMPANY#', '#equalsCOMPANY#' and '#colonCOMPANY#' in the header value
            #
            # 7 snat=<specific snat pool> .if empty : the default as define on the VS
            #   example : snat=snatpool-specific1
            #
            # 8 persist=<source IP specific>.if empty : the default as define on the VS
            #   example : persist=24=10  ( 24=mask (default=32), 10 = timeout (default=180))
            #
            # 9 errorpage message for 4* and 5* :
            #   - Syntaxe : list of <code>=<value> separated by ";". Example: 404=k;502=d;503=Message d'erreur specific<br>sur 2 lignes
            #   - If nothing defined for reponse code : uses error page strategy :
            #       - if env is prod, prodext or recext : rewrite all responses. Keep status if 4** and rewrite to 404 if 5**
            #       - if env is dev or rec : keep backend server response
            #   - If value = "k" : keep backend server response
            #   - If value = "d" : rewrite response page, use default error message and keep reponse status code
            #   - If value = <specific message> : rewrite reponse page, use <specific message> for error message and keep reponses status code  
            #   Each ',', ';' and '=' chars should be replaced by '#commaCOMPANY#', '#semiCOMPANY#' and '#equalsCOMPANY#' in the <specific message>
            #
            # 10 cookie handling : httponly;secure;samesite=Lax,Strict ou None. 
            #   example : httponly;samesite=Lax
            #           secure;samesite=Strict
            #
            # 11 cors allowed domains :
            #   - Syntaxe : example.customer.ch *.customer.ch
            #   - If empty : do nothing
            #   - If not empty:
            #       - If there is a match :
            #           - Respond to OPTIONS request with Access-Control-Request-Method header.
            #           - For other requests that contain Origin and Access-Control-Request-Method headers, it adds CORS headers to response 
            #       - If there is no match : do nothing
            #
            # 12 manage decompression and compression :
            #   - Syntaxe : list of number separated by ";". Example: 1;2;3
            #   - If empty : default to 0
            #   - If 0 : Do nothing
            #   - If 1 : Decompress gzip encoded request body
            #   - If 2 : Compress request
            #   - If 3 : Compress response
            #
            # 13 Geo IP location restriction :
            #   - Syntaxe : geoip profile defined in datagroup dg-ltm-geoip_profile by the F5 Team
            #   - If empty : do nothing
            #   - If defined : 
            #       - check the location of the IP via lookup in the geoip location database and block the connection if the country is not allowed to access the uri
			#	- Data Group Construction : 
			#		- key: geoip profile name -> usa_canada_and_europe
			#		- data: allowed country and continent -> US CA;EU
            #
            ##################################################################################

Impact

With this solution, we were able to provide the customer with a fully controllable solution allowing to trigger complex behaviours from a datagroup entry stored in a “CSV like” way. This allows to make this accessible to persons not familiar with the way the F5 works. Today, this approach even integrates with upfront application provisioning, as we can naturally derive a form based description of new applications that can be transformed into this CSV entry and extremely easily allow to add new applications.

Also, this iRules is coded such that appending new entries can easily be done to add new and more complex functionalities at almost no cost for existing applications, making it virtually infinitely expandable for adding new features. For example, GeoIP filtering control was added long after the migration project and seamlessly integrated with these existing features.

Code

when RULE_INIT {
    ###############################
    # Main irule for COMPANY/Customer
    # Redirect, select pool or vs, rewrite, ... All actions are made based on data found in various datagroups
    # The datagroup key is either fqdn/uri for redirect and debug (single datagroup for each)
    # The datagroup key can also be /uri for pools or vs (the datagroup name contains the host)
    #
    # F.Zeller - eXpert Solutions
    ############################

    ############################
    # Prefix in the logs
    set static::COMPANY_main_prefix "COMPANY-V1.20n"
    # Debug SSL session : 0=no, 1=yes
    set static::COMPANY_ssl_debug "0"
    # 1=LTM Policy  is still in use : do nothing if traffic doesn't match anything in DG.  0=No more LTM Policy: sending a 404 if no match in DG
    set static::COMPANY_main_do_nothing_if_no_match 1
    # Data group names. Warning: Those datagroups must exist. No check is performed.
    set static::COMPANY_main_DG_apm "dg-ltm-tf-global-apm"
    set static::COMPANY_main_DG_redirect "dg-ltm-tf-global-redirect"
    set static::COMPANY_main_DG_pool "dg-ltm-tf-pool" 
    set static::COMPANY_main_DG_debug "dg-ltm-tf-global-debug"

    set static::COMPANY_dg_geoip_profile "dg-ltm-geoip_profile"

    set static::COMPANY_dg_geoip_whitelist "dg-ltm-geoip_whitelist"

    set static::local_ip_range {
        "10.0.0.0/8"
        "192.168.0.0/16"
    }
}


when CLIENT_ACCEPTED priority 5 {
    # Set a unique id for transaction
    set uid [string range [AES::key 256] 15 23]
    # For debug, cannot log the debug base on uri in client_accepted
    set clientaccepted 1
    ACCESS::restrict_irule_events disable

    set ip_client_addr [IP::client_addr]

    # Set error_page_strat variable from wam_env set in a wam irule. Default is strategy 1 (prod and recext)
    set error_page_strat 2
    if { [info exists wam_env] && ($wam_env == "dev" || $wam_env == "rec") } {
        set error_page_strat 2
    }
}

when CLIENTSSL_HANDSHAKE {
    if { [SSL::extensions exists -type 0] } {
        set tls_sni_extension [SSL::extensions -type 0]
    }
    if { $static::COMPANY_ssl_debug } { log -noname local0. "$static::COMPANY_main_prefix [TCP::client_port] :: RSA Session-ID:[SSL::sessionid] Master-Key:[SSL::sessionsecret]" }
}

when SERVERSSL_HANDSHAKE {
    if { $static::COMPANY_ssl_debug } { log -noname local0. "$static::COMPANY_main_prefix [TCP::client_port] :: RSA Session-ID:[SSL::sessionid] Master-Key:[SSL::sessionsecret]" }
}

when SERVERSSL_CLIENTHELLO_SEND {
    if { [info exists newhost] && $newhost != "" } {
        set sni_value $newhost
        SSL::extensions insert [binary format SSScSa* 0 [expr { [set sni_length [string length $sni_value]] + 5 }] [expr { $sni_length + 3 }] 0 $sni_length $sni_value]
        if { $static::COMPANY_ssl_debug } { log -noname local0. "$static::COMPANY_main_prefix - Inject Hostname $sni_value into SNI of Server SSL Handshake" }
    } elseif { [info exists tls_sni_extension] } {
        SSL::extensions insert $tls_sni_extension
        if { $static::COMPANY_ssl_debug } { log -noname local0. "$static::COMPANY_main_prefix - Inject SNI from client SSL Handshake into SNI of Server SSL Handshake" }
    }
}

when HTTP_REQUEST {

    set requestreturn 0
    set path [string tolower [HTTP::path]]
    set newhost ""
    
    # This variable will be use to manage trailing / in request url during class match. 
    set pathwithslash ""
    if { $path ends_with "/" } {
        set pathwithslash $path
    } else {
        set pathwithslash "$path/"
    }

    # Corporate CSS settings for ASM response pages
    if { [URI::basename [HTTP::uri]] equals "waf_corp_style.css" } {
        HTTP::respond 200 content [ifile get waf_corp_style.css] noserver Content-Type "text/css" Connection Close
        event disable all
        return
    }

    if { [class match -- $path starts_with "$static::COMPANY_main_DG_apm"]} {
        set requestreturn 1
        set debughttp_lb_selected 0
        set debughttp_time_process 0
        return
    }

    set ltm_tf_host [string tolower [HTTP::host]]
    set ltm_tf_host [getfield $ltm_tf_host ":" 1]
    set uri [string tolower [HTTP::uri]]

    set VSIP [IP::local_addr]
    set poolname ""

    ########## BEGIN DEBUG CONFIG ##############
    # Debug based on DG
    # Exemple: www.customer.ch/debug1 := ipfilter=10.10.14.121/32,ipport=1,cert=0,request=1,request_rel=1,request_payload=1,lbselected=1,response=1,response_rel=1,responsepay=1,time=1,
    #   - For subnet:           ipfilter=10.10.0.0/16,...
    #   - No filter:            ipfilter=,... or ,...
    #   - Multiple filter:      ipfilter=10.10.0.0/16;10.11.12.0/24,...
    #   - All logs, no filter:  =,=1,=1,=1,=1,=1,=1,=1,=1,=1,=1
    #############################################
    set debug 0
    set debugipfilter ""
    set configdebug ""
    
    # This first condition allows to verify if host/path exists in debug DG and check if we don't have the "trailing slash" problem
    if { ([class match -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_debug]) } {
        set configdebug [class match -value -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_debug]
        set configdebugkey [class match -name -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_debug]
        if { not ($configdebugkey ends_with "/") } {
            set configdebugkeywithslash "$configdebugkey/"
            if { not ("$ltm_tf_host$pathwithslash" starts_with $configdebugkeywithslash) } {
                set configdebug ""
            }
        }
    }
    if { not ($configdebug eq "") } {
        set debugipfilter [getfield [getfield $configdebug "," 1] = 2]
        if { $debugipfilter eq ""} {
            set debug 1
        } else {
            if { $debugipfilter contains ";"} {
                set ipfilter [split $debugipfilter ";"]                
                if { [catch {
                    foreach ip $ipfilter {
                        if { [IP::addr [IP::client_addr] equals $ip] } { 
                            set debug 1
                            break
                        }
                    }
                }] } {}
            } else {
                if { [catch {
                    if { [IP::addr [IP::client_addr] equals $debugipfilter] } { 
                        set debug 1
                    }
                }] } {}
            }
        }
    }
    if { $debug } {
        set debugclient_dest_ip_port [getfield [getfield $configdebug "," 2] = 2]
        set debugclient_cert  [getfield [getfield $configdebug "," 3] = 2]
        set debughttp_request [getfield [getfield $configdebug "," 4] = 2]
        set debughttp_request_release [getfield [getfield $configdebug "," 5] = 2]
        set debughttp_request_payload [getfield [getfield $configdebug "," 6] = 2]
        set debughttp_lb_selected [getfield [getfield $configdebug "," 7] = 2]
        set debughttp_response [getfield [getfield $configdebug "," 8] = 2]
        set debughttp_response_release [getfield [getfield $configdebug "," 9] = 2]
        set debughttp_response_payload [getfield [getfield $configdebug "," 10] = 2]
        set debughttp_time_process [getfield [getfield $configdebug "," 11] = 2]
        set http_request_time [clock clicks -milliseconds]
    } else {
        set debugclient_dest_ip_port 0
        set debugclient_cert  0
        set debughttp_request 0
        set debughttp_request_release 0
        set debughttp_request_payload 0
        set debughttp_lb_selected 0
        set debughttp_response 0
        set debughttp_response_release 0
        set debughttp_response_payload 0
        set debughttp_time_process 0
    }
    ########## END DEBUG CONFIG ##############
    ########## BEGIN DEBUG HTTP_REQUEST ##############
    if { ($debugclient_dest_ip_port eq "1") && ($clientaccepted)} {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- client_dest_ip_port -----------"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  Client IP Src: [IP::client_addr]:[TCP::client_port]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  Client IP Dest:[IP::local_addr]:[TCP::local_port]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- client_dest_ip_port -----------"
        set clientaccepted 0
    }

    if { $debugclient_cert eq "1" } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- client_cert -----------"
        # SSL::cert count - Returns the total number of certificates that the peer has offered.
        if { [SSL::cert count] > 0 } {
            # Check if there was no error in validating the client cert against LTM's server cert
            if { [SSL::verify_result] == 0 }{
                for {set i 0} {$i < [SSL::cert count]} {incr i}{
                    log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cert number: $i"
                    log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cert subject: [X509::subject [SSL::cert $i]]"
                    log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cert Issuer Info: [X509::issuer [SSL::cert $i]]"
                    log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cert serial: [X509::serial_number [SSL::cert $i]]"
                }
            } else {
                # https://devcentral.f5.com/s/wiki/iRules.SSL__verify_result.ashx (OpenSSL verify result codes)
                log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cert Info: [X509::verify_cert_error_string [SSL::verify_result]]"
            }
        } else {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - No client certificate provided"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- client_cert -----------"
    }

    if { $debughttp_request eq "1"} {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix ----------- http_request ----------- "
        if { [PROFILE::exists clientssl] == 1 } {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Protocol: https"
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cipher name: [SSL::cipher name]"
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cipher version: [SSL::cipher version]"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - VS Name: [virtual]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Request: [HTTP::method] [HTTP::host][HTTP::uri]"
        foreach aHeader [HTTP::header names] {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $aHeader: [HTTP::header value $aHeader]"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix ----------- http_request ----------- "
    }

    if { $debughttp_request_payload } {
        set collect_length_request [HTTP::header value "Content-Length"]
        set contentlength 1
        if { [catch {
            if { $collect_length_request > 0 && $collect_length_request < 1048577 } {
                set collect_length $collect_length_request
            } else {
                set collect_length 1048576
            } 
            if { $collect_length > 0 } {
                HTTP::collect $collect_length_request
                set contentlength 1
            }
        }] } {

            # no DATA in POST Request
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix ----------- http_request_payload ----------- "
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Content-Length header null in request"
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix ----------- http_request_payload ----------- "
            set contentlength 0
        }
    }
    ########## END DEBUG HTTP_REQUEST ##############

    ####################################################################
    ######### First - Checking in redirect Data Group ##################
    ####################################################################
    set config ""
    set configkey ""
    
    # This condition allows to verify if host/path exists in redirect DG and check if we don't have the "trailing slash" problem
    if { ([class match -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_redirect]) } {
        set config [class match -value -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_redirect]
        set configkey [class match -name -- $ltm_tf_host$pathwithslash starts_with $static::COMPANY_main_DG_redirect]
        if { not ($configkey ends_with "/") } {
            set configkeywithslash "$configkey/"
            if { not ("$ltm_tf_host$pathwithslash" starts_with $configkeywithslash) } {
                set config ""
            }
        }
        # if redirect config is / to /<new uri>, check if actual uri begins by /<new uri>, if so, do not redirect
        if { [findstr $configkey "/"] == "/" && $pathwithslash starts_with [findstr [getfield $config "," 3] "/"] } {
            set config ""
        }
    } elseif { [class match -- "EQ:$ltm_tf_host$path" equals $static::COMPANY_main_DG_redirect] } {
        set config [class match -value -- "EQ:$ltm_tf_host$path" equals $static::COMPANY_main_DG_redirect]
    }

    if { not ($config eq "") } {

        if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Found $ltm_tf_host$path in redirect DG" }
        ########## REDIRECT DG EXPLANATIONS ##############
        # 0 key: hostname/path (stars_with)  ou EQ:hostname/path (equals)
        ##################################################
        # 1 comment
        # 2 s=static routing , d=dynamic routing
        # 3 host and path to route to 
        # 4 redirect code ( 301,302,... )
        #
        # examples
        # static redirect :     key:site1.customer.ch/redir1/    value TF,s,www.customer.ch/r1/,301
        # static redirect :     key:site1.customer.ch/redir1/     value TEMPO,s,www.customer.ch/r2/,302
        # dynamic redirect :    key:site2.customer.ch/dynamic1/   value tf,d,www.customer.ch/d1/,302 ( /dynamic1/toto... -> /d1/toto...
        ##################################################  

        set comment [getfield $config "," 1]
        set action [getfield $config "," 2]
        set dest [getfield $config "," 3]
        set dest [string map {"#amperCOMPANY#" "&"} $dest]
        set code [getfield $config "," 4]

        switch -- $action {
            s {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Static redirect $ltm_tf_host$path to $code $dest" }
                set redirect_location "https://$dest"
                if {[HTTP::query] ne ""} {
                     set redirect_location "https://$dest?[HTTP::query]"
                }
                HTTP::respond $code Location $redirect_location Connection Close
                event disable all
                return
            }
            d {
                #set newuri [string map -nocase [list "[findstr $configkey "/"]" ""] [HTTP::uri]]
                set configkeyuri [findstr $configkey "/"]
                set newuri [call ci_replace [HTTP::uri] $configkeyuri ""]
                set desthosturi [string map { "//" "/" } "$dest$newuri"]

                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Dynamic redirect $ltm_tf_host$uri to $code $desthosturi" }
                HTTP::respond $code Location "https://$desthosturi" Connection Close
                event disable all
                return
            }
            default {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Unknown action $action in redirect DG" }
                call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
                return
            }
        }

    } else {
        #########################################################################
        ####### Second - Not found in redirectt DG. Try to find in pool DG ######
        #########################################################################

        # Check if the needed DG exists
        if { not [class exists $static::COMPANY_main_DG_pool-$VSIP-$ltm_tf_host] } {
            ### Cannot find the VIP_HOST DG. Either relying of LTM Policy (do nothing) or replying with a 404
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - DG $static::COMPANY_main_DG_pool-$VSIP-$ltm_tf_host unknown" }
            if { $static::COMPANY_main_do_nothing_if_no_match } {
                set poolname ""
                set requestreturn 1
                set debughttp_time_process 0
            } else {
                call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
            }
            return
        }

        # This condition allows to verify if host/path exists in redirect DG and check if we don't have the "trailing slash" problem
        if { ([class match -- $pathwithslash starts_with $static::COMPANY_main_DG_pool-$VSIP-$ltm_tf_host]) } {
            set config [class match -value -- $pathwithslash starts_with $static::COMPANY_main_DG_pool-$VSIP-$ltm_tf_host]
            set configkey [class match -name -- $pathwithslash starts_with $static::COMPANY_main_DG_pool-$VSIP-$ltm_tf_host]
            if { not ($configkey ends_with "/") } {
                set configkeywithslash "$configkey/"
                if { not ($pathwithslash starts_with $configkeywithslash) } {
                    set config ""
                }
            }
        }

        if { not ($config eq "") } {
            ########## POOL DG EXPLANATIONS ##############
            # 0 key= /path
            ##################################################
            # comma separated entries :
            # 1 comment
            # 2 poolname or vs=vsname
            # 3 1->ssl, 0->nossl
            #
            # 4 s=rewrite host (static) , d=rewrite host (dynamic ), 0=dont rewrite host
            #   examples :
            #   s=newhost
            #   d=;ip1:newhost1;ip2:newhost2
            #   0 ou norewrite ou autre -> no rewrite
            #
            # 5 dg=specific rewrite -> check specific rewrite in DG ,  0 ou nodg -> no rewrtie
            #   examples
            #   dg=dg-COMPANY-specific-application1
            #   nodg ou autre
            #   example of specific DataGroup (4 lines )
            #   Attention : 
            #       request format is a pair of uri , space separated ( clientsenturi srvreceiveduri clientsenturi2 srvreceiveduri2 )
            #       response format is pair of uri, @ separated (@srvsendthisuri@clientreceivethis@ @srv2@client2@ ...)
            #       the the orider is reversed, and the syntax is different
            #   ContentTypeRewriteExclude xml
            #   ContentTypeRewriteInclude json
            #   request /rewrite1 /rewritten1 /torewrite2 /rewrittent2
            #   response @/srv1@/client1@ @/srv2@/client22@ @/rewritten2@/torewrite2@ 
            #
            # 6 various headers=ATTENTION ^ separated
            #   headerName=<action direction>
            #   action = i(insert), d(delete) ou r(replace)
            #   direction = q(reQuest),s(reSponse) ou b(Both side)
            #   :value(ATTENTION ; can be part of the value)
            #   examples of 3 headers : X-TEST1=iq:myvalue1^X-TEST2=rs:myvalue2;myvalue2bis^X-SRV=db
            #   Each ',', '^', '=' and ':' char should be replaced by '#commaCOMPANY#', '#HatCOMPANY#', '#equalsCOMPANY#' and '#colonCOMPANY#' in the header value
            #
            # 7 snat=<specific snat pool> .if empty : the default as define on the VS
            #   example : snat=snatpool-specific1
            #
            # 8 persist=<source IP specific>.if empty : the default as define on the VS
            #   example : persist=24=10  ( 24=mask (default=32), 10 = timeout (default=180))
            #
            # 9 errorpage message for 4* and 5* :
            #   - Syntaxe : list of <code>=<value> separated by ";". Example: 404=k;502=d;503=Message d'erreur specific<br>sur 2 lignes
            #   - If nothing defined for reponse code : uses error page strategy :
            #       - if env is prod, prodext or recext : rewrite all responses. Keep status if 4** and rewrite to 404 if 5**
            #       - if env is dev or rec : keep backend server response
            #   - If value = "k" : keep backend server response
            #   - If value = "d" : rewrite response page, use default error message and keep reponse status code
            #   - If value = <specific message> : rewrite reponse page, use <specific message> for error message and keep reponses status code  
            #   Each ',', ';' and '=' chars should be replaced by '#commaCOMPANY#', '#semiCOMPANY#' and '#equalsCOMPANY#' in the <specific message>
            #
            # 10 cookie handling : httponly;secure;samesite=Lax,Strict ou None. 
            #   example : httponly;samesite=Lax
            #           secure;samesite=Strict
            #
            # 11 cors allowed domains :
            #   - Syntaxe : example.customer.ch *.customer.ch
            #   - If empty : do nothing
            #   - If not empty:
            #       - If there is a match :
            #           - Respond to OPTIONS request with Access-Control-Request-Method header.
            #           - For other requests that contain Origin and Access-Control-Request-Method headers, it adds CORS headers to response 
            #       - If there is no match : do nothing
            #
            # 12 manage decompression and compression :
            #   - Syntaxe : list of number separated by ";". Example: 1;2;3
            #   - If empty : default to 0
            #   - If 0 : Do nothing
            #   - If 1 : Decompress gzip encoded request body
            #   - If 2 : Compress request
            #   - If 3 : Compress response
            #
            # 13 Geo IP location restriction :
            #   - Syntaxe : geoip profile defined in datagroup dg-ltm-geoip_profile by the F5 Team
            #   - If empty : do nothing
            #   - If defined : 
            #       - check the location of the IP via lookup in the geoip location database and block the connection if the country is not allowed to access the uri
			#	- Data Group Construction : 
			#		- key: geoip profile name -> usa_canada_and_europe
			#		- data: allowed country and continent -> US CA;EU
            #
            ##################################################################################

            set comment [getfield $config "," 1]
            set poolname [getfield $config "," 2]
            set needssl [getfield $config "," 3]
            set rewritehost [getfield $config "," 4]
            set needrewrite [getfield $config "," 5]
            set headerlist [getfield $config "," 6]
            set specificsnat [getfield $config "," 7]
            set specificpersist [getfield $config "," 8]
            set geoip_profile [getfield $config "," 13]


            if {[call :is_whitelisted $ip_client_addr $static::local_ip_range] or [class match -- $ip_client_addr starts_with $static::COMPANY_dg_geoip_whitelist]} {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $ltm_tf_host$path - Client IP $ip_client_addr is whitelisted."}
            } else {
                if {[info exists geoip_profile] && $geoip_profile != "" && [class exists $static::COMPANY_dg_geoip_profile] } {
                
                    if { ([class match -- $geoip_profile starts_with $static::COMPANY_dg_geoip_profile]) } {
                    
                        set country_list [getfield [class match -value -- $geoip_profile starts_with $static::COMPANY_dg_geoip_profile] ";" 1]
                        
                        set continent_list [getfield [class match -value -- $geoip_profile starts_with $static::COMPANY_dg_geoip_profile] ";" 2]

                        set client_ip_country [whereis $ip_client_addr country]

                        set client_ip_continent [whereis $ip_client_addr continent]

                        if { not ($country_list contains $client_ip_country or $continent_list contains $client_ip_continent) } {
                            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $ltm_tf_host$path - Client IP $ip_client_addr is in a forbidden country $client_ip_continent $client_ip_country from profile $geoip_profile  GEO-$uid."
                            call error_respond 403 "ifile-ltm-tf-403-geoip.html" "" $error_page_strat $ip_client_addr $uid
                            return
                        }
                    }
                }
            }

            ##### Section 3 - SSL or not ########
            if { [info exists needssl] && !($needssl) } {
                SSL::disable serverside
            }

            ##### END Section 3 - SSL or not ########

            set specificirule [getfield $poolname ";" 2]

            if {$specificirule != ""} {
                set resp ""
                set callirulehttprequest "${specificirule}::modify_http_request"
                if { [catch {
                    set resp [call $callirulehttprequest $debughttp_request $uid]
                }] } {
                    if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Unknown irule $specificirule" }
                    call error_respond 404 "ifile-ltm-tf-404.html" "irule $specificirule not found" $error_page_strat $ip_client_addr $uid
                    return
                }
                if {$resp eq "return"} {
                    return
                }
            }

            set selectpool [getfield $poolname ";" 1]

            # Selecting the pool
            if { [catch {
                pool $selectpool
            }] } {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Unknown pool $selectpool" }
                call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
                return
            }
            
        } else {
            # Cannot find host/uri in the VIP_HOST DG. Either relying of LTM Policy (do nothing) or replying with a 404
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $ltm_tf_host$uri not found in pool DG"}
            if { $static::COMPANY_main_do_nothing_if_no_match } {
                set requestreturn 1
                set debughttp_time_process 0
            } else {
                call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
            }
            return
        }
    }

    set rewriteresponse 0
    set responserewrite ""
    set ContentTypeRewriteInclude ""
    set ContentTypeRewriteExclude ""

    # Insert some headers that are always required
    HTTP::header insert X_Request_ID $uid
    HTTP::header remove X-Forwarded-Proto
    if { [info exists needssl] && !($needssl) } {
        HTTP::header insert X-Forwarded-Proto "http"
    } else {
        HTTP::header insert X-Forwarded-Proto "https"
    }
    HTTP::header remove X-Forwarded-Port
    HTTP::header insert X-Forwarded-Port [TCP::local_port]


    ####### BEGIN Section 11 - check cors allowed domains ############
    set allowed_origins [getfield $config "," 11]
    foreach pattern [split $allowed_origins " "] {
        set pattern "https://$pattern"
        if { [string match $pattern [HTTP::header "Origin"]] } {
            if { ( [HTTP::method] equals "OPTIONS" ) and ( [HTTP::header exists "Access-Control-Request-Method"] ) } {
                HTTP::respond 200 "Access-Control-Allow-Origin" [HTTP::header "Origin"] \
                    "Access-Control-Allow-Methods" "POST, GET, OPTIONS" \
                    "Access-Control-Allow-Headers" [HTTP::header "Access-Control-Request-Headers"] \
                    "Access-Control-Max-Age" "86400" \
                    "Connection" "close"
                event disable all
                return
            } else {
                set cors_origin [HTTP::header "Origin"]
                break
            }
        }
    }
    ####### END Section 11 - check cors allowed domains ############

##### BEGIN Section 4 - rewrite hostname ########
    set needresponsehostrewrite 0
    if { [info exists rewritehost] && $rewritehost != "" } {
        # Check if this is a APM request and do nothing if it is the case

        switch -glob -- $rewritehost {
            "s=*" {
                set newhost [substr $rewritehost 2]
                set needresponsehostrewrite 1
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Static rewrite host: $ltm_tf_host$uri -> newhost: $newhost." }
            }
            "d=*" {
                set newhostlist [substr $rewritehost 2]
                set needresponsehostrewrite 1
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Dynamic rewrite host: $ltm_tf_host$uri -> newhostlist : $newhostlist." }
            }
        }
        if { $needresponsehostrewrite } {
            if { [catch {
            STREAM::disable
            set remove_accept_encoding 1
            }]} {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Catch command has catched the error: [subst \$::errorInfo]" }
            }
        }
    }
    ##### END Section 4 - rewrite hostname ########
    ##### BEGIN Section 5 - specific rewrite  ########
    set urirewritefind ""
    set urirewritereplace ""
    set requestrewrite ""
    if { [info exists needrewrite] && $needrewrite != "" } {

        switch -glob -- $needrewrite {
            "dg=*" {
                set dg_specific [substr $needrewrite 3]
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Specific rewrite needed for $path in DG $dg_specific" } 
                if { [class exists $dg_specific] } {
                    # For request rewrite: dg contains the string in format '/original_url /destination_url' (selecting only the first susbstitution)
                    set requestrewrite [class match -value -- request equals $dg_specific]

                    # DG contains directly the stream @srv@client@ @srv2@client2@.
                    set responserewrite [class match -value -- response equals $dg_specific]
                    if { "$responserewrite" != "" } {
                        set rewriteresponse 1
                        set ContentTypeRewriteInclude [class match -value -- ContentTypeRewriteInclude equals $dg_specific]
                        set ContentTypeRewriteExclude [class match -value -- ContentTypeRewriteExclude equals $dg_specific]
                    }
                    if { $rewriteresponse } {
                        if { [catch {
                            # Needed because of APM ACCESS::restrict_irule_events disable
                            STREAM::disable
                            set remove_accept_encoding 1
                        }] } {
                            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Catch command has catched the error: [subst \$::errorInfo]" }
                        }                    
                        if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Response rewrite found in DG $dg_specific: $responserewrite" }
                    }  else {
                        if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - No reponse rewrite in DG $dg_specific" }
                    }
                } else {
                    if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - DG $dg_specific doesn't exist therefore no rewrite needed" }
                }
            }
            default {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - No rewrite needed"} 
            }
        }
    }
    ##### END Section 5 - specific rewrite  ########
    ##### BEGIN Section 6 - header handling  ########
    if { [info exists headerlist] && $headerlist != "" } {
        set headerresponselist ""
        set headers [split $headerlist "^"]
        foreach header $headers {
            set header_name [getfield $header "=" 1]
            set headeractions [getfield [getfield $header "=" 2] : 1]
            set headerdirection [string range $headeractions 1 2]
            set headeraction [string range $headeractions 0 end-1]
            set headervalue [getfield [getfield $header "=" 2] : 2]

            switch -glob -- $headerdirection {
                q {
                    # Reverting escaping of specific characters in header value
                    set headervalue [string map { "#commaCOMPANY#" "," "#HatCOMPANY#" "^" "#equalsCOMPANY#" "=" "#colonCOMPANY#" ":" } $headervalue]
                    if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Header $headeractions $header_name $headervalue $headerdirection" }
                    call mod_header $header_name $headeraction $headervalue request $uid
                }
                s {
                    set needheaderresponse 1
                    append headerresponselist "$header_name:$headeraction:$headervalue^"

                }
                b {
                    set needheaderresponse 1
                    append headerresponselist "$header_name:$headeraction:$headervalue^"
                    # Reverting escaping of specific characters in header value
                    set headervalue [string map { "#commaCOMPANY#" "," "#HatCOMPANY#" "^" "#equalsCOMPANY#" "=" "#colonCOMPANY#" ":" } $headervalue]
                    if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Header $headeractions $header_name $headervalue $headerdirection" }
                    call mod_header $header_name $headeraction $headervalue request $uid
                }
            }

        }

    }
    ##### END Section 6 - header handling  ########

    ##### Delete Accept-Encoding header if necessary #####
    if { [info exists remove_accept_encoding] &&  $remove_accept_encoding == 1 } {
        HTTP::header remove "Accept-Encoding"
    }

    ##### BEGIN Section 7 - specific SNAT  ########
    if { [info exists specificsnat] && $specificsnat != "" } {
        set snat [getfield $specificsnat "=" 2]
        snatpool $snat
        if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Specific SNAT selected: $snat" }
    } else {
        snatpool "snat-pool-gold"
    }
    ##### END Section 7 - specific SNAT  ########

    ##### BEGIN Section 8 - specific persist  ########
    if { [info exists specificpersist] && $specificpersist != "" } {
        set mask [getfield $specificpersist "=" 2]
        if { $mask eq ""} {
            set mask "32"
        }
        set timeout [getfield $specificpersist "=" 3]
        if { $timeout eq "" } {
            set timeout 180
        }
        persist source_addr $mask $timeout
        if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Specific source_addr persist: mask: $mask, timeout: $timeout" }
    }
    ##### END Section 8 - specific persist  ########

    ####### BEGIN Section 12 - manage decompression and compression ############
    set compression_config [split [getfield $config "," 12] ";"]
    set response_compression 0
    DECOMPRESS::disable
    COMPRESS::disable
    if { !($compression_config contains "0" ||  $compression_config == "") } {
        if { $compression_config contains "1" && [HTTP::header "Content-Encoding"] contains "gzip" } {
            DECOMPRESS::enable request
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Decompression of gzip encoded request body" }
        } 
        if { $compression_config contains "2" } {
            COMPRESS::enable request
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Request compression enabled" }
        }
        if { $compression_config contains "3" } {
            set response_compression 1
        }
    }
    ####### END Section 12 - manage decompression and compression ############
}

when HTTP_REQUEST_DATA {
    ##### BEGIN DEBUG - HTTP_REQUEST_DATA  ########
    if { $debughttp_request_payload } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix ----------- http_request_payload ----------- "
        if { $contentlength } {
            set postpayload [HTTP::payload]
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Post payload: $postpayload"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_request_payload ----------- "
    }
    ##### END DEBUG - HTTP_REQUEST_DATA  ########
}

when LB_SELECTED {
    if { [info exists newhostlist] } {
        set newhost [findstr $newhostlist ";[LB::server addr]:" [string length ";[LB::server addr]:"] ";"]
        if { $newhost != "" } {
            HTTP::header replace Host $newhost
            if { $debughttp_lb_selected } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Dynamic rewrite host: $ltm_tf_host$uri -> newhost : $newhost. APM disabled or allowed" }
        }
    }

    ##### BEGIN DEBUG - LB_SELECTED  ########
    if { $debughttp_lb_selected } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_lb_selected ----------- "
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Pool member IP: [LB::server]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_lb_selected ----------- "
    }
    ##### END DEBUG - LB_SELECTED  ########
}

when HTTP_REQUEST_RELEASE {
    if { $requestreturn } {
        return
    }

    ##### BEGIN DEBUG - HTTP_TIME_PROCESS  ########
    if { $debughttp_time_process } {
        set http_request_time_release [clock clicks -milliseconds]
    }
    ##### END DEBUG - HTTP_TIME_PROCESS  ########

    if {$specificirule != ""} {
        set resp ""
        set callirulehttprequestrelease "${specificirule}::modify_http_request_release"
        if { [catch {
            set resp [call $callirulehttprequestrelease $debughttp_request $uid]
        }] } {
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Unknown irule $specificirule" }
            call error_respond 404 "ifile-ltm-tf-404.html" "irule $specificirule not found" $error_page_strat $ip_client_addr $uid
            return
        }
        if {$resp eq "return"} {
            return
        }
    }

    ##### BEGIN Section 4 - rewrite hostname ########
    set needresponsehostrewrite 0
    if { [info exists rewritehost] && $rewritehost != "" } {
        # Check if this is a APM request and do nothing if it is the case

        switch -glob -- $rewritehost {
            "s=*" {
                set newhost [substr $rewritehost 2]
                HTTP::header replace Host $newhost
                set needresponsehostrewrite 1
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Static rewrite host: $ltm_tf_host$uri -> newhost: $newhost." }
            }
            "d=*" {
                set newhostlist [substr $rewritehost 2]
                set needresponsehostrewrite 1
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Dynamic rewrite host: $ltm_tf_host$uri -> newhostlist : $newhostlist." }
            }
        }
        if { $needresponsehostrewrite } {
            if { [catch {
            # STREAM::disable
            set remove_accept_encoding 1
            }]} {
                if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Catch command has catched the error: [subst \$::errorInfo]" }
            }
        }
    }
    ##### END Section 4 - rewrite hostname ########

    if { "$requestrewrite" != "" } {
        for {set i 0} {$i < [llength $requestrewrite]} {incr i 2} {
            set urirewritefind [lindex $requestrewrite $i]
            if { [HTTP::path] starts_with $urirewritefind } {
                break
            }
        }
        set urirewritereplace [lindex $requestrewrite [expr {$i+1}]]
        if { [HTTP::path] starts_with $urirewritefind } {
            regsub -- ***=$urirewritefind [HTTP::path] "" newpath
            set newpath "$urirewritereplace/$newpath"
            set newpath [string map { "///" "/" "//" "/" } "$newpath"]
            HTTP::path $newpath
            if { $debughttp_request } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - DG $dg_specific replaced $path with $newpath" }
        }
    }

    ##### BEGIN DEBUG - HTTP_REQUEST_RELEASE  ########
    if { $debughttp_request_release eq "1" } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_request_release ----------- "
        if { [PROFILE::exists clientssl] == 1 } {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cipher protocol: https"
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cipher name: [SSL::cipher name]"
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cipher version: [SSL::cipher version]"
        }

        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - VS Name: [virtual]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Request: [HTTP::method] [HTTP::host][HTTP::uri]"
        foreach aHeader [HTTP::header names] {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $aHeader: [HTTP::header value $aHeader]"
        }

        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_request_release ----------- "
    }
    ##### END DEBUG - HTTP_REQUEST_RELEASE  ########
}



when LB_FAILED {
    # If there is an issue with the backend (no members, connectivity problem, ...) respond with an error page.
    if { $error_page_strat == 1 } {
        if { $debughttp_lb_selected } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Sending default 404 response (lb failed and strategy is 1)" }
        call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
    } else {
        if { $debughttp_lb_selected } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Sending default 502 response (lb failed and strategy is 2)" }
        call error_respond 502 "ifile-ltm-tf-5xx.html" "unable to select backend server from pool $selectpool" $error_page_strat $ip_client_addr $uid
    }
    return
}

when HTTP_RESPONSE_RELEASE {
    if { $requestreturn } {
        return
    }  

    if { $debughttp_time_process } {
        set http_response_time_release [clock clicks -milliseconds]
    }

    ##### BEGIN DEBUG - HTTP_RESPONSE_RELEASE  ########
    if { $debughttp_response_release } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response_release ----------- "
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Status: [HTTP::status]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Pool member IP: [LB::server]"
        foreach aHeader [HTTP::header names] {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $aHeader: [HTTP::header value $aHeader]"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response_release ----------- "
    }
    ##### END DEBUG - HTTP_RESPONSE_RELEASE  ########

    ##### BEGIN DEBUG - HTTP_TIME_PROCESS  ########
    if { $debughttp_time_process && [info exists http_request_time_release] } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_time_process ----------- "
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Time to request  (F5 request time) = [expr {$http_request_time_release - $http_request_time}] (ms)"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Time to response (F5 response time) = [expr {$http_response_time_release - $http_response_time}] (ms)"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Time to server (server backend process time) = [expr {$http_response_time - $http_request_time_release}] (ms)"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_time_process ----------- "
    }
    ##### END DEBUG - HTTP_TIME_PROCESS  ########
}

when HTTP_RESPONSE_DATA {
    ##### BEGIN DEBUG - HTTP_RESPONSE_PAYLOAD  ########
    if { $debughttp_response_payload } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response_payload ----------- "
        set payload [HTTP::payload]  
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Response (Body) payload: $payload"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response_payload ----------- "
    }
    ##### END DEBUG - HTTP_RESPONSE_PAYLOAD  ########
}


when HTTP_RESPONSE {
    if { $requestreturn } {
        return
    }
    ##### BEGIN DEBUG - HTTP_TIME_PROCESS  ########
    if { $debughttp_time_process } {
        set http_response_time [clock clicks -milliseconds]
    }
    ##### END DEBUG - HTTP_TIME_PROCESS  ########


    ##### BEGIN DEBUG - HTTP_RESPONSE  ########
    if { $debughttp_response } {
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response ----------- "
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Status: [HTTP::status]"
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Pool member IP: [LB::server]"
        foreach aHeader [HTTP::header names] {
            log -noname local0. "uid=$uid $static::COMPANY_main_prefix - $aHeader: [HTTP::header value $aHeader]"
        }
        log -noname local0. "uid=$uid $static::COMPANY_main_prefix  ----------- http_response ----------- "
    }
    ##### BEGIN DEBUG - HTTP_RESPONSE  ########

    ##### BEGIN DEBUG - HTTP_RESPONSE_PAYLOAD  ########
    if { $debughttp_response_payload } {
        set content_length [HTTP::header "Content-Length"]
        if { $content_length > 0 && $content_length < 1048577 } {
            set collect_length $content_length
        } else {
            set collect_length 1048576
        } 

        if { $collect_length > 0 } {
            HTTP::collect $collect_length
        }
    }
    ##### END DEBUG - HTTP_RESPONSE_PAYLOAD  ########

    if {$specificirule != ""} {
        set resp ""
        set callirulehttpresponse "${specificirule}::modify_http_response"
        if { [catch {
            set resp [call $callirulehttpresponse $debughttp_response $uid]
        }] } {
            if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Unknown irule $specificirule" }
            call error_respond 404 "ifile-ltm-tf-404.html" "irule $specificirule not found" $error_page_strat $ip_client_addr $uid
            return
        }
        if {$resp eq "return"} {
            return
        }
    }

    ###### BEGIN cookie domain rewrite ###########
    foreach mycookie [HTTP::cookie names] { 
        set cookiedomain [HTTP::cookie domain $mycookie]
        if { $cookiedomain != "" } {
            if { "$cookiedomain" starts_with "." } {
                # Cookie subdomain: need to be compared to host domain
                set hostdomain [string range $ltm_tf_host [string first . $ltm_tf_host] end]
                if { "$cookiedomain" != $hostdomain } {
                    HTTP::cookie domain $mycookie $hostdomain
                }
            } else {
                # Cookie domain: need to be compared to hostname
                if { "$cookiedomain" != $ltm_tf_host } {
                    HTTP::cookie domain $mycookie $ltm_tf_host
                }
            }
            set newcookiedomain [HTTP::cookie domain $mycookie]
            if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Domain $cookiedomain for cookie $mycookie changed to $newcookiedomain" }
        } else {
            if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cookie $mycookie : no domain set, no change needed" }
        }
    }
    ###### END cookie domain rewrite ###########


    ####### BEGIN Section 10 - cookie handling ############
    set cookiehandling [getfield $config "," 10]
    set modifycookie 0
    if { $cookiehandling contains "httponly" } {
        set httponly 1
        set modifycookie 1
    } else {
        set httponly 0
    }
    if { $cookiehandling contains "secure" } {
        set secure 1
        set modifycookie 1
    } else {
        set secure 0
    }
    if { $cookiehandling contains "samesite" } {
        set samesite [findstr $cookiehandling "samesite=" 9]
        set modifycookie 1
    } else {
        set samesite ""
    }
    if { $modifycookie } {
        foreach mycookie [HTTP::cookie names] { 
            if { $httponly } {
                HTTP::cookie httponly $mycookie enable
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cookie $mycookie set to httponly" }
            }
            if { $secure } {
                HTTP::cookie secure $mycookie enable
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cookie $mycookie set to secure" }
            }          
            if { $samesite != "" } {
                if { not [HTTP::cookie attribute $mycookie exists {SameSite}] } {
                    switch -- [string tolower $samesite] {
                        "lax" { set samesite "Lax"}
                        "strict" { set samesite "Strict"}
                        "default" { 
                            set samesite "None" 
                            # samesite=none needs secure
                            HTTP::cookie secure $mycookie enable
                            }
                    }
                    HTTP::cookie attribute $mycookie insert "SameSite" "$samesite"
                    if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Cookie $mycookie set to samesite $samesite" }
                }
            }            

        }       

    }
    ####### END Section 10 - cookie handling ############

    ####### BEGIN Section 9 - error page handling ############
    set errorpage [getfield $config "," 9]
    if { $errorpage != "" && ([HTTP::status] starts_with "4" || [HTTP::status] starts_with "5") } {
        set errorpageconfiglist [split $errorpage ";"]
        set error_page_keep 0
        foreach errorpageconfig $errorpageconfiglist {
            set errorcode [getfield $errorpageconfig "=" 1]
            if { [HTTP::status] == $errorcode } {
                set errormessage [getfield $errorpageconfig "=" 2]
                switch -- $errormessage {
                    "k" - "" {
                        if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Keeping original error page from backend" }
                        set error_page_keep 1
                        break
                    }
                    "d" {
                        if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Reponding with default error pageand message" }
                        if { [HTTP::status] starts_with "4"  } {
                            call error_respond [HTTP::status] "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
                        } else {
                            call error_respond [HTTP::status] "ifile-ltm-tf-5xx.html" "" $error_page_strat $ip_client_addr $uid
                        }
                        return
                    }
                    default {
                        if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Reponding with default error page and specific message $errormessage" }
                        # Reverting escaping of specific characters in error message
                        set message [string map { "#commaCOMPANY#" "," "#semiCOMPANY#" ";" "#equalsCOMPANY#" "=" } $errormessage]
                        if { [HTTP::status] starts_with "4"  } {
                            call error_respond [HTTP::status] "ifile-ltm-tf-404.html" $message $error_page_strat $ip_client_addr $uid
                        } else {
                            call error_respond [HTTP::status] "ifile-ltm-tf-5xx.html" $message $error_page_strat $ip_client_addr $uid
                        }
                        return
                    }
                }
            } 
        }
        if { !($error_page_keep) && $error_page_strat == 1 } {
            if { [HTTP::status] starts_with "4" } {
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Sending default [HTTP::status] response because strategy is 1 and backend sent an 4** or 5**" }
                call error_respond [HTTP::status] "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
            } else {
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Sending default 404 response because strategy is 1 and backend sent an 4** or 5**" }
                call error_respond 404 "ifile-ltm-tf-404.html" "" $error_page_strat $ip_client_addr $uid
            }
            return
        }
    }
    ####### END Section 9 - error page handling ############

    ##### BEGIN Section 4 and 5 - rewrite hostname - specific rewrite ########
    if { [HTTP::is_redirect] } {
        set orilocation [HTTP::header value Location]
        if { $needresponsehostrewrite && [string tolower [URI::host $orilocation]] == $newhost } {
            HTTP::header replace Location [string map -nocase [list http:// https:// ":[LB::server port]" "" $newhost $ltm_tf_host] [HTTP::header value Location]]
            set orilocation [HTTP::header value Location]
            if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Rewrite Location header: $orilocation -> [HTTP::header value Location]" } 
        }
        if { $urirewritefind != "" && ([URI::host $orilocation] == "" || [string tolower [URI::host $orilocation]] == $ltm_tf_host) } {
            if { $orilocation starts_with "$urirewritereplace" } {
                regsub -- ***=$urirewritereplace $locationurl "" newlocation
                set newlocation "$urirewritefind/$newlocation"
                set newlocation [string map { "///" "/" "//" "/" } "$newlocation"]
                HTTP::header replace Location $newlocation
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Rewrite Location header: $orilocation -> [HTTP::header value Location]" } 
            } else {
                set locationurl "[URI::path $orilocation][URI::basename $orilocation]"

                if {[URI::query $orilocation] ne ""} {
                    set locationurl "$locationurl?[URI::query $orilocation]"
                }

                if { $locationurl starts_with "$urirewritereplace" } {
                    regsub -- ***=$urirewritereplace $locationurl "" newlocation
                    set newlocation "$urirewritefind/$newlocation"
                    set newlocation [string map { "///" "/" "//" "/" } "$newlocation"]
                    HTTP::header replace Location "[URI::protocol $orilocation]://[URI::host $orilocation]$newlocation"
                    if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Rewrite Location header: $orilocation -> [HTTP::header value Location]" } 
                }
            }
        }
    } elseif { $rewriteresponse } {
        STREAM::expression "$responserewrite"
        if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Rewrite response: stream expression is $responserewrite" }
        set ContentTypeList [list {text} {application/javascript} $ContentTypeRewriteInclude]
        set myContentType  [HTTP::header value "Content-Type"]
        set CTfound 0
        foreach CT $ContentTypeList {
            if { $myContentType contains $CT } {
                set CTfound 1
                break
            }
        }
        if { $CTfound } {
            set CTexclude 0
            foreach CT $ContentTypeRewriteExclude {
                if { $myContentType contains $CT } {
                    set CTexclude 1
                    break
                }
            }
            if { $CTexclude } {
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Content-Type $myContentType found but excluded" }
            } else {
                if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Content-Type $myContentType found and not excluded: proceeding with streaming" }
                STREAM::enable
            }
        }
    }

    if { [info exists headerresponselist] && $headerresponselist != "" } {
        set headers [split $headerresponselist "^"]
        foreach header $headers {
            set header_name [getfield $header ":" 1]
            set headeraction [getfield $header ":" 2]
            set headervalue [getfield $header ":" 3]
            # Reverting escaping of specific characters in header value
            set headervalue [string map { "#commaCOMPANY#" "," "#HatCOMPANY#" "^" "#equalsCOMPANY#" "=" "#colonCOMPANY#" ":" } $headervalue]
            if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Header $headeractions $header_name $headervalue response" }
            call mod_header $header_name $headeraction $headervalue response $uid
        }
    }
    ##### BEGIN Section 4 and 5 - rewrite hostname - specific rewrite ########

    ####### BEGIN Section 11 - check cors allowed domains ############
    if { [info exists cors_origin] && $cors_origin != ""} {
        HTTP::header replace "Access-Control-Allow-Origin" $cors_origin
        HTTP::header replace "Access-Control-Allow-Credentials" "true"
        HTTP::header replace "Vary" "Origin"
    }
    ####### END Section 11 - check cors allowed domains ############

    ####### BEGIN Section 12 - manage decompression and compression ############
    COMPRESS::disable
    if { [info exists response_compression] && $response_compression == 1} {
        COMPRESS::enable
        if { $debughttp_response } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Response compression enabled" }
    }
    ####### END Section 12 - manage decompression and compression ############
}

when STREAM_MATCHED {
    if { $debughttp_response_payload } { log -noname local0. "uid=$uid $static::COMPANY_main_prefix - Stream matched: [STREAM::match]" }
}

### Proc to modify header, used at several places in request or responses
proc mod_header { name action value direction uid } {
    switch -- $action {
        i {
            HTTP::header insert $name $value 
        }
        r {
            HTTP::header remove $name
            HTTP::header insert $name $value
        }
        d {
            HTTP::header remove $name
        }
    }
}

### Proc to send error page to the client 
proc error_respond { code ifile message err_page_strat ip_client_addr uid } {
    if { $err_page_strat == 1 && ($ip_client_addr starts_with "10." || $ip_client_addr starts_with "192.168.") } {
        set ifile "ifile-ltm-tf-404.html"
        set message "Une erreur est survenue..."
    }
    set response_content [subst -nocommands [ifile get $ifile]]
    binary scan $response_content @0
    HTTP::respond $code content $response_content noserver Content-Type "text/html; charset=utf-8" Connection Close
    event disable all
}

proc ci_replace {haystack search replace} {
    # Lowercase versions for searching
    set lower_haystack [string tolower $haystack]
    set lower_search   [string tolower $search]

    # Find the position of the match
    set pos [string first $lower_search $lower_haystack]

    # If not found, return original string
    if {$pos < 0} {
        return $haystack
    }

    # Build the new string using original casing
    set before [string range $haystack 0 [expr {$pos - 1}]]
    set after  [string range $haystack [expr {$pos + [string length $search]}] end]

    return "${before}${replace}${after}"
}

proc is_whitelisted { ip cidrs } {
    foreach cidr $cidrs {
        if { [IP::addr $ip equals $cidr] } {
            return 1
        }
    }
    return 0
}

Attribution

Although I am submitting this work, I would like to clarify that Fred is mostly the mastermind behind this. Thomas G also did great work on improving it and Jean also contributed to complex features. Thomas K also greatly participated in this irule’s engineering. Bref, this is a collaborative work and I would like to salute my colleagues for this one.

5 Likes
2

Hey, this is nice stuff. Took you a while to write this I guess.
You are using ‘call irule-ltm-tf-helper’, but this required code is not part of the iRule.
Can you add that code as a comment, maybe?

1 Like
3

Hi, I’m one of the co-author of this iRule. Here is the missing code :

proc error_respond { code ifile message err_page_strat ip_client_addr uid } {
    if { $err_page_strat == 1 && ($ip_client_addr starts_with "10." || $ip_client_addr starts_with "192.168.") } {
        set ifile "ifile-ltm-tf-404.html"
        set message "Une erreur est survenue..."
    }
    set response_content [subst -nocommands [ifile get $ifile]]
    binary scan $response_content @0
    HTTP::respond $code content $response_content noserver Content-Type "text/html; charset=utf-8" Connection Close
    event disable all
}

proc is_whitelisted { ip cidrs } {
    foreach cidr $cidrs {
        if { [IP::addr $ip equals $cidr] } {
            return 1
        }
    }
    return 0
}

3 Likes
4

That’s the Team !

1 Like