For more information regarding the security incident at F5, the actions we are taking to address it, and our ongoing efforts to protect our customers, click here.

Forum Discussion

Matt_Breedlove_'s avatar
Matt_Breedlove_
Icon for Nimbostratus rankNimbostratus
May 17, 2014

Proc for fully decoding URI's

Hi All

 

Looking to fully sanitize URI's with a proc. Similar to what the irules codeshare FullyDecodeURI does (https://devcentral.f5.com/wiki/iRules.FullyDecodeUri.ashx) but in a proc

 

Im not a tcl guy, but I see the one proc example on encoding a URI. It seems like this shouldn't be too hard since its mostly just a while loop working on a string. Should really clean up irules that are intentionally fully decoding inbound URI's for security

 

Running BIG-IP 11.4.1 Build 625.0 Hotfix HF1

 

I know that the bug is fixed in this version that FullDecodeURI codeshare references with a max loop variable, but this seems like something worth doing regardless of the LTM version

 

Any proc suggestions to get started? Looking to be able to pass the URI to the proc and get back a pure 100% decoded URI even with multiple levels of encode nesting in the source URI.

 

Should using the proc be more performant than using an in event while loop in the irule to call URI::Decode several times?

 

7 Replies

  • not sure if i understand correctly. you want to make FullyDecodeURI proc, don't you?

    e.g.

     config
    
    [root@ve11a:Active:In Sync] config  tmsh list ltm rule qux
    ltm rule qux {
        proc decode_uri { uri max } {
      set cnt 0
      set tmpuri [URI::decode $uri]
    
      while { $uri ne $tmpuri && $cnt < $max } {
        set uri $tmpuri
        set tmpuri [URI::decode $uri]
        incr cnt
      }
      return $tmpuri
    }
    when HTTP_REQUEST {
      set uri [call decode_uri [HTTP::uri] 10]
      log local0. "URI=[HTTP::uri] decoded URI=$uri"
      HTTP::uri $uri
    }
    }
    
     /var/log/ltm
    
    [root@ve11a:Active:In Sync] config  tail -f /var/log/ltm
    May 17 21:01:51 ve11a info tmm[14715]: Rule /Common/qux : URI=/%20something%20bha%20bha%20bha decoded URI=/ something bha bha bha
    
    • Matt_Breedlove_'s avatar
      Matt_Breedlove_
      Icon for Nimbostratus rankNimbostratus
      Hi Nitass...So I was thinking that constructs such as the LTM native irule command [URI::decode] were not available inside a proc since a proc does not have an event context requirement and so how could it reference an inflight HTTP request? So I thought that the TCL inside the proc was more stripped down to string level operations and not allowing all the same command availablity from the standard irules context inside a CLIENT_ACCEPT or HTTP_REQUEST event for example. Now that I think of it I am not sure why I assumed the language context inside the proc was more TCL native than outside the proc in the rest of an irule. As a result my plan was to essentially rewrite [URI::decode] natively with plain ole TCL inside the proc and specifically make sure it recursively decodes the inbound URL string fully or at least to my pre-planned iterative backstop with a cnt variable. URLs can be encoded with nesting several times and I want to make sure this is handled in every case to my planned limit. It is unfortunate that [URI::decode] doesn't solve the issue with nested encoded URL's more easily, not now it seems that maybe that command is more TCL than it is F5 in terms of irules and it is by design from the TCL perspective So bottom line, you are saying just move the regular irule portion of FullyDecodeURI from codeshare that uses a backstopped while loop and use the new Procedure functionality to have it declared just once and callable. Unless someone says "Oh, no you cannot use irules connection aware commands suchas [URI::decode] inside the proc you must work on it at the scalar string level with tcl string commands and handle all the exception cases with unicode and URL/Percent encoding that should be expected with some switch's." Then the decode command can just be used. Perhaps this is actually something native to TCL that irules reflects and I am just understanding it backwards? Related: What is the under the hood TCL that [URI::decode] runs on the string level to handle inbound strings? Does it change in terms of the rules followed to decode based on if an HTTP::query or HTTP::uri or HTTP::host is passed in
  • not sure if i understand correctly. you want to make FullyDecodeURI proc, don't you?

    e.g.

     config
    
    [root@ve11a:Active:In Sync] config  tmsh list ltm rule qux
    ltm rule qux {
        proc decode_uri { uri max } {
      set cnt 0
      set tmpuri [URI::decode $uri]
    
      while { $uri ne $tmpuri && $cnt < $max } {
        set uri $tmpuri
        set tmpuri [URI::decode $uri]
        incr cnt
      }
      return $tmpuri
    }
    when HTTP_REQUEST {
      set uri [call decode_uri [HTTP::uri] 10]
      log local0. "URI=[HTTP::uri] decoded URI=$uri"
      HTTP::uri $uri
    }
    }
    
     /var/log/ltm
    
    [root@ve11a:Active:In Sync] config  tail -f /var/log/ltm
    May 17 21:01:51 ve11a info tmm[14715]: Rule /Common/qux : URI=/%20something%20bha%20bha%20bha decoded URI=/ something bha bha bha
    
    • Matt_Breedlove_'s avatar
      Matt_Breedlove_
      Icon for Nimbostratus rankNimbostratus
      Hi Nitass...So I was thinking that constructs such as the LTM native irule command [URI::decode] were not available inside a proc since a proc does not have an event context requirement and so how could it reference an inflight HTTP request? So I thought that the TCL inside the proc was more stripped down to string level operations and not allowing all the same command availablity from the standard irules context inside a CLIENT_ACCEPT or HTTP_REQUEST event for example. Now that I think of it I am not sure why I assumed the language context inside the proc was more TCL native than outside the proc in the rest of an irule. As a result my plan was to essentially rewrite [URI::decode] natively with plain ole TCL inside the proc and specifically make sure it recursively decodes the inbound URL string fully or at least to my pre-planned iterative backstop with a cnt variable. URLs can be encoded with nesting several times and I want to make sure this is handled in every case to my planned limit. It is unfortunate that [URI::decode] doesn't solve the issue with nested encoded URL's more easily, not now it seems that maybe that command is more TCL than it is F5 in terms of irules and it is by design from the TCL perspective So bottom line, you are saying just move the regular irule portion of FullyDecodeURI from codeshare that uses a backstopped while loop and use the new Procedure functionality to have it declared just once and callable. Unless someone says "Oh, no you cannot use irules connection aware commands suchas [URI::decode] inside the proc you must work on it at the scalar string level with tcl string commands and handle all the exception cases with unicode and URL/Percent encoding that should be expected with some switch's." Then the decode command can just be used. Perhaps this is actually something native to TCL that irules reflects and I am just understanding it backwards? Related: What is the under the hood TCL that [URI::decode] runs on the string level to handle inbound strings? Does it change in terms of the rules followed to decode based on if an HTTP::query or HTTP::uri or HTTP::host is passed in
  • you are saying just move the regular irule portion of FullyDecodeURI from codeshare that uses a backstopped while loop and use the new Procedure functionality to have it declared just once and callable.

     

    yes. i do not think URI::decode is connection-aware command.

     

    Does it change in terms of the rules followed to decode based on if an HTTP::query or HTTP::uri or HTTP::host is passed in

     

    do you mean the decode result? i understand it is only a command return (i.e. no change to object which is passed in).

     

  • If I may add, the URI::decode command is actually a string-level parser. You could just as easily use it like this:

    when RULE_INIT {
        set uri "/something%20something%20something/blah"
        log local0. [URI::decode $uri]
    }
    

    What you couldn't do is use a protocol-level command, like HTTP::uri, inside a proc. Otherwise a proc is just a function that takes inputs and returns an output. There are of course going to be exceptions to this rule, but you can mostly say that any command that's prefixed with a specific protocol (ie. HTTP::, DNS::, SSL::, TCP::, etc.) isn't going to be directly accessible inside a proc. Anything else is either fair game or can be passed in as a parameter.

  • Thanks for clarifying this and confirming that HTTP:: TCP::, etc commands are offlimits in proc's even if the HTTP_REQUEST event or CLIENT_ACCEPT is calling the proc

    I would just add that this shouldn't be a problem. You generally need a protocol-level event to trigger iRule processing, so whatever data is accessible in that event can be sent to the proc as a parameter. A dirty example:

    proc doSomethingWithUri { uri } {
        set foo [URI::decode $uri]
        return $foo
    }
    when HTTP_REQUEST {
        if { [string tolower [HTTP::uri]] starts_with "/app1" } {
            set ret [call doSomethingWithUri [HTTP::uri]]
            log local0. $ret
        }
    }