Application Request Parsing

Let's imagine you've been given the task to interrogate all form data that is passed into your web application for the purpose of auditing and validation.  This logic is often built into the application but often security flaws are found and a fix to block that flaw is required before the development team is able to fix the hole in their application. 

This article will illustrate how to write an iRule to interrogate GET or POST form data to allow for auditing and policy enforcement.

HTTP application requests can take the format of HTTP GET and POST commands (well, there are others, but these are the most common).  With a GET command, all parameter names and and values are appended to the URI separated by a question mark.

GET /index.html?p1=pv1&p2=pv2 HTTP/1.1
Host: www.foo.com

With  a POST command, the parameters are not appended to the URI, but sent as part of the HTTP payload with a size specified with the "Content-Length" header.

POST /index.html HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: www.foo.com
Content-Length: 13
Expect: 100-continue

p1=pv1&p2=pv2

Applications also use HTTP headers to pass information around.  This can be in the form of HTTP cookies, authentication tokens, mime types, content lengths and other data.  Since headers are often tied to applications, I'l throw in some code to inspect the HTTP headers as well.

To inspect the HTTP headers, you'll want to use the HTTP::header command to extract the header names as well as their associated values as illustrated here.

when HTTP_REQUEST {
  # Inspect Headers
  set names [HTTP::header names]
  foreach name $names {
    set val [HTTP::header value $name]
    log local0. "    $name: $val"
  }
}

Next, you'll likely want to do some URI parsing to determine which application the request is going to.  This code uses the URI::path depth subcommand to determine the number of directories (the depth) in the URI (retrieved with the HTTP::uri command).  It then enters a for loop and extracts each directory, again with the URI::path command.  Finally it uses the URI::basename command to extract the trailing file component from the URI.

when HTTP_REQUEST {
  # Inspect URI
  set depth [URI::path [HTTP::uri] depth]
  for {set i 1} {$i <= $depth} {incr i} {
    set dir [URI::path [HTTP::uri] $i $i]
    log local0. "    dir\[$i\]: $dir"
  }
  log local0. "    Basename: [URI::basename [HTTP::uri]]"
}

Once the URI has been parsed, you'll want to determine whether the request is a GET or POST request.  This can be done with the HTTP::method command

when HTTP_REQUEST {
  switch [HTTP::method] {
    "GET" {
      log local0. "GET Request"
    }
    "POST" {
      log local0. "POST Request"
    }
  }
}

For a GET request, you'll want to use the HTTP::query command along with the TCL split command to extract the query string arguments from the end of the URI

when HTTP_REQUEST {
  # Inspect Query String
  set namevals [split [HTTP::query] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
    log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}

For a POST request, you'll have to do a little more work to ensure that the body of the request containing the POST data has been received.  For small payloads, the full body has likely been retrieved from within the HTTP_REQUEST event but it's better to be safe than sorry.  You'll want to issue a HTTP::collect with the value supplied in the "Content-Length" header and process the payload in the HTTP_REQUEST_DATA event.

One thing to mention here, is that it might be good form to verify the Content-Type to verify it's value is "application/x-www-form-urlencoded"  Other applications make use of HTTP POST commands (XML-RPC, SOAP, etc) so to make sure it's a HTML.

when HTTP_REQUEST {
  if { [HTTP::header Content-Type] eq "application/x-www-form-urlencoded" } {
    HTTP::collect [HTTP::header Content-Length]
  }
}

when HTTP_REQUEST_DATA {
  set namevals [split [HTTP::payload] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
   log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}

You now have all the tools to extract and inspect the HTTP Headers, HTTP query string (for GET Requests) and HTTP post data (for POST requests).  If you wanted to put it all together in one iRule, it might look something like this:

when HTTP_REQUEST {

  # Inspect Headers
  log local0. "============================"
  log local0. "<<< HTTP Headers >>>"
  set names [HTTP::header names]
  foreach name $names {
    set val [HTTP::header value $name]
    log local0. "    $name: $val"
  }

  # Inspect URI
  log local0. "<<< URI >>>"
  log local0. "    HTTP::uri: [HTTP::uri]"
  log local0. "    HTTP::path: [HTTP::path]"
  set depth [URI::path [HTTP::uri] depth]
  for {set i 1} {$i <= $depth} {incr i} {
    set dir [URI::path [HTTP::uri] $i $i]
    log local0. "    dir\[$i\]: $dir"
  }
  log local0. "    Basename: [URI::basename [HTTP::uri]]"

  switch [HTTP::method] {
    "GET" {
      # Inspect Query String
      log local0. "<<< Query Information >>>"
      log local0. "    HTTP::query: [HTTP::query]"
      set namevals [split [HTTP::query] "&"]
      for {set i 0} {$i < [llength $namevals]} {incr i} {
        set params [split [lindex $namevals $i] "="]
        log local0. "    [lindex $params 0] : [lindex $params 1]"
      }
    }
    "POST" {
      if { [HTTP::header Content-Type] eq "application/x-www-form-urlencoded" } {
        log local0. "<<< Post Data Information >>>"
        HTTP::collect [HTTP::header Content-Length]
      }
    }
  }
}

when HTTP_REQUEST_DATA {
  log local0. "    POST Data: [HTTP::payload]"
  set namevals [split [HTTP::payload] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
    log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}

Well, now you have all the tools you need to interrogate HTTP form requests.  Inspect away...