iRule recognize blank space in URI?

Hi, I need to detect blank space in URI and either remove it or rewrite it:

www.mysite.com/some content/showme.html

to

www.mysite.com/some%20content/showme.html

Current solution in place is only recognizing up to the space so it reads:

www.mysite.com/some. and breaks the client request to the server.

I am thinking if I can find out how BIG-IP can detect the space, I will just replace it with characters that I need in its place.

Thanks.

If the uri in the HTTP request contains a space, then that is actually an illegal request according to the HTTP RFC. This is because HTTP uses spaces to separate the method, uri, and version on the first line of the HTTP request. I’m assuming you are using some kind of homegrown client? To do this, you’ll have to parse the request at the TCP layer using the CLIENT_DATA event and the TCP::payload commands. This is because our HTTP engine will only parse up to the space when parsing for the uri. You could try looking in the version to see if the rest of the uri ends up there (see HTTP::version).

HTH

So using TCP::payload command, I can do a “contains” to find the URI, but the URI is not the same all the time and I will need to find a way to find a URI with space in it? Is it possible to “match” on the space within the URI?

thanks.

You may want to use a regular expression and bound it by the method and the version and/or newline. Maybe something like:

/^GET .+ (HTTP/.+)?\r?\n/

I just started testing this and interestingly enough, BIG-IP is replacing space within URI with %20, when I looked at the tcpdump, as well iRule log statement.

http://server.xxx.com/vMoD/vMoDGenres?title=R and B

shows up as

http://server.xxx.com/vMoD/vMoDGenres?title=R%20and%20B

so is BIG-IP LTM (v9.1.1) able recognize malformed URI and convert it in this case? Other than the log statement to print the URI, I have not done any iRule processing.

John

So, once you find the match, how do you extract that portion into a variable?

I used

when CLIENT_ACCEPTED {

TCP::collect 200

}

when CLIENT_DATA {

if { [TCP::payload] matches_regex “/vMoD.+Y”} {

set uri [[TCP::payload] matches_regex “/vMoD.+Y”]

log local0. $uri

} else {

log local0. “no match”

}

}

to match on the URI as the sample Joe provided would not find the URI b/w GET and HTTP/1.0. However, the ltm log is giving me

Mar 9 13:44:26 tmm tmm[824]: 01220001:3: TCL error: Rule vmod_search - invalid command name "GET /vMoD/vMoDSearch?search_type=1&search_string=Bob%20Dylan&suppress_art=Y HTTP/1.0 Host: moddev1.ztango.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.7,ko-kr;q=0.3 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive " while executing “[TCP::payload] matches_regex “/vMoD.+Y””

While I am still allowed to connect to the server. I am not sure if I am doing something illegal within regex pattern?

"[TCP::payload] matches_regex “/vMoD.+Y” is a boolean comparison. It’s outcome of 0 or 1 is not a command and can’t be enclosed in brackets (which indicate execution of a command). If you want to extract the content, then you will have to use one of the builtin regular expression commands like regexp (Click here).

You could do something like this to look for your pattern

when CLIENT_DATA {
   regexp returns 1 if a match is found
  if { [regexp "/vMoD.+Y" [TCP::payload] match] }
    log local0. "match found: $match"
  } else {
    log local0. "no match"
  }
}

Keep in mind that this will not return the entire URI but only the matched content. So for an URI of “http://www.foo.com/vMoDblahblahY?foobar” the value of match would be “/vMoDblahblahY” since that is the exact content that matches the pattern you requested.

To look for the full URI you are going to have to expand you regular expression to cover the rest of it.

-Joe

Thank you, I was able to extract the URI b/w “GET” and " HTTP" (to remove the last space before HTTP) with:

when CLIENT_ACCEPTED {

TCP::collect 200

set ::uri 0

}

when CLIENT_DATA {

if { [regexp “/vMoD.+HTTP” [TCP::payload] match] } {

log local0. “match found: $match”

set ::uri [getfield $match " HTTP" 1]

log local0. $::uri

} else {

log local0. “no match”

}

}

I tried adding

when HTTP_REQUEST_DATA {

HTTP::redirect http://[HTTP::host]$::uri

}

I was looking for a simple way to test and see if I can substitue the URI with a new one, but obviously it does not. Do I need to figure out a way to replace it within the CLIENT_DATA event?

John

I am also trying to see if I can use HTTP::collect to access HTTP::request or HTTP::payload, but HTTP::request generates “undefined procedure” and the payload comes up empty… oh, and HTTP::decode is not supported?

when HTTP_REQUEST {

HTTP::collect

set my_request [HTTP::request]

log local0. “request is $my_request”

set my_data [HTTP::payload]

log local0. “my data is $my_data”

}

when HTTP_REQUEST_DATA {

if { [regexp “/vMoD.+HTTP” [HTTP::payload] match]} {

log local0. “match found $match”

set uri [getfield $match " HTTP" 1]

set my_uri [HTTP::decode $uri]

log local0. “My uri is $uri”

} else {

log local0. “No Match”

}

}

With a lot of help, I am down to the rule below. But the URI is coming out with leading “{” from "regexp -inline… "(I think) and I tried using string trimright to the result with “\}” as the character, but still can’t remove that last brace. Any ideas?

when CLIENT_ACCEPTED {

TCP::collect 300

}

when CLIENT_DATA {

set matchedURI [regexp -inline /vMoD.+HTTP [TCP::payload]]

log local0. “first match $matchedURI”

if { [string length $matchedURI] > 0 } {

set newURI [getfield $matchedURI " HTTP" 1]

log local0. “URI was matched. Result: $newURI”

set uriRewrite 1

} else {

log local0. “no match”

set uriRewrite 0

}

}

when HTTP_REQUEST {

if {$uriRewrite == 1} {

set encURI [URI::encode $newURI]

HTTP::uri $encURI

log local0. “It worked! $newURI”

}

}

Thanks.

another question if someone can help. Encoding all of URI is easy but turns all specical characters into encoded format and the server is not accepting the request. The cahllenge is that I need some way to ONLY find spaces and replace that with %20. Anyone with a good idea?

Thanks!

John

I think

regsub -all " " $newURI %20 encURI

will do it, if there is a better way, please let me know.

Thanks for all the help.

John

string map would probably be a lot faster.

Thanks, that worked. It is going to be pounded with probably 40.000 to 50,000 request per second so anything that can make it more efficient is great.

John

ok, I am back, strange problem.

Test request I am sending:

[root@bigip1:Active] config telnet 10.10.50.203 80 Trying 10.10.50.203…

Connected to 10.10.50.203.

Escape character is ‘^]’.

GET /vMoD/vMoDGenres?title=R and B HTTP/1.0 ^]

telnet> q

Connection closed.

Seeing in the ltm log:

Mar 24 15:13:31 tmm tmm[824]: Rule vmod_search : URI was matched. Result: /vMoD/vMoDGenres?title=R and B Mar 24 15:13:31 tmm tmm[824]: Rule vmod_search : It worked! /vMoD/vMoDGenres?title=R%20and%20B

But the tcpdump shows:

15:13:31.821046 10.10.50.202.38458 > 10.10.50.236.http: P [tcp sum ok] 1:58(57) ack 1 win 4380 (DF) (ttl 255, id 50573, len 109)

0x0000 4500 006d c58d 4000 ff06 3c33 0a0a 32ca E..m..@…<3..2.

0x0010 0a0a 32ec 963a 0050 8280 d372 c314 39d8 ..2..:.P…r..9.

0x0020 8018 111c 811f 0000 0101 080a 4e3a 2f0e …N:/.

0x0030 5e35 f30e 4745 5420 2f76 4d6f 442f 764d ^5..GET./vMoD/vM

0x0040 6f44 4765 6e72 6573 3f74 6974 6c65 3d52 oDGenres?title=R

0x0050 2532 3061 6e64 2532 3042 2061 6e64 2042 %20and%20B.and.B

0x0060 2048 5454 502f 312e 300d 0a0d 0a .HTTP/1.0…

You can see that BIG-IP is putting “and B” and the end of the URI.

I just can’t figure out where it is padding that. I changed the logging and it shows the same if I comment out the “encoding” section. I am wondering if it is being padded after the rule is executed.

any help would be appreciated.

Well it certainly looks as though the rule thinks things are being processed properly. I’d make sure that your last log statement saying “it worked!..” is actually capturing the HTTP::uri value, after it’s set.

I’m also curious what your rule looks like at this point.

According to what I can see in the logs though, it looks like the rule is functioning properly.

-Colin

The rule is:

when CLIENT_ACCEPTED {

TCP::collect 60

}

when CLIENT_DATA {

set matchedURI [regexp -inline “/vMoD.+HTTP” [TCP::payload]]

log local0. “first match $matchedURI”

if { [string length $matchedURI] > 0 } {

set newURI [getfield [getfield $matchedURI " HTTP" 1] “\{” 2]

log local0. “URI was matched. Result: $newURI”

set uriRewrite 1

} else {

log local0. “no match”

set uriRewrite 0

}

}

when HTTP_REQUEST {

if {$uriRewrite == 1} {

log local0. “Encoding $newURI”

regsub -all " " $newURI %20 encURI

set encURI [string map { " " %20} $newURI]

HTTP::uri $encURI

log local0. “It worked! [HTTP::uri]”

}

}

Interesting thing is, if I send a request without space, or %20 already in it, then it does not pad extra characters, and it always seems to padd whatever is after the first space that gets replaced.

I suspect the problem is that you are trying to manipulate the HTTP::uri variable in the HTTP_REQUEST and you have already confirmed that an issue exists when there is a space in the URI and the HTTP parser gets a hold of it.

i.e.

You are modifying the HTTP::uri contents, but some misc crud (The end of a portion of the URI string is still hanging around and not being accounted for) is still in the buffer to send to the client.

My guess.