Check protocol in iRule

The HTTP_REQUEST event is triggered when the HTTP headers in the request are parsed. If the request doesn't contain valid HTTP, the event won't be triggered. Events following HTTP_REQUEST will not be triggered either.

To validate the request is HTTP as you're intending, you'd need to trigger the collection of the TCP payload in CLIENT_ACCEPTED and look for a string like HTTP (as in the HTTP in 'GET / HTTP/1.0') once the data is collected in CLIENT_DATA.

If you wanted to pass non-HTTP traffic through the VIP, you would want to disable the HTTP profile after determining it's not HTTP in the CLIENT_DATA event.

Enforcing HTTP method used against a list of allowed methods is a good step towards validating HTTP requests. As a step towards tigher security, you might want to remove the PUT, DELETE, CONNECT, TRACE and OPTIONS unless the application specifically requires their use.

Here is an (untested) example of the above steps:

 
 when RULE_INIT { 
  
     Log debug to /var/log/ltm? 1=yes, 0=no. 
    set ::http_debug 1 
  
 } 
 when CLIENT_ACCEPTED { 
  
    if {$::http_debug}{ log local0. "[IP::client_addr]:[TCP::client_port]: Collecting data"} 
  
     Trigger the collection of at least 15 bytes of data. 
    TCP::collect 15 
 } 
 when CLIENT_DATA { 
  
     Log the collected data. 
    if {$::http_debug}{ log local0. "[IP::client_addr]:[TCP::client_port]: collected payload ([TCP::payload length]): [TCP::payload]"} 
  
     Check if the collected payload doesn't contain HTTP. 
    if { not ([TCP::payload] contains "HTTP") } { 
  
        Disable the HTTP profile as this doesn't appear to be HTTP. 
       HTTP::disable 
  
        Forward request?  Else, it will be sent to the VIP's default pool. 
  
       if {$::http_debug}{ log local0. "[IP::client_addr]:[TCP::client_port]: Releasing.  We collected this much data: [TCP::release]"} 
  
        Stop processing the rule for this connection. 
       return 
  
     } 
 } 
 when HTTP_REQUEST { 
  
     This event will only be triggered if the HTTP profile is enabled 
    if {$::http_debug}{ log local0. "[IP::client_addr]:[TCP::client_port]: request was parsed as HTTP"} 
  
     Perform validation of HTTP method? 
 } 
 

You can check the iRule wiki for details on the various commands (Click here😞

Aaron

What happens if the client uses a protocol, such as ssh, to the virtual server where the client doesn't send any data until it has received some data from the server? Will the connection not hang as the CLIENT_DATA event will never be triggered?

 

 

Is there same way to timeout the TCP::collect on the client side and make an assumption that if no-data arrives then it is not HTTP?

 

 

Thanks.

Aaron,

 

we've thought about using an array to keep the time a client connects to a server. The array would be indexed by src and dest IP address. The first connection request would timeout but for the second one we could use the data in the array to make a qualified guess that the client isn't talking HTTP.

 

Getting the dest IP at the time we need it (CLIENT_CONNECT) remains to be a challenge as I was unable to do so. I think the example for "serverside" at devcentral.com is wrong and the error message is not encouraging:

 

Jun 13 07:14:42 tmm tmm[953]: 01220001:3: TCL error: Rule basicHttpDetection - Error: No peer connection established IP::remote_addr needs an established peer connection! (line 1) invoked from within "IP::remote_addr" peer expression (line 1) invoked from within "serverside {IP::remote_addr}"

 

BTW we're using LTM 9.3.1 HF1.

 

 

As a single client can do multiple e.g. ssh requests over port 80 to the same server one needs locking which I haven't found neither for TCL nor iRules.

 

Finally since the array would be global we also have to think about garbage collection to avoid the array growing unlimited.

 

 

For my taste these are too many problems for a kludge and I'd really appreciate if you could give us TCP::collect with a timeout like

 

TCP::collect n t

 

where n is the number of bytes to collect and t is the timeout in milli secs.

 

 

Thanks.

 

I agree it would be a kludge to let the first connection hang and then know to disable HTTP for the subsequent connections. I think it would technically be possible to do using the steps you've outlined. You can get the destination IP address (ie, what IP the client requested) using IP::local_addr in the clientside context. The server IP address doesn't exist in the CLIENT_ACCEPTED event as the server side connection hasn't been established yet. So running 'serverside {IP::remote_addr} will result in a TCL error. You could use the session table (Click here) to record the client and destination IP's with a timeout value. If a matching session table exists in CLIENT_ACCEPTED, then disable HTTP. If it doesn't exist, then add a session table entry in CLIENT_ACCEPTED. In HTTP_REQUEST, you would remove the session table entry as you know the client-server IP pair is HTTP.

 

 

One question I have is: do you need to use an HTTP profile for some connections? Are you wanting to parse the HTTP traffic as HTTP? If not, you could just remove the HTTP profile from the VIP. Do you have a list of known HTTP servers? If so, you could create either more specific virtual servers or a datagroup containing the list of IP addresses (or subnets) for which to leave the HTTP profile enabled. Or is this VIP handling traffic for an arbitrary collection of port 80 connections?

 

 

I like the idea of adding a timeout option to the collect commands. However, I'm not a developer (or an F5 employee). You could propose this officially by opening a 'request for enhancement' case with F5 Support (Click here). If you end up doing this and getting a change request (CR) number, please reply with it here so others can reference it if they want to add another request to the RFE.

 

 

Aaron

I still think you'll hit a chicken & egg problem there... Or even a why bother, depending on how you look at it, and exactly WHY you want to do this.

 

 

Firstly, to disable HTTP processing, you need to gather enough characters to see if it's an HTTP request or not (For protocols that aren't expecting a greeting)... And that's going to be dependent on whether the client is asking for a really long URI, or a really short one... (Because the HTTP/x.x is at the END of the line. A long URI pushes it out as far as several hundred chars).

 

 

However...

 

 

Protocols that do expect a greeting are going to time themselves out anyway... At the expense of using up a connection to your F5 until they do (Or until the F5 gets tired of waiting and closes the connection itself). You have no way of knowing at this time what sort of banner they want, or even if thy want a banner... (Hence chicken & egg).

 

 

But...

 

 

You don't explain why you're trying to solve the problem... Or even what the problem is other than some users 'accidentally' open a connection to the VS and aren't intending on talking HTTP. But that doesn't really constitute a problem... (Sorry, thinking out loud here).

 

 

Are you experiencing any actual issues from this? The default idle timeout on a tcp profile is 300 seconds... What about setting that a lot lower? 10s? Would that help?

 

 

H

 

 

 

Posted By hoolio on 06/13/2008 2:58 AM

 

 

I like the idea of adding a timeout option to the collect commands. However, I'm not a developer (or an F5 employee). You could propose this officially by opening a 'request for enhancement' case with F5 Support (Click here). If you end up doing this and getting a change request (CR) number, please reply with it here so others can reference it if they want to add another request to the RFE.

 

 

Aaron

 

 

 

A CR exists for implementing the TCL 'after' command (CR83581). My understanding is that it was requested for exactly this scenario (SSH over HTTP).

How to have your client and server too:

 

 

I want to address a few things before delving into how to solve this problem, because there are some twists and turns ahead.

 

 

First, this configuration does make sense since azzurrow has a forwarding virtual server. There's no real load balancing going on here; we already know what the server is from the original TCP destination address. The LTM in this setup is acting as a sort of transparent proxy.

 

 

Second, there have been some very clever attempts at solving this problem, and I'd like to commend you all for trying to come up with ways to use the tools at your disposal. Thanks also to hoolio for succinctly explaining the serverside command.

 

 

Third, as rfcarr mentions, there is a CR for implementing timers, and we're working on implementing them, but no promises for when they'll be released. However, a timer is fundamentally a bad way to solve this problem as it would introduce some amount of unnecessary and annoying latency into connections, and would result in weird timing issues that are frustrating and hard to debug.

 

 

Fortunately, the theoretical solution is at hand. As explained above, the LTM already knows what the server is as soon as the client connects to it. So in theory, it could connect to the server and see which side sends it data first. The problem, as you have all discovered, is that TCP::collect (which you will need to call in order to inspect the data) prevents the serverside connection from being established.

 

 

There is a way around this, and azzurrow, you were shockingly close to what it is. The TCP::collect command will take a (currently undocumented) second numeric parameter, which is not a timeout value, but rather a "skip" value, a number of bytes to skip (allow through) before doing a collect. By providing this, the serverside connection is not held up and will be initiated normally. In this case, we don't actually want to skip any bytes, so we can specify a "0", and this will fortunately still initiate a serverside connection. Now, this parameter isn't documented yet for a reason: this mode of operation is not well-tested. Until relatively recently, I suspect it was never used, in fact. So using this parameter carries a high risk of unexpected behavior. I'd highly recommend extensive testing before deployment. Also, several bugs related to this were fixed in v9.4.5, so I'd strongly recommend an upgrade to that version before using it. If that is absolutely not an option, the at the very least you should build your iRule on v9.4.5 and then test on whatever version you can use in production to see if it still works for you. More caveats are discussed below.

 

 

So, now that you know this, how can you use it? Here's a simple example:

 

  
  when CLIENT_ACCEPTED {  
    TCP::collect 20 0  
  }  
  when SERVER_CONNECTED {  
    TCP::collect 20  
  }  
  when CLIENT_DATA {  
    log local0. "Got client data first"  
    event disable all  
  }  
  when SERVER_DATA {  
    log local0. "Got server data first"  
    event disable all  
  }  
  

 

 

This rule will log a message as to which side sent data first, and not much else. How about something more complicated? I have not tested this precise version of the code, but I have used similar iRules, and this does pass the syntax checker:

 

  
  when CLIENT_ACCEPTED {  
      TCP::collect 8 0  
      set server_hold 0  
      log local0. "Buildup -- collecting data"  
  }  
  when CLIENT_DATA {  
        Since we're here, the clientside sent data first  
       log local0. "Got client data"  
    
   Extract the HTTP command, if any.  
        Find the first space in the text, and trim off  
        any characters after it. If there's no space,  
        $spcpos will be -1, so [string range] will   
        return the empty string.  
       set spcpos [string first " " [TCP::payload 8]]  
       set command [string range [TCP::payload 8] 0 $spcpos]  
    
        Now see if this is a recognized HTTP command  
       if { [matchclass $command equals ::http_cmds] } {   
            log local0. "HTTP traffic -- command $command"  
            if { $server_hold eq 1 } {  
              serverside { TCP::release }  
            }  
            event SERVER_CONNECTED disable  
            event SERVER_DATA disable  
            return  
       }  
       log local0. "Not HTTP -- staying in passthru"  
       event disable all  
  }  
  when SERVER_CONNECTED {  
       log local0. "Server connection established"  
       TCP::collect  
       set server_hold 1  
  }  
  when SERVER_DATA {  
       log local0. "Banner protocol -- staying in passthru"  
       clientside { TCP::release }  
       TCP::release   
       event disable all  
  }  
  

 

 

This detects HTTP connections by extracting the command and comparing it against a http_cmds class, very similarly to what you wanted your original rule to do. The server_hold logic is to handle the case where the client might send us data before the serverside connection has completed (so SERVER_CONNECTED may or may not have fired by the time CLIENT_DATA does).

 

 

However, despite the fact that this iRule is looking for HTTP data, note that there are no HTTP events or commands here. This rule will not work if you have an HTTP profile on your virtual, as the HTTP profile will also prevent a serverside connection from being initiated. The clever way around that would be to run HTTP::disable in CLIENT_ACCEPTED and then run HTTP::enable after you knew you were dealing with HTTP traffic... except that in many versions of LTM, HTTP::enable just doesn't work. This is something else that was fixed in v9.4.5, so if true HTTP functionality is something you need, then v9.4.5 is the way to go. Again, this may work in earlier versions (like v9.3.x), but it may crash your box instead, so "test, test, test" is the order of the day. As I said earlier, v9.4.5 also fixes some other problems with using this mode of operation, so even if you don't need HTTP but you do want to do something fancy like use the "virtual " command, or LB::detach or something, then you'll still need to upgrade to that version.

 

 

I hope this helps.