The TAO of Tables - Part One

This is a series of articles to introduce you to the many uses of tables.

Many developers have heard about them but few have had the opportunity to use them. In this series of articles I will take you on a journey from the very beginning to the complex and marvellous creations we can make using them. Their true power lies solely in your mind and how you might use them.

For instance; recently I was asked how can I track the hosts connecting to my service and if possible the number of times they have connected?

table incr –subtable client_list [IP::client_addr]

That’s it? One command! Yes. That’s it. Let’s break it down… in a subtable name called “client_list” store entries who’s key is the clients IP address and value is the number of times they have hit your virtual service. But… hang on are we talking connections here or requests? Ahh well, that will depend on the iRule event you use. CLIENT_CONNECTED will represent TCP connections whereas HTTP_REQUEST will represent every single request. So let go with HTTP_REQUEST and this becomes

when HTTP_REQUEST {

table incr –subtable client_list [IP::client_addr]

}

So now we focus on HTTP requests however this will register all the elements on a page such as images and css. If that’s not what you are expecting then you need to add a filter so only HTML pages are captured. If your site uses aspx pages then check for that…

when HTTP_REQUEST {

                if { [URI::basename [HTTP::uri]] ends_with “.aspx” } {

table incr –subtable client_list [IP::client_addr]

}

}

This is not going to match “/”. However many sites these days will redirect “/” to the proper page name and since you are here to measure HTML page calls and not redirects you may not have to modify this further.

This looks good but we have missed a few things. All table entries have a timeout and optional expiry time. By default this is 120 seconds. We need to specify how long we want this information to be stored. In this case, since we want absolute page counts we don’t want the records to expire. Since we cannot set the timeout using the table incr command then we have to use another.

when HTTP_REQUEST {

                if { [URI::basename [HTTP::uri]] ends_with “.aspx” } {

table incr –subtable client_list [IP::client_addr]

table timeout –subtable client_list [IP::client_addr] indef

}

}

Ok we are progressing but now we have introduced another problem to consider. By using the indef command these table entries will never be removed unless we remove them or there is a box reset. While they do not take up a lot of memory, when you add something like this, it is effectively a memory leak. It will reduce the available memory to the TMM kernel over time and therefore you should be careful to manage this usage. We will get to that later but first, having this information stored in your F5 is great but how do you get to it? Well the simplest way is to display it! Now remember that tables are global objects in memory so you can use something like this on any virtual on the same F5 to display your results.

when HTTP_REQUEST {

                if {[HTTP::uri] ne “/status” } { return }

                set response “<html><head>Client Connections</head><body>”

                foreach ip [table keys –subtable client_list] {

                                append response “$ip = [table lookup –subtable client_list $ip]<br>”

                }

                append response”</body></html>”

                HTTP::respond 200 content $response Content-Type “text/html”

}

And if you want an xml response which you can parse into a database then you can use something similar to the following.

when HTTP_REQUEST {

                if {[HTTP::uri] ne “/xml” } { return }

                set response “<clients>”

                foreach ip [table keys –subtable client_list] {

                                append response “<$ip>[table lookup –subtable client_list $ip]</$ip>”

}

                append response”</clients>”

                HTTP::respond 200 content $response Content-Type “application/xml”

}

So that’s the solution. It’s a very simple command, triggered in the right place at the right time that will store a ton of useful information. The kind of that can be used for developing firewall rules for your service. Especially in a circumstance where you come across an existing service where the clients are unknown and auditing is required.

Now I said we would get back to memory management. If you want to reset your solution then you can use the following, again on any virtual, to reset the solution.

when HTTP_REQUEST {

                if {[HTTP::uri] ne “/reset” } { return }

                set response “<html><head>Client Connections</head><body>”

                table delete –subtable client_list -all

                append response “Table deleted.</body></html>”

                HTTP::respond 200 content $response Content-Type “text/html”

}

So the fundamental lessons from part one are tables are global memory storage across the device. They can be used in powerful ways quite simply to produce detailed information about what is connecting to, or passing through a virtual. I encourage readers to sit back and think of ways they might find storing information useful in their environment.

I have kept this first article quite simple as an introduction. Next week we will show you some of the more funky uses of tables.

Kevin Davies
iRules for Breakfast ~ How many do you do?
kevin.davies@rededucation.com

Updated Aug 16, 2022

Version 3.0