Getting Started with iRules: Directing Traffic
The intent of this getting started series was to be a journey through the basics of both iRules and programming concepts alike, bringing everyone up to speed on the necessary topics to tackle iRules in all their glory. Whether you’re a complete newbie to scripting or a seasoned veteran, we want everyone to be able to enjoy iRules equally, or at least as near as we can manage. As we wrap this series, it is our hope that it has been helpful to that end, and that you'll be eager to move on to intermediate topics in the next series. In this final entry in the series, we’re taking a look at what I think is likely the thing that many people believe is the primary if not one of the sole uses of iRules before they are indoctrinated into the iRuling ways: routing. That is, directing traffic to a particular location or in a given fashion based on … well, just about anything in the client request. Of course, this is only scratching the surface of what iRules is capable. Whether it's content manipulation, security, or authentication, there’s a huge amount that iRules can do beyond routing. That being said, routing and the various forms that it can take with a bit of lenience as to the traditional definition of the term, is a powerful function which iRules can provide. First of all, keep in mind that the BIG-IP platform is a full, bi-directional proxy, which means we can inspect and act upon any data in the transaction bound in either direction, ingress or egress. This means that we can technically affect traffic routing either from the client to the server or vice versa, depending on your needs. For simplicity’s sake, and because it’s the most common use case, let’s keep the focus of this article to only dealing with client-side routing, e.g. routing that takes place when a client request occurs, to determine the destination server to deliver the traffic to. Even looking at just this particular portion of the picture there are many options for routing from within iRules. Each of the following commands can change the flow of traffic through a virtual, and as such is something I’ll attempt to elucidate: pool node virtual HTTP::redirect reject drop discard As you can see there are some very different commands here, not all of them what you would consider traditional “routing” style commands, but each has a say in the outcome of where the traffic ends up. Let’s take a look at them in a bit more detail. pool The pool command is probably the most straight-forward, “bread and butter” routing command available from within iRules. The idea is very simple, you want to direct traffic, for whatever reason, to a given pool of servers. This simple command gets exactly that job done. Just supply the name of the pool you’d like to direct the traffic to and you’re done. Whether your criteria for allowing traffic to a given pool, or whether you’re trying to route traffic to one of several different pools based on client info or just about anything else, an iRule with a simple pool command will get you there. The pool command is the preferred method of simple traffic routing such as this because by using the pool construct you’re getting a lot of bang for your buck. The underlying infrastructure of monitors and reporting and such is a powerful thing, and shouldn’t be overlooked. There is, however, one other permutation of this command: pool [member []] Perhaps you don’t want to route to one of several pools, but instead want to route to a single pool but with more granularity. What if you don’t want to just route to the pool but to actually select which member inside the pool the traffic will be sent to? Easy enough, specify the “member” flag along with with the IP and port, and you’re set to go. This looks like: when CLIENT_ACCEPTED { if { [IP::addr [IP::client_addr] equals 10.10.10.10] } { pool my_pool } } when HTTP_REQUEST { if { [HTTP::uri] ends_with ".gif" } { if { [LB::status pool my_Pool member 10.1.2.200 80] eq "down" } { log "Server $ip $port down!" pool fallback_Pool } else { pool my_Pool member 10.1.2.200 80 } } } node So when directing traffic to a pool or a member in a pool, the pool command is the obvious choice. What if, however, you want to direct traffic to a server that may not be part of your configuration? What if you want to route to either a server not contained in a particular pool, whether that’s bouncing the request back out to some external server or to an off the grid type back-end server that isn’t a pool member yet? Enter the node command. The node command provides precisely this functionality. All you have to do is supply an IP address (and a port, if the desired port is different than the client-side destination port), and traffic is on its way. No pool member or configuration objects required. Keep in mind, of course, that because you aren’t routing traffic to an object within the BIG-IP statistics, connection information and status won’t be available for this connection. when HTTP_REQUEST { if { [HTTP::uri] ends_with ".gif" } { node 10.1.2.200 80 } } virtual The pool command routes traffic to a given pool, the node command to a particular node…it stands to reason that the virtual command would route traffic to the virtual of your choice, right? That is, in fact, precisely what the command does – allows you to route traffic from one virtual server to another within the same BIG-IP. That’s not all it does, though. This command allows for a massively complex set of scenarios, which I won’t even try to cover in any form of completeness. A couple of examples could be layered authentication, selective profile enabling, or perhaps late stage content re-writing post LB decision. Depending on your needs, there are two basic functions that this command provides. On is to add another level of flexibility to allow users to span multiple virtuals for a single connection. This command makes that easy, taking away the tricks the old timers may remember trying to perform to achieve something similar. Simply supply the name of whichever virtual you want to be the next stop for the inbound traffic, and you’re set. The other function is to return the name of the virtual server itself. If a virtual name is not supplied the command simply returns the name of the current VIP executing the iRule, which is actually quite useful in several different scenarios. Whether you’re looking to do virtual based rate limiting or use some wizardry to side-step SSL issues, the CodeShare has some interesting takes on how to make use of this functionality. when HTTP_REQUEST { # Send request to a new virtual server virtual my_post_processing_server } when HTTP_REQUEST { log local0. "Current virtual server name: [virtual name]" } HTTP::redirect While not something I would consider a traditional routing command, the redirection functionality within iRules has become a massively utilized feature and I’d be remiss in leaving it out, as it can obviously affect the outcome of where the traffic lands. While the above commands all affect the destination of the traffic invisibly to the user, the redirect command is more of a client-side function. It responds to the client browser indicating that the desired content is located elsewhere, by issuing an HTTP 302 temporary redirect. This can be hugely useful for many things from custom URIs to domain consolidation to … well, this command has been put through its paces in the years since iRules v9. Simple and efficient, the only required argument is a full URL to which the traffic will be routed, though not directly. Keep in mind that if you redirect a user to an outside URL you are removing the BIG-IP and as such your iRule(s) from the new request initiated by the client. when HTTP_RESPONSE { if { [HTTP::uri] eq “/app1?user=admin”} { HTTP::redirect "http://www.example.com/admin" } } reject The reject command does exactly what you’d expect: rejects connections. If there is some reason you’re looking to actively terminate a connection in a not so graceful but very immediate fashion, this is the command for you. It severs the given flow completely and alerts the user that their session has been terminated. This can be useful in preventing unwanted traffic from a particular virtual or pool, for weeding out unwanted clients all-together, etc. when CLIENT_ACCEPTED { if { [TCP::local_port] != 443 } { reject } } drop & discard These two commands have identical functionality. They do effectively the same thing as the reject command, that is, prevent traffic from reaching its intended destination but they do so in a very different manner. Rather than overtly refusing content, terminating the connection and as such alerting the client that the connection has been closed, the discard or drop commands are subtler. They simply do away with the affected traffic and leave the client without any status as to the delivery. This small difference can be very important in some security scenarios where it is far more beneficial to not notify an attacker that their attempts are being thwarted. when SERVER_CONNECTED { if { [IP::addr [IP::client_addr] equals 10.1.1.80] } { discard log local0. "connection discarded from [IP::client_addr]" } } Routing traffic is by no means the most advanced or glamorous thing that iRules can do, but it is valuable and quite powerful nonetheless. Combing advanced, full packet inspection with the ability to manipulate the flow of traffic leaves you with near endless options for how to deliver traffic in your infrastructure, and allows for greater flexibility with your deployments. This kind of fine grained control is part of what leads to a tailored user experience which is something that iRules can offer in a very unique and powerful way. Internal Virtual Server This isn't directly an iRules routing technology, though there are plenty of iRules entry points into this unique routing scenario on BIG-IP, so I thought I'd share. The internal virtual server is reachable only by configuration of an adapt profile on a standard virtual server. Once the configuration routing is place, events and commands related to content adaption (ICAP, though not required) are available to make decision on traffic manipulation and further routing. Check out the overview on AskF5.iRules Style Guide
This article (formatted here in collaboration with and from the notes of F5er Jim_Deucker) features an opinionated way to write iRules, which is an extension to the Tcl language. Tcl has its own style guide for reference, as do other languages like my personal favorite python. From the latter, Guido van Rossum quotes Ralph Waldo Emerson: "A foolish consistency is the hobgoblin of little minds..," or if you prefer, Morpheus from the Matrix: "What you must learn is that these rules are no different than the rules of a computer system. Some of them can be bent. Others can be broken." The point? This is a guide, and if there is a good reason to break a rule...by all means break it! Editor Settings Setting a standard is good for many reasons. It's easier to share code amongst colleagues, peers, or the larger community when code is consistent and professional looking. Settings for some tools are provided below, but if you're using other tools, here's the goal: indent 4 spaces (no tab characters) 100-column goal for line length (120 if you must) but avoid line continuations where possible file parameters ASCII Unix linefeeds (\n) trailing whitespace trimmed from the end of each line file ends with a linefeed Visual Studio Code If you aren't using VSCode, why the heck not? This tool is amazing, and with the F5 Networks iRules extension coupled with The F5 Extension, you get the functionality of a powerful editor along with the connectivity control of your F5 hosts. With code diagnostics and auto formatting based on this very guide, the F5 Networks iRules Extension will make your life easy. Seriously...stop reading and go set up VSCode now. EditorConfig For those with different tastes in text editing using an editor that supports EditorConfig: # 4 space indentation [*.{irule,irul}] indent_style = space indent_size = 4 end_of_line = lf insert_final_newline = true charset = ascii trim_trailing_whitespace = true Vim I'm a vi guy with sys-admin work, but I prefer a full-fledge IDE for development efforts. If you prefer the file editor, however, we've got you covered with these Vim settings: # in ~/.vimrc file set tabstop=4 set shiftwidth=4 set expandtab set fileencoding=ascii set fileformat=unix Sublime There are a couple tools for sublime, but all of them are a bit dated and might require some work to bring them up to speed. Unless you're already a Sublime apologist, I'd choose one of the other two options above. sublime-f5-irules (bitwisecook fork, billchurch origin) for editing Sublime Highlight for export to RTF/HTML Guidance Watch out for smart-quotes and non-breaking spaces inserted by applications like Microsoft Word as they can silently break code. The VSCode extension will highlight these occurrences and offer a fix automatically, so again, jump on that bandwagon! A single iRule has a 64KB limit. If you're reaching that limit it might be time to question your life choices, I mean, the wisdom of the solution. Break out your iRules into functional blocks. Try to separate (where possible) security from app functionality from stats from protocol nuances from mgmt access, etc. For example, when the DevCentral team managed the DevCentral servers and infrastructure, we had 13 iRules to handle maintenance pages, masking application error codes and data, inserting scripts for analytics, managing vanity links and other structural rewrites to name a few. With this strategy, priorities for your events are definitely your friend. Standardize on "{" placement at the end of a line and not the following line, this causes the least problems across all the BIG-IP versions. # ### THIS ### if { thing } { script } else { other_script } # ### NOT THIS ### if { thing } { script } else { other_script } 4-character indent carried into nested values as well, like in switch. # ### THIS ### switch -- ${thing} { "opt1" { command } default { command } } Comments (as image for this one to preserve line numbers) Always comment at the same indent-level as the code (lines 1, 4, 9-10) Avoid end-of-line comments (line 11) Always hash-space a comment (lines 1, 4, 9-10) Leave out the space when commenting out code (line 2) switch statements cannot have comments inline with options (line 6) Avoid multiple commands on a single line. # ### THIS ### set host [getfield [HTTP::host] 1] set port [getfield [HTTP::host] 2] # ### NOT THIS ### set host [getfield [HTTP::host] 1]; set port [getfield [HTTP::host] 2] Avoid single-line if statements, even for debug logs. # ### THIS ### if { ${debug} } { log local0. "a thing happened...." } # ### NOT THIS ### if { ${debug} } { log local0. "a thing happened..."} Even though Tcl allows a horrific number of ways to communicate truthiness, Always express or store state as 0 or 1 # ### THIS ### set f 0 set t 1 if { ${f} && ${t} } { ... } # ### NOT THIS ### # Valid false values set f_values "n no f fal fals false of off" # Valid true values set t_values "y ye yes t tr tru true on" # Set a single valid, but unpreferred, state set f [lindex ${f_values} [expr {int(rand()*[llength ${f_values}])}]] set t [lindex ${t_values} [expr {int(rand()*[llength ${t_values}])}]] if { ${f} && ${t} } { ... } Always use Tcl standard || and && boolean operators over the F5 special and and or operators in expressions, and use parentheses when you have multiple arguments to be explicitly clear on operations. # ### THIS ### if { ${state_active} && ${level_gold} } { if { (${state} == "IL") || (${state} == "MO") } { pool gold_pool } } # ### NOT THIS ### if { ${state_active} and ${level_gold} } { if { ${state} eq "IL" or ${state} eq "MO" } { pool gold_pool } } Always put a space between a closing curly bracket and an opening one. # ### THIS ### if { ${foo} } { log local0.info "something" } # ### NOT THIS ### if { ${foo} }{ log local0.info "something" } Always wrap expressions in curly brackets to avoid double expansion. (Check out a deep dive on the byte code between the two approaches shown in the picture below) # ### THIS ### set result [expr {3 * 4}] # ### NOT THIS ### set result [expr 3 * 4] Always use space separation around variables in expressions such as if statements or expr calls. Always wrap your variables in curly brackets when referencing them as well. # ### THIS ### if { ${host} } { # ### NOT THIS ### if { $host } { Terminate options on commands like switch and table with "--" to avoid argument injection if if you're 100% sure you don't need them. The VSCode iRules extension will throw diagnostics for this. SeeK15650046 for more details on the security exposure. # ### THIS ### switch -- [whereis [IP::client_addr] country] { "US" { table delete -subtable states -- ${state} } } # ### NOT THIS ### switch [whereis [IP::client_addr] country] { "US" { table delete -subtable states ${state} } } Always use a priority on an event, even if you're 100% sure you don't need them. The default is 500 so use that if you have no other starting point. Always put a timeout and/or lifetime on table contents. Make sure you really need the table space before settling on that solution, and consider abusing the static:: namespace instead. Avoid unexpected scope creep with static:: and table variables by assigning prefixes. Lacking a prefix means if multiple rules set or use the variable changing them becomes a race condition on load or rule update. when RULE_INIT priority 500 { # ### THIS ### set static::appname_confvar 1 # ### NOT THIS ### set static::confvar 1 } Avoid using static:: for things like debug configurations, it's a leading cause of unintentional log storms and performance hits. If you have to use them for a provable performance reason follow the prefix naming rule. # ### THIS ### when CLIENT_ACCEPTED priority 500 { set debug 1 } when HTTP_REQUEST priority 500 { if { ${debug} } { log local0.debug "some debug message" } } # ### NOT THIS ### when RULE_INIT priority 500 { set static::debug 1 } when HTTP_REQUEST priority 500 { if { ${static::debug} } { log local0.debug "some debug message" } } Comments are fine and encouraged, but don't leave commented-out code in the final version. Wrapping up that guidance with a final iRule putting it all into practice: when HTTP_REQUEST priority 500 { # block level comments with leading space #command commented out if { ${a} } { command } if { !${a} } { command } elseif { ${b} > 2 || ${c} < 3 } { command } else { command } switch -- ${b} { "thing1" - "thing2" { # thing1 and thing2 business reason } "thing3" { # something else } default { # default branch } } # make precedence explicit with parentheses set d [expr { (3 + ${c} ) / 4 }] foreach { f } ${e} { # always braces around the lists } foreach { g h i } { j k l m n o p q r } { # so the lists are easy to add to } for { set i 0 } { ${i} < 10 } { incr i } { # clarity of each parameter is good } } What standards do you follow for your iRules coding styles? Drop a comment below!Getting Started with iRules: Intro to Programming with Tcl
In this Getting Started with iRules series, We’ll cover topics ranging from this first installment, which includes some programming basics and concepts, up through F5 terminology and concepts, iRules basics and usage, components, etc. This primer is low level, intended for those that might be looking to connect some core concepts of programming and how they interact, as well as a refresher for those that haven't spent much time in the trenches scripting. In this first installment of the series the idea is to discuss programming at more of a general, basic level. Think of it as a sort of “What you need to know” style introduction. For many this will be extremely low level review, so feel free to skip ahead in the series. For those that may be truly new to programming as a whole or feel that you’re rusty enough that you could use a “from the beginning” refresher, the concepts detailed here are important and hopefully will be useful information. They’re certainly concepts that are important to understand if you hope to begin your journey into iRules Fu greatness. So what do you need to know? Let’s get started. A Programmer's Glossary First let’s talk about some very core programming concepts. It’s important that as the series goes on the basic terms and concepts are understood, so think of this as sort of a getting started glossary. The basic things that you need to know at least a bit about to describe “programming” as a concept are: Commands Variables Conditionals Operators We’ll take each of these in turn and give a very brief overview. Commands The concept of commands is an exceedingly simple one, for the most part. Each programming language has an array of many different commands, which can be executed to achieve different desired results. Host name lookups, spawning child processes, accessing the shell to run a command, storing information in memory, directing traffic, or any number of other things are achieved by a command, or a series of commands called from within the code in question. At the core the commands in any code are the part of the code that actually “does something”. The rest is there to manipulate information, create logical support and flow, etc. Without the commands to actually perform an action, there would be no effect when any script was run, despite how pretty the logic and code was. For a basic look at how you'd call some commands programatically in a few different languages, see below: Variables Variables are integral parts of any programming language as they are used to store information in memory. By storing information in a variable you are then able to later recall it later, manipulate that information, pass it to a given command, etc. For instance: If I have code that performs a host lookup to retrieve an IP address, I could execute commands on the result of that and things would work just fine. What happens the next time I want to execute commands against the resulting IP address? My code has to call the operation that did the lookup again. What if, on top of multiple iterations, there were also multiple actions involved? Perhaps I want to compare the IP destination of a given host to value, then based on the result execute a command using that host’s IP. Without storing the result from the host lookup I’m now looking at two lookups every time I execute my snippet of code. By storing the resulting IP in a variable I would be able to perform a single lookup and use the variable for as many iterations and operations as I want without having to perform another lookup every time. Variables are also essential for many things such as conditional testing, string manipulation and more, but more on that later. You can see a basic example of a few different languages and the way they handle simple variables below: Conditionals Conditional statements are a type of control structure in programming that allows you to create a logical flow within your code. For instance if you want to perform a given action every time your script is run, you’d just execute the relevant command. If, however, you only want to run a specific command under particular circumstances, such as only redirecting traffic to a particular server if the primary server is down for instance, you need a way to logically state that. The most basic conditional statement, the if conditional, allows you to do things like dictate that a command or set of code only be executed if a given condition is met, hence the term “conditional”. For instance “look up the IP of this hostname but only if no one has requested it before”. Again, to see what basic conditional constructs look like, we go to a simple chart of a few relatively standard languages: Operators There are two main types of operators that we’ll cover here, logical and comparison. Comparison Operators With commands, variables and conditionals you’ve got a pretty strong skeleton of basic programming functionality, but a very important piece is missing. Comparison operators are an integral part of any language as they are what allow you to evaluate and compare values against one another. What good is a conditional statement saying “if x = 1 do y” without the equals? Standard comparison operators are things we’ve all seen since early math classes. =, >=, <=, != are among the commonly used. There are, of course, others however. Things like regular expression based matching, substring matching and more are all available in most languages. The general concept of a comparison operator is simple though, at its core it is designed to compare one value against another. Whether that is the result of a command being compared against a static value to see if they are equal or two variables being compared to see which one has a greater numeric value, it’s all thanks to comparison operators. Logical Operators The other major type of operator that we’re concerned about in this series is the logical operator. Logical operators are used to build logical expressions which can be useful in conjunction with conditionals to create a slightly more complex logical flow in your code. For instance a standard conditional might say “if x = y … do something”. But what if you want to do that same thing if x is equal to either y or z? Well, you could either write the same code block twice, once for the case of “x=y” and again for “x=z”, but not only would that be wildly inefficient, it would cause all sorts of problems down the road. A simple logical or can simplify this immensely. Simply changing the conditional statement to “if x = y or if x = z … do something” will get the result you’re looking for without the need to repeat your logic and without some of the major pitfalls that making such blatant repetitions can cause if you’re not careful. That should serve as a very rudimentary glossary of terms for just about any programming language out there. Every language has its own nuances and intricacies of course. The language on which iRules is based is Tcl (Tool Command Language). With it Tcl brings its own unique traits in syntax as well as functionality. Tcl Specifics - The What and Why There were many choices when building iRules as to which language would be best suited for the purpose. There are many reasons that Tcl was chosen, but chief among them are performance, customization and ease of use. While it may not be the most robust language when comparing raw, base functionality to others, Tcl serves the purposes of iRules quite well. Performance was a massive factor when considering a language on which to base iRules. Tcl is lightweight, compact and highly performant. In an environment where each “script” may be executed hundreds of thousands of times per second, performance is absolutely paramount. The ability to be compiled ahead of run time is a massive win in the battle for performance. Unlike many interpreted languages Tcl does not have to be interpreted at run time, but rather it can be pre-compiled and run in an extremely efficient manner, which allows for exceptionally high processing rates compared to many similar languages. The efficient and easy to use C API within Tcl is also a major boon as most of the commands built into iRules are custom built. Many of the calls made from iRules make use of this API. Because of this, the customization capabilities of a language had to be taken into account as well when considering languages. Tcl is easily modified and amended to fit our purposes. As the majority of commands that make it into iRules are in fact custom, this is a necessity that Tcl provides well. No language was as network aware as required for our purposes and adding custom commands is the way in which we were able to shape a language to fit the needs of the ADC role, blending network and application functionality, that iRules plays so well. To that end, any language also needed to be easy to use to properly fit the bill for iRules. Tcl is simple to read, quick to learn and easy to troubleshoot even when passing from person to person, at least compared to some of the more complex options. This is especially important given the nature of the code often written in iRules (somewhat simple, quickly produced code when compared to a product release cycle). For the complete story on the why of Tcl you can read more in depth here. For now though, this should hopefully serve as a brief introduction into what benefits Tcl itself brings into the iRules picture. Note - This content is restructured from earlier series written by Colin Walker, Joe Pruitt, Jason Rahm, & Deb Allen.Getting Started with iRules: Events & Priorities
As this series steams on we go deeper and deeper into what actually drives iRules as a technology. So far we have covered very basic concepts, from core programming ideas and F5 basic terminology through to what makes iRules unique and useful, when you’d make use of them, etc. This is all-important as a base of knowledge from which we can draw when speaking about the technology, but now it’s time to dig in deeper and start discussing the inner workings of iRules and what makes them tick. If you haven’t yet read the first three primer style introduction installments of this article series it is likely worth your time to at least peruse them to be sure you’re on the same page before delving into the rest of the series. In this installment, we’ll cover two core concepts within iRules: Events and Priorities. While we will eventually get to commands and different fun tricks that you can do with iRules, sure enough, it’s important to understand how iRules actually work first. I can think of no better starting place than events and our built in eventing structure. Events What is an event? iRules is an event driven language. What this means is that there is no freestanding code to be executed each time the script body as a whole is called. Rather code is placed under events, which act like containers, categorizing and separating the code within an iRule into logical sections. This is important to allow iRules to be truly network aware. When programming in a traditional scripting environment it is normal for the entirety of a script to be executed each time it is called. In a network based scripting environment however you ideally want sections of code to be called at specific moments within the connection flow. I.E. you want to look up an HTTP host name, want to do it after the connection is established, the handshake is performed and the appropriate headers are sent that contain the information you’re looking for, but before that request is sent off to the destination server. This means you have to have some way to understand and describe, in your code, that moment in time. There are many similar moments, such as when a connection is first established, when a server is selected for load balancing, when the connection is sent to that server, when a server responds, and so on. It is because of this that iRules is designed as an event driven language. Thanks to the flexibility of Tcl and our ability to easily modify it to our needs, we were able to extend it to understand a vast number of these moments within the session flow. It is those moments that we refer to as events. Rather than reading every packet that comes across the wire and waiting until you see the information that depicts the HTTP request has finally come across from the user, an iRules user can simply call the HTTP_REQUEST event and trust that the BIG-IP will execute the code contained within that event block at the appropriate moment. So, very simply, an “Event” when speaking about iRules is a Tcl command that we’ve added to iRules to describe specific moments within the session flow of a network connection, such as "CLIENT_ACCEPTED" and "HTTP_REQUEST" below. when CLIENT_ACCEPTED { if { [IP::client_addr] eq “192.168.1.1” } { drop } } when HTTP_REQUEST { log local0. “Host: [HTTP::host], Client IP: [IP::client_addr]” } What are the benefits of an event driven language? In addition to being able to accurately execute code at the specific time desired, which is important considering that often times information will be needed that is only available at specific points in the flow, an event driven language is also a large benefit to performance. In traditional programming when you make a call to a script the entire script body is executed. This means that if you have a 100 line script written in say Perl or Ruby (nothing against those languages, merely an example), every time that script is called all 100 lines will be executed, generally speaking. This is not true in iRules. With an iRule that is 100 lines long, but spread across 3 or 4 different events, only the specific code applied to each individual event is executed when that event occurs, rather than the entirety of the script. For instance, if you have a 100 line iRule with 50 lines in a request event and 50 lines in a response event, your BIG-IP is only executing 50 lines of code for every inbound request, rather than having to execute the entire rule. This may not seem like a massive savings, but keep in mind that everything when talking about network side scripting performance becomes more important than in traditional environments, because of the scale generally being discussed. While a traditional script may be executed several hundred times per second in a busy environment, an iRule may be executed several hundred thousand times per second, in a high traffic deployment. Another way of looking at this, perhaps, is that events themselves are the containers for the scripts being executed against the network. The iRule is a way of grouping those events into a logical container to express a function or segment of application or business logic that you wish to apply as a single object against your network traffic. What events are available? Events are broken down into protocols, with usually several events per supported protocol. Events will always follow a profile within BIG-IP, so you can largely correlate which available protocol profiles are available with which protocols have specific events called out within iRules. The list is wide-ranging and I won’t list it here, but it is available on DevCentral here (requires registration) for your perusal. From HTTP to SIP to DNS, most major protocols are represented in the listing and have specific iRules Events, and likely associated commands, designated to them. Keep in mind, however, that even if a particular protocol does not have events specifically set out for it you can accomplish almost anything with base TCP and UDP events and commands. With a strong understanding of iRules and the protocol in question it is entirely possible to parse and manipulate nearly anything passing across the wire with the commands and events already available within the TCP and UDP command sets. It’s also worth noting that particular events can be disabled for the duration of the connection with the event disable command. Be cautious when using this, as it will most certainly affect any other iRules applied to the same virtual. In which order do events fire? This is a question that gets asked time and time again on DevCentral and out in the field, and it is admittedly a bit difficult to answer. The way that events fire is largely based on the network flow through the proxy within BIG-IP. The proxy architecture within the BIG-IP is made up of a series of filters, each designed to perform specific functions, without getting into tedious detail. Each filter builds upon the last, I.E. the HTTP filter uses information drawn out of the session by the TCP filter, etc. Each of these filters has an associated set of iRules events that fire at that point in the session’s life cycle as well, when the connection is being interrogated or affected by the particular filter in question. As the connection is passed up the chain more and more specific filters fire, along with events to match, until finally the session is handed over from the client side of the proxy to the server side, and it traverses its way down the chain on that side. The return trip for responses follows a similar path with different filters, and as such events, in place; herein lies part of the problem with describing the order of events within an iRule. There is no static set of events that happen for every single connection. There may be a few specific events that always fire, such as CLIENT_ACCEPTED once per connection, but in general there is no set event structure that fires. The events that fire are dependent on the configuration of the virtual to which the iRule in question is applied. Because of this dynamic nature it is extremely hard to make a blanket statement about which events will fire in a given iRule, let alone event order. Given the entire config of a virtual server it is possible to determine which events will fire, and in which order, but that requires case by case information and analysis. Below are screenshots of iRule events graphed with visio and dot/graphviz, respectively, followed by links to the pdfs of the event flows from each project. This shows the way events relate given configuration and traffic considerations, and it is a good starting point for understanding event ordering. BIG-IP Event Order (Visio) BIG-IP Event Order (GraphViz) What happens if I have the same event in multiple iRules on a virtual server? Many people will have multiple iRules on a single virtual server, some of which will make use of like events. The question of what happens in such a case is often asked, or a user will want to know “Which CLIENT_ACCEPTED will fire first?” If you have a virtual server with multiple iRules all making use of a particular event, there is no “first” or “last” occurrence of that event, it only happens once. To understand a bit more, first let me explain how iRules are stored and interpreted. iRules are not, for all intents and purposes, a dynamically interpreted language. That is, Tcl interpreter is not spun up every time a request comes into a virtual server with an iRule applied. This would be horribly inefficient. Instead, when an iRule is saved as part of a configuration, it is interpreted by the Tcl engine and pre-compiled down into what is known as “byte code”. This is a compiled state that breaks down most of the logic and commands within a given iRule so that at execution time it can very easily and efficiently be fired by the Tcl engine, in this case part of TMM. During this process the iRule is also effectively broken down and re-organized. The commands that are associated with each event within the iRule are removed from the context of that particular iRule and are instead associated with that event. Remember above where I mentioned the idea of events being tied to a particular filter in the proxy architecture, and that the events fire when those filters are called? This is why that concept is important. The commands you put under your HTTP_REQUEST events are all taken out of whichever iRules they were associated with and instead relegated to live under the event that is responsible for executing those commands. As such, there is no concept of multiple instances HTTP_REQUEST or CLIENT_ACCEPTED or SIP_RESPONSE or any other event occurring with a given virtual server. The session traffic proceeds up the chain, events fire, and commands are executed in that context, be it client or server. Regardless of how many iRules you have that call a given event, that event only ever fires once per connection/request, depending on the event. Even if you can’t have multiple instances of a given event executed from a particular virtual server, you can still control the order in which your iRules execute. This is done via the next concept we’ll cover, priorities. Priorities What are priorities? Priorities are an integer value associated with every iRule that determine the order in which iRules applied to a given virtual server execute, the lower the number, the higher the relative priority. I.E. an iRule with a priority of 1 would fire before an iRule with a priority of 900. If you want to ensure that the variable you’re setting in iRuleA exists when iRuleB executes, you’ll want to set priorities accordingly. How do I set priority? Every iRule has a priority, whether you set one or not. The system will implicitly set a priority value (500 is the default) to all iRules so that it can keep straight the order in which to execute commands at run time. This is done automatically when saving iRules without a specified priority. One nuance to this is that all iRules without an explicit priority ordered in the GUI will be prioritized top to bottom, even though they all share the implicit value. If, however, you want to set a specific priority, it is done so simply via the priority command. At the beginning of your iRule you can state “priority 100” to designate the priority for that iRule and, by association, all of the commands contained within. If you’re going to start setting specific priorities it is recommended to do so for all of your iRules, however, to be sure that the execution order you want is achieved. priority 100 when HTTP_REQUEST { if {[HTTP::host] eq “mydomain.com” } { set host [HTTP::host] } } Why does priority matter? In many cases, it doesn’t. The vast majority of iRules deployments that I’ve seen do not have specific priorities set. It can be useful, however, in cases where you want to ensure that a particular order of execution is achieved. A couple of possible examples may be if you’re passing particular information from one iRule to another, or want an iRule to act on a modified value of some sort. Whatever the reasons, it is likely that if you aren’t already looking for a way to set priority, you don’t need to worry about it. Can I supersede the session flow? First the simple answer: No. Now the explanation: People sometimes ask if they’re able to do things like act on a response before a request, view a load balancing decision as soon as the connection is established, or other things that are unnatural in regards to the flow of a session through the BIG-IP. This is not possible. By their very nature events occur based on the way traffic traverses the proxy, and no amount of priority will change that. You can absolutely ensure that iRuleA’s response event occurs before iRuleB’s response events, but you cannot force iRuleA’s response events to occur before iRuleB’s request events. You can achieve a similar effect with a couple of conditionals and variables, and this is the logic trick that I’d recommend to anyone who needs to mimic this behavior. Keep in mind that this immediately ceases to function effectively when you have multiple iRules on the virtual server, unless all of them are built with this in mind. Use caution, as this is absolutely risky. It is not, however, possible to alter the actual event structure or ordering. when HTTP_REQUEST { if { [info exists response_fired] } { unset response_fired ... } } when HTTP_RESPONSE { set response_fired 1 ... } Picking up what we’re laying down so far? Proceed to the next installment in this series, which will cover variables.Getting Started with iRules: Control Structures & Operators
So far in the first several installments of this series we’ve talked about some solid introduction topics, from programming basics to F5 terminology to covering an introduction of what iRules is as a technology, how it works, and how to make use of it. These introductions are a solid stepping-stone to get people on the same page so that we can delve into iRules without losing people along the way, or so goes the theory. In the last couple of installments we moved past introductions and started talking about iRules proper, discussing events, which are a foundational piece of the iRules framework, and priorities. It’s important to understand events and the way iRules work within an event driven framework, so I highly suggest giving at least that installment, as well as the article on variables a read if you are new to the series. Building from those ideas we’re now going to cover control structures and operators, two more fundamental and important pieces of iRules that are integral to just about any iRules script that you may write. Combining these along with events will give you the basic framework for a logical flow within almost any iRule, with varying degrees of complexity, of course. Control Structures What is a control structure? A control structure is a logical statement that will allow you to make a comparison and, based on the result of that comparison, perform some set of actions. If you’ve done any form of scripting whatsoever you’ve used them, if, else, switch…etc. Whether this is checking to see if a value is equal to zero and then only executing a block of code if that is true, or determining whether or not an inbound IP address is in the allowed list before granting access to a restricted portion of your application, any decision you’re making within your iRule is made using control structures. Just like without events, you’d be unable to execute code at the proper moment in time, based on what’s happening with network flow; without control structures you wouldn’t be able to make logical separations in which code gets executed under certain circumstances which, obviously, is paramount to building a functional script. What different control structures are there? In iRules the main control structures you’ll be working with are “if”, “else”, “elseif”, “switch”, and “class”, which is a bit of a special case, but we’ll discuss how that works. Let’s go through a simple look at each of these available options: if The if statement is by far the commonly used and widely understood form of a logical control structure out there. It’s in every language I’ve ever seen, and makes sense to just about anyone without any form of explanation whatsoever. Quite simply, this statement says “only execute the below code if the statement contained results as true”. A quick example would be: if {[info exists $auth]} { pool auth_pool } else Else, in and of itself, isn’t really a control structure, but is rather an optional addition to the if statement. It allows you to make a logical separation between the two outcomes of an if statement. So logically the above example reads as “if the auth variable is set, send traffic to the pool named auth_pool”. But…what happens in this case if the auth variable isn’t set? Nothing, as it stands right now. If we want to change that, we can use the else statement to describe what we want to happen. For example: if {[info exists $auth]} { pool auth_pool } else { pool other_pool } elseif So now we can make a pretty solid logical statement based on a single comparison (does the auth variable exist, in the above example). What if, however, we want to be able to make multiple comparisons and act differently based on which outcome proves to be true? We need to create a multi-choice logical statement, which can be done a couple of ways. The first, likely most common way is with the elseif statement, which is also associated with the if structure. With an if/else statement you are saying “if x, do y else do z”. With an elseif clause you can effectively say, “if x do y, if not x but a do b”, etc. It sounds much more complex trying to spell it out than it actually is, an example is probably a simpler way of depicting how elseif fits in with if and else: if {[info exists $auth]} { pool auth_pool } elseif {[info exists $secondary]} { pool secondary_pool } else { default_pool } switch Having covered if, else, and elseif we’ve covered the most common control structures used in most cases. That does not, however, mean that they are necessarily the best choice in every case. They are widely used because they are widely known, but there’s another option that, in many cases, can prove superior in not only readability but also in all important performance. A switch statement can be used to replace an if, an if/else, an if/elseif chain, and more. It’s a very versatile, compact statement and it would behoove any would be iRuler to learn the ins and outs of bending switch statements to their will. The one big caveat with a switch statement is that while it supports multiple match options, it does so against a single comparison string. Taking a look at a basic switch statement that inspects the URI and acts based on the contents looks like this: switch [HTTP::uri] { “/app1” { pool http_pool } } This effectively says “Inspect the HTTP::uri variable, and if it is /app1, send traffic to the http_pool”. This is a pretty basic if statement. That, however, isn’t anywhere near all of what switch can do. What if you wanted to do multiple possible matches against the URI. I.E. “If the HTTP::uri is /app1, send to http_pool, if it is /app2, send to http_pool2, if it is /app3, send to http_pool3”. That would be simple with switch, and would look like: switch [HTTP::uri] { “/app1” { pool http_pool } “/app2” { pool http_pool2 } “/app3” { pool http_pool3 } } Note the lack of need for else or elseif statements, and the simple, clean logical flow. Last let’s look at a logical or statement, a particularly useful trick that you can perform with a switch statement. If your logic statement looks like “If the URI is /app1, /app2, or /app3, send to the http_pool”. The switch statement to satisfy that requirement would be as simple as: switch [HTTP::uri] { “/app1” – “/app2” – “/app3” { pool http_pool } } The dash after a switch statement allows it to fall through in a logical “or” fashion, making it easy to chain multiple match cases to go along with a single action, as above. Finally, with the switch statement, you can use glob-style pattern matching. switch -glob [HTTP::uri] { "*\\?*ghtml*" { #do work here } {*\?*ghtml*} { #do work here } } Where the pattern is something like *\?*ghtml* and you are using double quotes, note that string substitution occurs before processing the glob pattern argument, so two back slashes are required to escape the question mark. By using the curly braces, you avoid the string substitution and the glob pattern can be matched as expected. Which control structure is right for me? There is much more to learn about the ins and outs of the different control structures, but that should be enough to get you started. Now the tough choice – which to use? While there is not a hard and fast set of rules saying “this is when you must use x vs y”, there are some general guidelines to live by: If is the most commonly used for a reason. If you are doing 1-3 comparisons, if or if/else(if) are probably the most appropriate, and easiest to use and understand. If you are making 3-10 comparisons, you should likely be using a switch statement. They are higher performance, easier to read, and much simpler to debug, generally speaking. Past 10-15 entries, you should be using a data group and the class command, but we’ll cover more on that in another article. What should I keep in mind when setting up my structure? When building out your control structures it’s important, regardless of whether it’s a series of if/else comparisons or a switch statement, to do your best to order them for efficiency. By putting the most frequently matched items towards the top of the list you can improve efficiency dramatically. The deeper the match depth in the if/elseif chain or the switch statement the more comparisons need to be made before arriving at the match. Comparison Operators What is a comparison operator? A comparison operator (just operator for our purposes) is how you define what type of comparison you’re going to use when comparing two items. Every if or switch requires some kind of true/false result, and most of these will make use of a comparison of some sort. Deciding “if {$x eq $y}” or “if {$z > 5}” requires the use of such operators, as does just about any logical decision being made. As such, they’re exceedingly common and frequently used. What different comparison operators are there? The main comparison operators are: Operator Example equal “$x eq $y” or “$x == 5” not equal “$x ne $y” or “$x != 5” contains [HTTP::uri] contains “abc” starts_with [HTTP::uri] starts_with “/app1” ends_with [HTTP::path] ends_with “.jpg” There are obviously more (>, %lt;, >=, <=, etc) but the above are the most commonly used, generally speaking. Comparison operators aren’t iRules specific, so I won’t belabor all of them here. Suffice to say most commonly accepted comparison operators will work in Tcl, and if they work in Tcl they’ll work in iRules. What are the pros and cons of the different operators? The most important thing to remember when dealing with operators is “the more specific the better”. Always use equals when you can, use starts_with or ends_with over contains when you can, etc. The more specific the operator you’re using the less work that needs to be done to identify whether or not the comparison will return true or not. This is directly visible in a savings in CPU cycles, and as such is worth some thought when writing your iRules. There is also the notion of polymorphism to take into account but that has already been covered, and tends to be a more advanced concept, so if you’d like to read more please do, but we don’t need to cover it in any great detail here. That wraps up control structures and operators. Next week we’ll dig into delimiters, covering the different types available within iRules, the pros and cons, the situations in which you’d use each, and more.Getting Started with iRules: Variables
If you've been following along in this series, it's time to add another building block to the framework of what iRules are and can do. If you're new, it would behoove you to start at the beginning and catch up. So far we've covered introductions across the board for programming basics and concepts, F5 terminology and basic technology concepts, the core of what iRules are and why you'd use them, as well as a couple of cornerstone iRules concepts like events and priorities. All of these concepts are required to get a proper understanding of the base iRules infrastructure and functionality. Using those concepts as foundations we're going to add to the mix something that's integral to any programming language: variables. We'll cover a bit about what variables are, but also some iRules specific variables functionality and commentary. Things to look for in this article: What is a variable? How do I work with variables in my iRule? What types of variables are available to me in iRules? Is there a performance impact when using variables? When should I use variables? What is a variable? A variable, in simple terms, is a piece of data stored in memory. This is done usually with the notion of using that data again at some point, recalling it to make use of it later in your script. For instance if you want to store the hostname of an incoming connection so that you can reference that hostname on the response, or if you want to store the result of a given command, etc. those things would be stored in a variable. Every scripting language has variables of some form or another, and they're quite important in the grand scheme of things. Without them you'd be building static scripts to perform one off, iterative tasks. Variables are a large part of what allows dynamic programming to account for multiple conditions and use cases. Whether it's storing the data from a math operation to be compared against a desired result, or checking to see if a given condition has been met yet, variables are at the very core of just about everything in programming, and aren't something that we could live without in modern coding. The idea is simple, take a piece of data, whether it's a number or a string or...anything, and store it in memory with a unique name. Then, later, you can recall the value of that data by simply referencing the name you created to represent it. You can, of course, modify, or delete variables at will also. How do I work with variables in my iRule? There are two main functions when it comes to any variable: setting and retrieving. To set a variable in Tcl you simply use the set command and specify the desired value. This can be a static or dynamic value, such as an integer or the result of a command. The desired value is then stored in memory and associated with the variable name supplied. This looks like: #Basic variable creation in Tcl set integer 5 set hostname [HTTP::host] To retrieve the value associated with a given variable you simply reference that variable directly and you will get the resulting value. For instance "$integer" and "$hostname" would reference the values from the above example. Of course calling them by themselves won't do much good. Simple referring to a variable within your code will do pretty much nothing. You'll almost certainly be referencing the variables in relation to something or from another command. I.E.: #Basic variable reference to retrieve and use the value stored in memory if {$integer > 0) { log local0. "Host: $hostname" } What types of variables are available to me in iRules? This topic can creep in scope pretty darn quickly, given that variables are just a simple memory structure, and there are many different types of memory structures available to you via Tcl and iRules. From simple variables to arrays to tables and data groups, there are many ways to manage data in memory. For the purposes of this article, however, we're going to focus on the two main types of actual variables and leave the discussion of other data structures for later. There are two main types of variables in iRules, local and global. Local Variables All variables, unless otherwise specified, are created as local variables within an iRule. What does that mean? Well, a local variable means that it is assigned the same scope as the iRule that created it. All iRules are inherently connection based, and as such all local variables are connection based as well. This means that the connection dictates the memory space for a given iRule's local variables and data. For instance, if connection1 comes in and an iRule executes, creating 5 variables, those variables will only exist until connection1 closes and the connection is terminated on the BIG-IP. At that time the memory allocated to that flow will be freed up, and the variables created while processing that particular connection's iRule(s) will no longer be accessible. This is the case with all iRules and all variables created from within an iRule using the basic set command structure pictured above. Local variables are low cost, easy to use, you never have to worry about memory management with them, as they are automatically cleaned up when the connection terminates. It's important to remember that iRules as a whole, and the variables therein are connection bound. This can cause some confusion when people are expecting a more static state. Local variables will account for the vast majority of your variable usage within iRules. They're efficient, easy to use, and highly useful depending on the situation. These can be set directly as shown above but are also often the result of a command that provides output. There are also multiple ways to reference a variable within iRules. When using a command that directly affects the variable it is usually appropriate to leave the "$" off and reference the name directly, other times braces allow you to more clearly define the beginning and end of a variable name. Some examples would look like: #Standard variable reference log local0. "My host is $host" #Set variable to the output of a particular command using brackets [] set int [expr {5 + 8}] #Directly manipulating a variable's value means no "$", most times incr int #Bracing can allow you to deliniate between variable name and adjacent characters log local0. "Today's date is the ${int}th" Global Variables "Global variables" is a bit of a misnomer, actually. This is the general terminology used within most programming languages, including Tcl, for variables that exist outside of the local memory space. I.E., in the case of iRules, variables that exist beyond the constraints of a single connection. For instance, what if I want to store the IP address of my logging server, and have that always available, to every connection that comes through the BIG-IP, without having to re-set that variable every single time my iRule fires? That would be a global variable. Tcl has a mechanism for handling this, and it's relatively easy to use. Interestingly, though...we aren't using it. The global variable handling within Tcl requires a shared memory space, which in our world does something referred to as "demoting" from CMP. As you'll recall from the "Introduction to F5 Technology & Terms" article, CMP is Clustered Multi-Processing, and is the technology that allows us to distribute tasks within the BIG-IP to multiple cores in an efficient, scalable manner. Because of the requirements of the default Tcl global variable handling, making use of a global variable within Tcl forces any connection going through the Virtual to which the offending iRule is applied to be demoted from CMP, I.E. use only a single processing core, rather than all of them to process the traffic for that Virtual. This is a bad thing, and will severely limit performance. As such, we strongly recommend avoiding global variables in their traditional sense all together. But what about that log server address? There's still a definite need for long lived data to be stored in memory and available at will. It's for this purpose that we've included a new namespace within iRules called the "static" namespace. You can effectively set static global variable data in the static namespace without breaking CMP, and thereby not decreasing performance. To do so simply set up your static::varname variables, likely in RULE_INIT, since that special event only runs once at load time and static variables are global in scope, meaning they stay set until the configuration is reloaded. Once these variables are set you can call them like you would any other variable from within any iRule, and they will always be available. It looks something like this: #Set a static variable value, which will exist until config reload, living outside of the scope of any one particular iRule set static::logserver "10.10.1.145" #Reference that static variable just as you would any other variable from within any iRule log $static::logserver "This is a remote log message" For a more complete look at the different types of memory structures that you can access in iRules, I've included a handy dandy table to show memory structures by type along with some information about each, and an example of using that structure. We'll cover some of these in more detail in later installments of this series, but I want to make you aware of the different types of memory structures available. Note: There is a slight error in the table. It IS possible to unset static variables. Is there a performance impact when using variables? Running any command that isn't cached in any script has some cost associated with it, there's just no way around it. Variables in iRules happen to have a miniscule cost, generally speaking, so long as you're using them appropriately. The cost associated with a variable is in the creation process. It takes resources, albeit a very small amount, to store the desired data in memory and create a reference to that data to be used in the future. Accessing a variable is simply making a call to that reference, and as such doesn't have an additional cost. A common misunderstanding, however, is just what constitutes a variable within iRules, vs. a command or function that will perform a query. Many people see things like "HTTP::host" or "IP::client_addr" and, due to the format, assume it is a command or function and as such will cost CPU cycles to query the value and return it. This is not the case at all. These type of references are cached within TMM, so whether you call "HTTP::host" once or 100 times, you're not going to require more resources, as you're not performing a query to determine the hostname each time, rather just referencing a value that's already stored in memory. Think of these and many other similar commands as pre-populated variable data that you can use at will. When should I use variables? While the cost of creating variables is low, there still is some overhead associated with it. In iRules, due to the exorbitantly high rate of execution in some deployments, we tend to lean towards extreme efficiency mindedness. In this vein we recommend only using variables where they are actually necessary, rather than many programming practices which dictate to use them often as a means of keeping code tidy, even when not truly warranted. For instance, a common practice with many new iRule programmers is to do things like set host [HTTP::host] As I just explained above, however, this is completely unnecessary. Rather, it would be more efficient to simply re-use the HTTP::host command any time you need to reference this information. When it does make sense to use a variable, however, is when you are going to modify the data in some way. For instance log local0. "My lower case URI: [string tolower [HTTP::uri]]" The above will give you an all lowercase representation of the HTTP URI. This is a very common use case, and it is not abnormal to have the need to reference the lowercase version of the URI multiple times in a given iRule. While the HTTP::uri command is cached and will not incur additional overhead regardless of how many times you reference it, the string tolower command is not. As such, it would make sense in this case, assuming you're going to reference the lowercase URI at least 2 or more times, to create a variable and reference that: set loweruri [string tolower [HTTP::uri]] log local0. "My lower case URI: $loweruri" Effectively you want to use variables any time you are going to have to repeat any operation against a value that has a cost associated with it. Rather than repeat that operation multiple times and accumulate extra overhead it's better to perform the operation once, store the result, and reference the variable from there. That covers the basics of variables: What they are, how they work, when to use them and when not, which types you can make use of and how they'll affect your performance, if at all. Hopefully that allows you to approach your iRules with that much more confidence, understanding such a core piece of their makeup. In the next article we'll dig into control structures and operators.Getting Started with iRules: Delimiters
So far in this series we've covered introductions across the board for programming basics and concepts, F5 terminology and basic technology concepts, the core of what iRules are and why you'd use them, as well as cornerstone iRules concepts like events, control structures, and variables. In this article, we will add to the foundation with a detailed discussion on delimiters. Buckle up! Whitespace The primary (and oft overlooked) delimiter in tcl and iRules is whitespace. In tcl, basically everything is a command. Commands have names, options, and arguments. Whitespace (space or tab) is used to separate the command name and its options & arguments, which are simply strings. A newline or semicolon is used to terminate a command. In this example log local0. "iRule, Do you?" the components of the command when you break it down are: Command log local0. "iRule. Do you?" Command Name log Command Parameter local0. Command Parameter "iRule. Do you?" The space character is also used to separate the elements in a tcl list. Double Quotes Let's take a closer look at that last example. You might have noticed that the 2nd parameter above is a string consisting of multiple words separated by whitespace, but enclosed within double quotes. This is an example of grouping with double quotes. Everything between a pair of double quotes " ", including newlines and semicolons, are grouped and interpreted as a single contiguous string. When the enclosed string is evaluated, the enclosing quotes are discarded and any whitespace padding is preserved. A literal double-quote character can be included in a string delimited by double quotes if it is escaped with a backslash ( \" -- more on backslash substitution below). Variable substitutions are always performed within strings delimited by double quotes. Avoid using double quotes on already-contiguous alpha-numeric strings because that results in an extra trip through the interpreter to create an unnecessary grouping. However, even in contiguous strings, if using any characters besides alpha-numeric it's a good idea to enclose the entire string in quotes to force interpretation as a contiguous string. For example, one of the most commonly used iRules commands, HTTP::redirect, takes a fully qualified URL as an argument which should be double quoted: HTTP::redirect "http://host.domain.com/uri" Strings as operands in expressions must be surrounded by double quotes or braces to avoid being interpreted as mathematical functions. Square Brackets A nested command is delimited by square brackets [ ], which force in-line evaluation of the contents as a command. Everything between the brackets is evaluated as a command, and the interpreter rewrites the command in which it's nested by discarding the brackets and replacing everything between them with the result of the nested command. (If you've done any shell scripting, this is roughly equivalent to surrounding a command with backticks, although tcl allows multiple nesting.) In this example, the commands delimited by square brackets are evaluated and substituted into the log command before it is finally executed: log local0. "[IP::client_addr]:[TCP::client_addr] requested [HTTP::uri]" Strings contained within square brackets which are not valid tcl or iRule commands will generate an "undefined procedure" error when the interpreter attempts to interpret the string inside the brackets as a command. The use of square brackets for purposes besides forcing command evaluation requires escaping or backslash substitution. It's most optimal if you will be referring to the result of a command only once to use inline command substitution instead of saving the result of the command to a variable and substituting the variable. Backslash Substitution (Escaping) Backslash substitution is the practice of escaping characters that have special meaning ( $ {} [] " newline etc) by preceding them with a backslash. This example generates "undefined procedure: 0-9" error because the string "0-9" inside the [ ] is interpreted as a command: if { [HTTP::uri] matches_regex "[0-9]{2,3}"} { TCL error: undefined procedure: 0-9 while executing "[0-9] This example works as expected since the square brackets have been escaped: if { $uri matches_regex "\[0-9\]{2,3}" } { body } This example also works since the contents of the square brackets within the braces are not evaluated as a command (and it's also a bit more readable): if { $uri matches_regex {[0-9]{2,3}}}{ body } (The 2 approaches above are roughly equivalent performance-wise.) Another common use of backslash substitution is for line continuation: Continuing long commands on multiple lines. When a line ends with a backslash, the interpreter converts it and the newline following it and all whitespace at the beginning of the next line into a single space character, then evaluates the concatenated line as a single command: log local0. "[IP::client_addr]:[TCP::client_addr] requested [HTTP::uri] at [clock \ seconds]. Request headers were [HTTP::header names]. Method was [HTTP::method]" Backslash substitution, when required does not substantially affect performance one way or the other. Parentheses The most common use of parentheses ( ) in iRules is to perform negative comparisons. It's a common mistake to exclude the parentheses and unintentionally negate the first operand rather than the result of the comparison, which can result in the mysterious 'can't use non-numeric string as operand of "!"' error: If the first operand is non-numeric, the evaluation will fail, since a non-numeric value cannot be negated in the mathematical sense. Here's an example demonstrating the use of parentheses to properly perform a negative comparison. The following approach logs "no match", as expected, since $x is a string and not equal to 3: set x xxx if { !($x == 3) } { log "no match" } else { log "match } However, this (erroneous) approach results in the runtime error mentioned above: set x xxx if { !$x == 3 } { log "no match" } else { log "match" } TCL error: can't use non-numeric string as operand of "!" while executing "if { !$x == 3 } { log "no match" } else { log "match" }" Other uses for parentheses in iRules are for regular expression grouping of submatch expressions: regexp -inline (http://)(.*?)(\.)(.*?)(\.)(.*?) and to reference elements within an array: set array(element1) $val Braces And last but by no means least, braces are the most extensively used and perhaps least understood form of grouping in tcl and iRules. They can be confusing because they are used in a number of different contexts, but basically braces define a space-delimited list of one or more elements, or a newline delimited list of one or more commands. I'll try to give examples of each of the different ways braces are commonly used in iRules. Braces with Lists Lists in general are space delimited words enclosed in braces. Lists, like braces, may be nested. This example shows a list of 3 elements, each of which consists of a list of 2 elements: { { {ab} {cd} } { {ef} {gh} } { {ij} {kl} } } Braces with Subcommand Lists Some commands use braces to delimit lists of subcommands. The "when" command is a great example: In every iRule, braces are used to delimit all iRules events, encapsulating the list of commands that comprise the logic for each triggered event. The first line of each event declaration must end with an open brace, and the last line with a closing brace, like this: when HTTP_REQUEST { body } There are several other commands that use braces to group commands, such as switch: Syntax: switch condition body condition body Examples: switch { case1 { body } case2 { body } } or switch { case1 { body } case2 { body } } Braces with Parameter Lists Other commands use braces to define a parameter which is really a group of values - a list. The "string map" command, for example, requires a parameter containing the value pairs for substitution operations. The value pairs parameter is a list passed as a single parameter. Syntax: string map charMap string Example ("a" will be replaced by "x", "b" by "y", and "c" by "z"): string map {a x b y c z} aabbcc Braces with Parameter Lists and Subcommand Lists Combined And some commands use braces for both parameter lists and subcommand lists: Syntax: for start test next body Example: for {set x 0 } { x = 10 } { incr x } { body... ... } Nesting Braces Regardless of what context they are used in, multiple sets of braces can be nested. The innermost group will become an element of the next group out, and so on. All characters between the matching left and right brace are included in the group, including newlines, semicolons, and nested braces. The enclosing (i.e., outermost) braces are not included when the group is evaluated. There must be a matching close brace for every open brace -- even those enclosed in comments. Mismatched braces will generate a syntax error to that effect when you try to save the iRule. (Mismatches in other delimiters and invalid command syntax may also generate a mismatched braces error, so if you can't find mismatched braces, start commenting out lines without braces until the syntax check passes, then fix the problem in the last line you commented.) Special Cases for Using Braces You can also use braces to enclose strings, just as you would double quotes, but variables won't expand within them unless a command also inside the braces refers to them. Because of that difference, if you need to handle strings containing $, \, [], (), or other special characters, braces will work where double quotes won't. Braces may also be used to delimit variable names which are embedded into other strings, or which include characters other than letters, digits, and the underscore, enforcing the start and end boundaries of the variable name rather than relying on whitespace to do so. This example results in a runtime error since the end of the first variable name is unclear: set var1 111 set var2 222 log local0. "$var1xxx$var2" TCL error: can't read "var1xxx": no such variable while executing "log local0. "$var1xxx$var2"" But if you delimit the variable name in braces, it works: set var1 111 set var2 222 log local0. "${var1}xxx$var2" Returns: 111xxx222 Class names containing the dash character must be delimited by braces to ensure the class name is evaluated correctly. These are both valid class references: matchclass $::MyClass contains "Deb" matchclass ${::My-Class} contains "Deb" Forcing Variable Expansion within Braces If you want to use variables in the charMap value, you'll need to use a different way to pass a list to the string map command. As I mentioned above, variables aren't automatically expanded inside of braces. However, for some commands that accept a list as a parameter, it makes sense to pass variables to the command within a list, which is delimited by braces. To pass a list of expanded variables and/or literal strings, you can use an embedded "list" command to echo both the values of the variables and the literals into the enclosing command. This example shows how to pass a variable and a literal string as the charMap parameter to the "string map" command, substituting the string "NewName" wherever the string contained in $oldname is found in the HTTP Host header: Syntax: string map charMap string Example: string map [list $oldname NewName] [HTTP::host] Other Resources This article is by no means an exhaustive review of tcl delimiter use & optimization, but hopefully covers the basics well enough to get you started. Comments, corrections, & observations are welcome as always. Here are some additional resources you may want to check out: Summary of Tcl language syntax (tcl.tk) Summary of Tcl language syntax (sourceforge.net) Tcl Fundamentals Is white space significant in Tcl Quoting Hell Quoting Heaven! tcl list command If you have detailed questions about this or any other iRules questions, please ask! Join us for the next article in this series when we will cover logging and comments.Getting Started with iRules: Technology & Terms
In the last installment, we took a look at some programming fundamentals, with the hopes to get people on the same page so that we could start building towards some more advanced concepts. To do so, however, we would be remiss if we didn’t look at both sides of the equation. You see when looking to write iRules the programming side of things is only half of the picture. This is one of the things that makes iRules so unique. It is truly a network aware programming language, and as such you need to not only understand the fundamentals of programming, but also how F5 devices work, what options you have within the product, the terminology we use, etc. Let’s face it, if I hand you a bucket full of iRules commands and tell you to go solve a problem you aren’t going to get very far if you don’t know what a pool is, what to call a client IP address, or how to make use of a VIP. Consider the last article a starter’s guide for programming basics to help get the non-programmers up to speed. If that’s the case, this installment of #The101: iRules is intended to be a primer for non networking (or at least, non F5 aware) users to get up to speed. iRules is an amazingly powerful language, but it is only as powerful as the abilities of the person doing the coding, and those abilities start squarely with an understanding of what the device you’re programming is capable of. Using the same format as before, let’s take a look at a glossary of basic terms/technologies that you will see often come up when talking about or working with F5 technology. Here is a glossary of F5 technology for iRules programmers: Virtual Server Pool Pool Member Node Profile Client Side Server Side TMM CMP We’ll take each of these concepts in turn and give a brief overview to illuminate what they mean when speaking in the context of F5 devices. For the more detailed information as it relates to specific versions of TMOS, head over to My F5. Virtual Server (Virtual IP) A Virtual IP, or VIP, also known as a Virtual Server is a key component in any BIG-IP configuration. It’s kind of the starting point most times when people are thinking about building a configuration for a given application. The VIP is the destination (combination of IP and port) to which requests will be sent when bound for whatever application lives behind the BIG-IP. For instance if you have a server hosting your web application living behind an F5 device, it would no longer have a public facing internet address. Instead you would assign that public address to the BIG-IP in the form of a VIP with whichever accompanying port you are expecting traffic on, likely 80, 443 or all (*) in this case. So you would end up with a VIP on the front (or “client side”) of your BIG-IP that directs traffic to the server(s) on the back end. The VIP is integral as it is where all traffic is directed from the outside, where profiles and other configuration options are defined, and much more. It is not uncommon for application configurations to make use of multiple VIPs, especially when they receive traffic on multiple ports, or if they need to make use of multiple profiles for some reason or another (perhaps some requests use a client SSL certificate and others don’t). So it is important to remember that one VIP does not necessarily mean one application. A VIP is a configuration object on the BIG-IP that allows you to tie together a destination IP:port combination and process traffic for that combination. Whether it is to route to a back end server, redirect elsewhere, deny, discard, inspect, or simply log information about said traffic…there is a near limitless number of options with what you can do with the traffic once it enters the F5 device. To get it there, however, you’ll need a VIP. Pool A pool, in the simplest terms I can muster, is a grouping of servers. Like a VIP, a pool is an integral BIG-IP configuration object. This one, however, can be considered effectively one step lower in the configuration stack, as it were. Meaning you must have a VIP in place to allow traffic into your F5 device, generally speaking, and only once it’s there do pools become relevant. A pool is a collection of one or more servers, referred to as members, which we will get into In a moment. The pool is where the type of load balancing desired is chosen, where some options such as rate limiting and the like are applied, and one of the most important of the many layers of monitoring that can be applied to an application’s stack within a BIG-IP. Monitoring the pool level is important because it will allow you a clear representation of which groups of servers are or aren’t available at a given time. Each VIP has the option of selecting a default pool, but it is also possible to direct to another pool should the primary pool be unavailable. In some configurations there isn’t a default pool stated, and instead a pool is chosen based on criteria that is gleaned from the connection once it is in place. Regardless, pools are where the servers hosting the application being served live, and as such they are a crucial part of any deployment. Pool Member A member is one of the servers associated with a given pool. You will hear the term “pool member” often, and this is why, it is a term referring to one of the particular servers associated with the designated pool. Pool members play an important role, obviously, because they are the representation of the actual servers themselves in your configuration. The combination of a VIP, pool and pool members makes up the overall, general structure of a basic application stack within a BIG-IP. There are thousands of permutations and possible options of course, but this is the most basic, generic view, and is important to understand for a starting point. Pool members can have many options toggled on them as well, in addition to the configuration options already inherently in place on any traffic destined to the members of a pool due to the configuration of the pool or VIP upstream from the members in the config hierarchy. So again, generally speaking, traffic will come in destined for a particular VIP. That VIP will then route the traffic to a given pool, based on either the default pol selected for the VIP or some other criteria, perhaps an iRule. The traffic will then arrive at that pool and a load balancing decision will be made within the pool, based on currently available members and the select load balancing algorithm, and traffic will finally be directed to a pool member, which is the final destination (I.E. server) which will process the request and respond accordingly. Node What if you have a VIP, but don’t want to route traffic to the member of one of the pools on your BIG-IP. What if instead you would rather route traffic to a particular server via IP address, regardless of whether it is part of your configuration, lives behind your BIG-IP, or is a member of a pool? This is the concept of a node. A node is any destination IP to which you would like to direct traffic. It is not treated the same as a pool, no load balancing decision is made, no monitoring is done, it is a rather “fire and forget” type of action. It has many uses, but is not generally recommended for the majority of a normal application’s traffic flow. Still, the term will come up in the iRules world, as there is a specific command for sending traffic to a node from within an iRule, so it’s important to understand what one is. Profile A profile is the heart of much of the processing done for each session that is established and flows through the BIG-IP. A profile gets applied to a VIP and dictates what type of traffic is expected, TCP or UDP, it dictates whether or not SSL offloading will be done on the client side, which SSL profile is used if it is, whether or not a particular protocol will be used such as HTTP or SIP, whether or not ONECONNECT will be enabled, and much, much more. If the VIP is the destination for your application traffic think of the profile attached to said VIP as the control center that tags, inspects, interprets and categorizes the traffic once it arrives. A profile is essential for many reasons, not the least of which is that, from an iRules perspective, there are several commands that are only available for use on a VIP that has certain profiles applied. For instance the popular and often used HTTP commands are only available on VIPs that have an HTTP profile applied. This is because the profile applied to the VIP does a large amount of processing, as I’ve said, and the commands rely on some of the data that results from that processing and interrogation. There are many different kinds of profiles when talking about F5 technology. SSL profiles, Auth Profiles, Protocol Profiles and more. The concept is similar for all of them, with different nuances and outcomes. Most often when dealing with iRules you’ll need to be sure you’re using the appropriate Protocol profile (such as HTTP) and applying an SSL profile when required, if you’re trying to inspect and process SSL traffic. To do so you must terminate SSL at the BIG-IP, which is done by applying a ClientSSL profile to the VIP in question. Client Side Unlike the terms in this glossary up this point, client side is not an actual configuration object. It is instead a concept that I feel is important to describe and express to those looking to gain a level of comfort with F5 technology, and iRules in particular. The BIG-IP is a full proxy architecture, which means that there are separate IP stacks for each side of the proxy, client side and server side. This means that as traffic progresses through the BIG-IP , at some point it is transferred from one stack to the other, and vice versa on the responses from the server to the client. This is important because different profiles, configuration objects and iRules commands themselves are only available or function differently within different contexts. It is important to know whether you are trying to or need to affect client side traffic or server side traffic to accomplish what it is you’re looking to do. This is a term you will hear thrown around quite a bit as you delve deeper into working with F5 technology and particularly iRules, and the simplest way to describe it is “anything that occur on the client side of the proxy architecture is in the client side context”. Server Side Server side, as you can likely deduce, is the exact opposite of the above. It is a term used to describe something occurring on the server side of the proxy architecture. This is usually the responses from application servers, but keep in mind that client side and server side depend entirely on which side of the proxy initiates the transaction. If you have a server in a pool reaching out through your BIG-IP to perform an action of its own accord, that server is now the “client” for the purposes of the proxy discussion, and as such it is on the client side of the transaction, despite it being a server itself. TMM Someone could easily write a several thousand-page paper on the TMM in and of itself, so I will not attempt to discuss it in detail. I only want to ensure that when the term is referenced it is not completely foreign. The TMM is the Traffic Management Microkernel. It is the custom kernel that F5 developed specifically to handle traffic processing and routing. It is designed from the ground up to be high performance, reliable, and flexible for our needs. The TMM is what does all of the actual traffic processing on any BIG-IP. Whether it is an iRule being executed, a profile inspecting traffic as it comes through a VIP, or just about anything else that touches the traffic as it traverses an F5 device, it happens within the TMM. The TMM lives separately from the “host OS”, which is likely the only other important thing to know in regards to TMM at this point. The hostOS handles things such as syslog, sshd, httpd, etc. while leaving the TMM free to do only what it is best at – processing traffic in absurdly high volumes. CMP Again, there are papers that already exist that depict CMP in a far more thorough and articulate manner than I could hope to achieve, so I will give the primer version to elucidate the very basics of the concept. CMP is “Clustered Multi-Processing”. This is F5’s proprietary way of dealing with multiple core devices. In essence, in a very rough sense, each core in a device is assigned its own individual TMM (see above) to handle processing of traffic for that core. There is then a custom disaggregator (DAG) built into the system that decides which TMM, and as such, which core to send traffic to for processing. In this way F5 is able to achieve massively linear scalability in multiple CPU, multiple core systems. That barely skims the surface of this technology, but when discussing iRules it is important to know that CMP is a good thing, and breaking or demoting from CMP is, generally speaking, bad. Hopefully this explains why that is, and what CMP is at a basic level. Armed with a basic understanding of these concepts I am hopeful that the world of F5 and iRules will be far easier to understand for those that may not have been exposed to such before. Now that everyone has a solid basis in both programming and F5 concepts we will move forward with discussing iRules in particular. In the next article we will delve into iRules as a technology, how it works, why it exists, how to make use of an iRule and more.Getting Started with iRules: Logging & Comments
So far in this series we’ve covered some pretty varied topics, from a rudimentary primer on programming generalities to basic iRules components (and why they’re important.) In this article, we’ll cover a truly powerful yet simple function in nearly any programming language. Something that makes development easier, troubleshooting possible and has saved many developers long nights of agony trying to find metaphorical needles in haystacks. Today we’ll talk about logging and cover some basic questions: What is logging? What types of logging are available in iRules? Is there a cost to logging? When should I log and when shouldn't I? What are comments and how do they work? What does commenting my code cost me? When should I comment my iRule? Troubleshooting iRules - Where do I start? What is logging? Logging is simple, really. It’s the act of sending information from whatever script or program you’re writing to a log file. That may seem simple and mundane, and in many ways I suppose it is, but making good use of that functionality is paramount if you want to stay sane and build effective code. Logging is integral to almost any language for a number of reasons. Trapping errors, returning information to the system, relaying troubleshooting and development information, and more. Without log files and the logging functions built into most languages, including iRules, you’d be flying largely blind. Whether you’re on a BIG-IP or just about any other platform the logging concept stays the same. You have a script, you tell it to output information, and you have, generally speaking, two options on where to send it, assuming you’re not sending it off to a socket or another system somewhere. You can send it to one of the available outputs, often times the command line or screen, or you can capture it in a file. To make that whole “capture it in a file” bit easier and more predictable, we have standard log files available to us in most cases. In the case of the BIG-IP iRules log entries will go to /var/log/ltm by default. All you have to do is fire the log command and your information will show up for processing or perusal. A basic log entry contains the data and time of the entry, the facility, severity, log message and more. An example entry looks something like this: Oct 4 00:42:51 tmm err tmm[17084]: 01220001:3: - "Host: domain.com" What types of logging are available in iRules? That explains the “what”, but what about the “how”? In iRules, there are three main ways in which you can log information. Local Logging, Remote Logging, and High Speed Logging Local Local logging is the most basic and common form of logging information. This means that you execute a log command in your iRule and the information is sent to /var/log/ltm, as mentioned above. This is where all standard iRules information will go also, including any iRules error messages that may crop up from the system. Those standard logs are handled by syslog-ng, which means they’re stored in standard syslog format, in case that matters to you. This also means that the logging is not handled by TMM, but rather by the host OS, which will become relevant shortly. Logging locally is great for troubleshooting and for temporary log statements while in the development process. Local log statements do, however, write data back to the host OS and to disk, and that means they aren’t necessarily ideal for high traffic production deployments. It isn’t as great for performance in production deployments, especially high traffic production deployments. Local logging is simple to use. All you need is a log statement that specifies the log command, the log facility, and the message you want to enter into the log file. Log facilities are used to identify the sender of a log message, often times a daemon, and can dictate different logging behaviors in some cases. For our purposes in iRules we’re going to always use a log facility of “local0.”, except in rare, customized cases. The finished log command is quite simple to use, like so: when HTTP_REQUEST { log local0. “Requested hostname: [HTTP::host] from IP: [IP::local_addr]” } For the definitive documentation, more examples and notes check the log command pageon Clouddocs. Remote Logging remote is effectively the exact same idea as logging locally, I.E. still sent from the TMM in standard syslog format, still uses the log command, etc.; with one major difference: the logs aren’t actually stored locally. Instead, with remote logging, you’re able to specify a log server IP that will receive the formatted log statements sent directly from TMM. This is a far preferable option for performance reasons, though beware, a highly trafficked BIG-IP can easily overwhelm many logging servers if sending robust log information on each request. To remotely log using the log command you’d just modify the log command to include a remote IP address, like this: when HTTP_REQUEST { log local0 10.10.10.1 “Requested hostname: [HTTP::host] from IP: [IP::local_addr]” } High Speed Logging As of version BIG-IP version 10.1 there is a third and quite powerful option for logging. High Speed Logging, quite often referred to as HSL, is a way in which you can use TMM to send data off of the BIG-IP at an extremely high rate of speed, in a very efficient manner. These commands were originally designed to be used for logging, as the name indicates, but the reality is that they can be used to send nearly any data you want, as the data sent via HSL is not formatted in any specific way. The general idea behind HSL from a logging sense is the same as remote logging via the log command, except that you have much more control with the HSL commands. Because you are specifically sending data directly to an open HSL connection with the HSL::send command, you can send whatever data you want, in any format you’d like. This has also led people to using the HSL commands for things other than strictly log information. You can read more about the specific HSL commands here but a simple example looks like: when CLIENT_ACCEPTED { set hsl [HSL::open -proto UDP -pool syslog_server_pool] } when HTTP_REQUEST { # Log HTTP request as local7.info; see RFC 3164 Section 4.1.1 # "PRI Part" for more info HSL::send $hsl " [IP::local_addr] [HTTP::uri]\n" } Note that with HSL we had to specifically open a connection to a pool before we could send any log info…this is important. Without the HSL connection opened your send command(s) will fail. This is extremely similar to the sideband connections behavior introduced in a later (11.0) version. That is because sideband connections were developed as an extension of the HSL model once it was realized just how powerful this type of functionality could be. Is there a cost to logging? Yes. Technically speaking there is a cost for any operation you perform in an iRule, it’s all about how often you are performing each action, and the actual overhead of that action. While the log command itself is quite low in overhead, the things that happen after TMM executes its portion of the work become rather costly quickly. iRules, and just about everything else that gets run against traffic on the wire within a BIG-IP, are run from within TMM (Traffic Management Microkernel). These processes are highly optimized and tuned to handle wire-speed operations. An iRule telling TMM to fire the log command and send off data is low drag and happens quickly and easily, once that’s done however, the message is sent over to syslog-ng. Keep in mind that syslog-ng is run by the host OS, meaning it is not handled by TMM. Any time you have to involve the host OS in a wire-speed transaction it can be a rather expensive operation, as the host OS is just not optimized the way that TMM is to handle massive traffic volumes. On top of that, your log entry is going to be stored on disk. If accessing the host OS is bad, depending on disk I/O, possibly multiple times at that, for each request while processing traffic is…a bad idea. It’s because of this that we recommend logging be used for development and testing, but disabled or at least greatly minimized when iRules are put into production. Of course remote logging and HSL logging will have far less impact on the performance of the iRule and the BIG-IP as a whole since the expensive part of the equation, namely actually storing the log entry to disk via syslog, is completely offloaded. If you need real-time logging in a high paced production environment, I strongly recommend using remote logging or HSL where possible. When should I log, and when shouldn’t I? Given the above discussion about performance, we generally recommend logging at development, debugging and test times, but not in production. Can you log in prod? Yes, of course. Do people do it without major issues? Absolutely. But the possible performance implications can be enough of a drawback that it’s not worth it unless truly necessary for your deployment. Why risk the performance of your application when you can easily send the log information to a system that isn’t making thousands of critical business decisions per second? What are comments and how do they work? Commenting is an extremely simple concept. It is the ability to add text to an iRule (or any script, really) without affecting the actual functionality of the script itself. For instance, say I want to leave a note for myself or, even more importantly, another engineer that is likely to encounter my cod, about what a particular section of code does, why I chose a variable name, or anything else. That’s precisely what comments are for. It’s important to keep in mind that someone else will almost inevitably have to deal with your code in some fashion or another eventually. Commenting can be the difference between readable, usable, easy to maintain code and a jumble of strings that make no sense to anyone but the original developer. To begin a comment you just start any line with a hash symbol (#), and anything on that line will be considered part of the comment: #This will log the IP address of the incoming connection when CLIENT_ACCEPTED { log local0. “IP: [IP::client_addr]” } What does commenting my code cost me? Aside from whatever time it takes to enter the comments? Nothing. Comments are a zero cost operation within iRules because they are never actually seen by TMM. When an iRule is loaded from configuration into running memory it undergoes a process we’ve talked about in which it is pre-compiled. During this process the iRule is interpreted by TCL and formatted into a much simpler, lower level set of commands and strings and variables. At this point the comments are completely removed, so there are no comments in line when it comes time to actually execute the code against the traffic passing through a virtual. As such, they are a truly no cost option for your code. When should I comment my iRule? Always. Often. As much as you see fit. No, seriously, commenting has no cost and can be extremely beneficial to yourself, engineers (including F5 engineers attempting to help in many cases), and any future engineer or developer looking at your code. It’s always a good practice to document your code, whether in an external format or directly in line, and comments are an easy, simple way to do just that and be sure that the explanations for how and why you did things a certain way always follow the code wherever it ends up. Comments can also be very useful in development and testing to compare two commands or sections of code, or to temporarily remove a given piece of functionality or logic. You can go overboard of course, but that is really a readability issue more than anything. Use your judgment and don’t write a book inside your code, but generally speaking I have very rarely found code over-commented for my liking. This is doubly true when I’m the one inheriting someone else’s iRules and trying to make sense of them. Troubleshooting iRules – Where do I start? This is a question that gets asked often regarding iRules and the best answer is…right here. There is no formal debugging platform for iRules so logging and solid comments are you best friends. Keeping things clear and easy to understand by properly commenting your code, and using logging to determine what is happening while troubleshooting are invaluable. Generally it is a good practice to determine the depth of code execution when troubleshooting a behavioral problem, by placing log messages inside different logical constructs to see which cases are being matched and executed. From there it’s a matter of checking the output of commands and string formats to narrow down where the misbehavior is occurring. I would also highly recommend the use of tclsh, which as of 11.1 is available on the BIG-IP itself, in both development and troubleshooting of iRules. Tclsh is a tcl shell that allows you to natively execute Tcl commands and immediately observe the outcome. This can be extremely beneficial and a large time savings as opposed to having to re-generate traffic each time you want to make a small iRules change and test the functionality of a given command string to be sure of the output. Of course, tclsh can’t simulate any of the customized F5 additions to Tcl, but the basics are covered.Getting Started with iRules: Basic Concepts
Welcome to the third installment of this series, wherein we will, for the first time, actually discuss iRules at length. That may sound odd, but there has been some important foundational work to do before diving too deep into the technology behind iRules themselves. If you’re new to F5, new to programming, new to both, or just looking for a refresher on some very rudimentary concepts and terminology, I recommend checking out the first two articles in this series so that we’re all on the same page as we wade into iRules proper. Don't worry, we'll wait! Back for more? Great! Now that everyone is equally equipped, let’s dig in a bit to the meat of the topic at hand: iRules. Following a similar style as the first two introduction articles in this series, we’ll outline a few topics to cover, and then delve into them. To give a general introduction to iRules as a technology we’ll try to answer the following questions: What is an iRule? How does an iRule work? When would I use an iRule? When would I not use an iRule? What is an iRule? An iRule, in its most simple terminology, is a script that executes against network traffic passing through an F5 device. That’s pretty vague, though, so let’s try and define a bit more about what actually occurs within an iRule. The idea is pretty straightforward; iRules gives you the capability to write simple, network aware pieces of code that will influence your network traffic in a variety of ways. Whether you’re looking to do some form of custom persistence or rate limiting that isn’t currently available within the product’s built-in options, or looking to completely customize the user experience by granularly controlling the flow or even the contents of a given session/packet(s), that’s what iRules was built for. iRules can route, re-route, redirect, inspect, modify, delay, discard or reject, log or … do just about anything else with network traffic passing through a BIG-IP. The idea behind iRules is to make the BIG-IP nearly infinitely flexible. We recognized early on the need for users to be able to configure their systems to interact with network traffic in many ways that either we haven’t thought of, or are simply corner cases and/or in the minority of traffic being dealt with by our users. As such, rather than forcing them to submit requests for us to modify our core architecture every time they wanted to be able to use their F5 devices in a manner that slightly diverged from the collection of check boxes and drop downs available in the standard UI, we offered them iRules, and thereby a way to do what they need, when they need it. At the end of the day iRules is a network aware, customized language with which a user can add business and application logic to their deployment at the network layer. You can see a basic example iRule below, this is what iRules look like, and we will explore the different parts of an iRule in far more depth in coming parts of this series. If you're not fully comfortable with the code yet, don't let that scare you, we'll dig into each part of what you'll need to build iRules as the series continues. For now the idea is to start making iRules look and feel more familiar. # Rename a cookie by inserting a new cookie name with the same value as the original. Then remove the old cookie. when HTTP_REQUEST { # Check if old cookie exists in request if { [HTTP::cookie exists "old-cookie-name"] } { # Insert a new cookie with the new name and old cookie's value HTTP::cookie insert name "new-cookie-name" value [HTTP::cookie value "old-cookie-name"] # Remove the old cookie HTTP::cookie remove "old-cookie-name" } } How does an iRule work? To start at the beginning, as it were, an iRule is first and foremost a configuration object, in F5 terms. This means that it is a part of your general bigip.conf along with your pools, virtual servers, monitors, etc. It is entered into the system either via the GUI or CLI, generally speaking. There is also an iRules Editor available for download on DevCentral that is a windows tool for editing and deploying/testing iRules which can be extremely useful. Unlike most configuration objects, though, an iRule is completely user generated and customizable. An iRule is a script, at its core after all. Regardless of how an iRule gets there, be it UI, CLI or Editor, once an iRule is part of your config, it is then compiled as soon as that configuration is saved. One of the gross misconceptions about iRules is that, as with most interpreted scripting languages such as TCL, and interpreter must be instantiated every time an iRule is executed to parse the code and process it. This is not true at all, because every time you save your configuration all of your iRules are pre-compiled into what is referred to as “byte code”. Byte code is mostly compiled and has the vast majority of the interpreter tasks already performed, so that TMM can directly interpret the remaining object. This makes for far higher performance and as such, increase scalability. Now that the iRule is saved and pre-compiled, it must then be applied to a virtual server before it can affect any traffic. An iRule that is not applied to a virtual is effectively disabled, for all intents and purposes. Once you’ve applied an iRule to a given virtual server, however, it will now technically be applied against all traffic passing through that virtual. Keep in mind though, that this does not necessarily mean that all traffic passing through the virtual in question will be affected. IRules are most often very selective in which traffic they affect, be it to modify, re-route or otherwise. This is done through both logical constructs within the iRules, but also through the use of events within the iRule itself. Events are one of the ways in which iRules have been made to be network aware, as a language. An event, which we’ll dig into in much more detail in the next installment of this series, is a way of executing iRules code at a given point in time within the flow of a networking session. If I only want to execute a section of code once for each new connection to the virtual server to which my iRule is applied, I could easily do so by writing some simple code in the appropriate event. Events are also important because they indicate at which point in the proxy chain (sometimes referred to as a hud chain) an iRule executes. Given that BIG-IP is a bi-directional proxy, it is important for iRules to execute on not only the right side of the proxy, but at the right moment in the network flow. So now you have an iRule added to your configuration, it has been automatically pre-compiled to byte code when the configuration was saved, you have it applied to the appropriate virtual server, and the code within the iRule calls out the desired event in which you want your code to execute; now is when the magic happens, as it were. This is where the massive collection of iRules commands comes into play. From header modification to full on payload replacement to creating a socket connection to an outside system and making a request before processing traffic for your virtual, there are very few limitations to what can be achieved when combining the appropriate series of iRules commands. Those commands are then processed by TMM, which will affect whatever change(s) it needs to the traffic it is processing for the given session, depending on what you’ve designed your iRule to do. The true power of iRules largely comes into play thanks to the massive array of custom commands that we’ve built into the language, allowing you to leverage your BIG-IP to the fullest. When would I use an iRule? The ideal time to use an iRule is when you’re looking to add some form of functionality to your application or app deployment, at the network layer, and that functionality is not already readily available via the built in configuration options in your BIG-IP. Whether it’s looking to perform some kind of custom redirect or logging specific information about users’ sessions or a vast array of other possibilities, iRules can add valuable business logic or even application functionality to your deployment. iRules have a single point of management, your BIG-IP, as opposed to being distributed to every server hosting whichever application you’re trying to modify or affect. This can save valuable management time, and can also be a large benefit in time to deployment. It is often far easier to deploy an iRule or an iRule change than it is to modify your application for a quick fix. As an example, one of the most common uses of iRules when it was first introduced was to redirect all traffic from HTTP (port 80) to HTTPS (port 443) without affecting either the host or the requested URI for the connection. This was (and still is, pictured below) a very simple iRule, but it wasn’t at the time a feature available in the standard configuration options for BIG-IP. when HTTP_REQUEST { HTTP::redirect "https://[HTTP::host][HTTP::uri]" } When would I not use an iRule? The above example of an HTTP to HTTPS redirect iRule actually depicts perfectly when to not use an iRule, because that functionality was so popular that it has since been added as a profile option directly in the BIG-IP configuration. As such, it is more appropriate, and technically higher performance, to use that feature in the profile as opposed to writing an iRule to perform the same task. A general rule of thumb is: Any time you can do something from within the standard config options, profiles, GUI or CLI – do it there first. If you’re looking to perform a task that can’t be accomplished via the “built-in” means of configuration, then it is a perfect time to turn to iRules to expand the possibilities. For examples of using local traffic policies in lieu of iRules, check out Chase’s To iRule or Not to iRule article. This is for a few reasons, not the least of which is performance. iRules are extremely high performance, as a rule, and if written properly, but there is always a slight benefit in performance when you can run functionality directly from built in, core features as opposed to a custom created script, even an iRule. Also, though, it is easier to maintain a feature built into the product through upgrades, rather than re-testing and managing an iRule that could be easily replaced with a few configuration options. This concludes the introductions of core concepts in the series. Hopefully this gives everyone a solid place to start to allow us to begin digging into an array of iRules topics with more confidence and a unified starting place. Starting with the next installment of the series we will begin to delve deeper into individual iRules concepts and functionality, beginning with events and priorities.
