Forcing a reload of External Data Groups within an iRule

This article shows how to use iControl to force an iRule to reload external configuration data when that data has changed.


One of the more common uses of iRules is to extract part of the request (client-ip, host, uri, payload, ...) and use that as input to a policy making decision.  Most often, the data you are comparing your input to (a list of blacklisted ip addresses, a list of valid hosts, uri to server mappings, etc) is too long to store directly within an iRule.  

Data Groups

That's where Data Groups come in.  Data Groups (or classes) are simply lists that can be references at the global scope from within an iRule to make the iRule itself more manageable.  You can use the matchclass and findclass helper commands to lookup values in Data Groups or you could use the TCL list commands to loop through the list manually.  There are four types of data groups: Address, String, Integer, and External.

Address Data Groups are used to store a list of IP Addresses (and optionally subnet masks).  These are often used when comparing an incoming client address against a list of valid or invalid addresses.

String Data Groups store a list newline delimited strings.  These are multi-purpose as you can use them for straight lookups (valid requested uri or hostname) or for building mapping (this uri gets routed to that pool).  Given a token delimiter, the findclass command can be used to look for the first token in the string and return the second token.

Integer Data Groups store a list of integers.  A common use for this type of data group is to compare an ID passed in as part of the URI to a valid value.  While this could be accomplished with a string data group, integer data groups require less resources and comparisons are faster.

External Data Groups are lists that are stored externally the data base (on the file system).  These can be of any of the other three types (Address, String, and Integer).  A common use for this type of Data Group is when you are storing thousands of values and those values change often.  It is much easier to manage an external file via a text editor, or a remote API like iControl.  

The Problem

The iRules engine utilizes a caching mechanism for external Data Groups.  The problem is that when a change is made to an External Data Group, The iRules engine has no way of knowning that the changes have actually been made.  Even a configuration reload will not do it as a reload is just an overwrite on the current configuration and since the iRule didnt' change the internal caching doesn't get invalidated and the next time the iRule is processed it still reads the cached values in the Data Group.

The Solution

Fear not, there is a fairly simple solution.  If you invalidate the iRule itself by changing it, the cache will be invalidated and the External Data Group will be reloaded.  How do you do this?  Well, we've found that the easiest way is to add a space to the end of the iRule.  This will not change the behaviour of the iRule but will cause it's checksum to change.  Just add a space to the end, save the iRule, then delete the space, and then save the iRule again!

This can be accomplished via the GUI, CLI, or iControl.  The GUI and CLI are fiarly intuitive, so I'll show you how to use iControl to do this.  More than likely for large External Data Groups, you are using iControl already to remotely add/remove items from the lists.  If that is the case, it is a very simple addition to your application to add this logic.  If you are not using iControl to manage your External Data Group, then you can plop this perl script right on your BIG-IP and use it from the command line.  Either way, we've got you covered.

Here's an iControl application written in Perl that will invalidate an iRule to force reloading of Data Groups.

# The contents of this file are subject to the "END USER LICENSE AGREEMENT FOR F5 
# Software Development Kit for iControl"; you may not use this file except in
# compliance with the License. The License is included in the iControl
# Software Development Kit. 
# Software distributed under the License is distributed on an "AS IS"
# basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See
# the License for the specific language governing rights and limitations
# under the License.
# The Original Code is iControl Code and related documentation
# distributed by F5.
# The Initial Developer of the Original Code is F5 Networks,
# Inc. Seattle, WA, USA. Portions created by F5 are Copyright (C) 1996-2006 F5 Networks,
# Inc. All Rights Reserved.  iControl (TM) is a registered trademark of F5 Networks, Inc.
# Alternatively, the contents of this file may be used under the terms
# of the GNU General Public License (the "GPL"), in which case the
# provisions of GPL are applicable instead of those above.  If you wish
# to allow use of your version of this file only under the terms of the
# GPL and not to allow others to use your version of this file under the
# License, indicate your decision by deleting the provisions above and
# replace them with the notice and other provisions required by the GPL.
# If you do not delete the provisions above, a recipient may use your
# version of this file under either the License or the GPL.

#use SOAP::Lite + trace => qw(method debug);
use SOAP::Lite;

# Variables
my $DEBUG = 0;
my $sHost = "localhost";
my $sPort = 80;
my $sUID = "";
my $sPWD = "";
my $sProtocol = "http";
my $sRuleName = "MSM_reputation";

# Transport Information
sub SOAP::Transport::HTTP::Client::get_basic_credentials
  return "$sUID" => "$sPWD";

# iControl interface declarations
$LocalLBRule = SOAP::Lite
  -> uri('urn:iControl:LocalLB/Rule')
  -> readable(1)
  -> proxy("$sProtocol://$sHost:$sPort/iControl/iControlPortal.cgi");

# Main logic

# sub invalidateRule
# This iRule will query the contents of an iRule, save it with an added space
# and then re-save it with the original contents.
sub invalidateRule()
  if ( $DEBUG ) { print "Querying rule definition for rule '$sRuleName'\n"; }

  # Query the rule definition
  $soapResponse = $LocalLBRule->query_rule(SOAP::Data->name(rule_names => [$sRuleName]));
  @RuleDefinitionList = @{$soapResponse->result};

  # Extract the first element in the array
  $RuleDefinition = @RuleDefinitionList[0];
  # save the original and temporary changed values
  $rule_definition = $RuleDefinition->{"rule_definition"};
  $temp_rule_definition = "$rule_definition ";

  # update RuleDefinition with the temporary rule details
  if ( $DEBUG ) { print "modifying with temporary changes...\n"; }
  $RuleDefinition->{"rule_definition"} = $temp_rule_definition;
  $soapResponse = $LocalLBRule->modify_rule(SOAP::Data->name(rules => [$RuleDefinition]));

  # update RuleDefinition with the original rule details.
  if ( $DEBUG ) { print "modifying with reverted changes...\n"; }
  $RuleDefinition->{"rule_definition"} = $rule_definition;
  $soapResponse = $LocalLBRule->modify_rule(SOAP::Data->name(rules => [$RuleDefinition]));

  if ( $DEBUG ) { print "rule initialized...\n"; }

# sub checkResponse
# Exit on SOAP Fault.
sub checkResponse()
  my ($soapResponse) = (@_);
  if ( $soapResponse->fault )
    print $soapResponse->faultcode, " ", $soapResponse->faultstring, "\n";


Published Feb 07, 2007
Version 1.0

Was this article helpful?


  • Hi,



    it doesn't seem to work anymore with 9.3 and 9.4.



    Looks like updating an iRule doesn't affect anymore the external class.



    You need to update the class now to reload it...



    I guess you can play with set_external_class_file_name for example to update the class and force it to be reloaded.



    It works in CLI so should work with iControl !
  • Thanks for the feedback. I'll test this out with 9.4 and see what's up.



  • tkarni_92568's avatar
    Historic F5 Account



    I like the idea of the ability of an iControl updating an external class for an iRule to refer to it in real time.


    A good usage I see is updating a black list of IPs by some application (using iControl) and have an iRule reject connections from these IPs.



    I do find it a bit funny that such a wonderful mechanism needs such a clumsy way to reload the external list. This is a basic need and I would think of a simple single line of code (style "class->reload")



    Joe, I read you were looking into this code forcing a reload on 9.4.


    Now that 9.4.4 is out, would you know if there is an easy way to force a reload in 9.4.4?




  • Does the same occur for changes to interally defined classes? I changed an entry in the class and then did a 'b load'. If I examine the class, the entry is changed, but all code references to it still return the old value. BIG-IP 9.4.5 Build 1086.1 Hotfix HF2
  • Internal classes behave differently with regards to iControl. If you use the API to make changes to a internal class, then change is resident in memory (not in the disk configuration) until you issue a ConfigSync.save_configuuration() call. This will flush the config changes to disk. After that you can call load_configuration() to reload it but it is not necessary. We chose not to automatically flush all API calls to the disk's configuration because the time it took to do so for large configurations caused problems for apps making large number of calls. We chose to make it the users choice when the config was saved.



  • HI all,


    I just resolve this problem for my tests, maybe some variations will be needed for each particular irule, but this is the general idea:



    1.- Put the statement that assigns the variable value from the class, in a place were the value needs to be utilized.


    Put the statement inside other event that will use that variable other than "when RULE_INIT"



    2.- Save the Irule



    3.-Run your test. Run the scenario that will force to re-read the variable (in the new place). This action will force to assign the new value to the variable when needed in the particular event, example in "when HTTP_REQUEST" or "when HTTP_REQUEST_DATA "



    4.- The new value will be the F5 memory


    5.- Remove the statement from that place and leave your irule as before.


    6.- Save the irule





  • You could just make the data group read/write, update, make it read only, update.


    This works for me via the GUI on 10.x. Haven't tried to do it via iControl.
  • Hi,



    You suggest that add a space to the end of iRule, save the iRule, then delete the space, and then save the iRule again. But which iRule?


    I suppose that we changes all irules which contains chnged datagroup. But how we find these iRules. I can not find an interface of icontrol about this subject.