Forum Discussion

Deb_Allen_18's avatar
Deb_Allen_18
Historic F5 Account
Oct 05, 2006

Optimizing large lookup tables

I have several customers requesting various functionality such as selective rate limiting, access filtering, geo load balancing, etc, all of which would require very large reference tables (4000+ rows).

 

 

This post (http://devcentral.f5.com/Default.aspx?tabid=28&view=topic&forumid=5&postid=8241 Click here) seems to indicate that an external class is NOT the ideal way to approach this issue, instead perhaps the data could be imported into an array for more optimal access.

 

 

I'm hoping to get some clarification on unRuleY's comment in that post about initializing the array.

 

 

Is the recommended approach there to use another high-priority rule specifically written to access the external class and re-write it into an array? I'm assuming we'd need to leverage the RULE_INIT event, but I'm not sure how to parse through a class and re-write all the records to an array.

 

 

I'm a bit concerned that initializing the array in this manner might consume excessive system resources each time the config is reloaded.

 

 

Bottom line, I know I can create the rule logic to gain the filtering functionality required, but need to know for sure how to manage the large lookup tables and some idea as to real-world max lookup table size.

 

 

Any feedback much appreciated.

 

/deb
application delivery
dev
devops
iRules
  • unRuleY_95363's avatar
    unRuleY_95363
    Historic F5 Account
    Oct 19, 2006
    Nope, I was not referring to using the RULE_INIT event to initialize the array. What I was referring to do was to create a seperate VIP/iRule that would be then used to manage the array by some external device/program.

     

    This seperate VIP/iRule would serve the sole purpose of managing the array. So, you would want to come up with a protocol for doing that. Let's say we use HTTP, so the following "PUT " request would actually add the to the array. "GET " could be used to test if was in the array.

     

    You could come up with all sorts of commands to access/manage the array from the external program. The biggest challenge with this approach is having the external program know when it needs to initialize the array. There are a number of ways I can think of to do that, but I'll leave it up to you to discover one.
  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    Oct 23, 2006
    Thanks unRuleY -

     

     

    I think I understand, but do you think that approach could be leveraged to import a large static lookup table into an array? Or is that even a good idea? My concerns center around what happens on config re-load or reboot: Is that data then gone until it is added back again? (iRule is intended to deny access unless matching data is in the list, so I don't want to create a false "site down" condition by having the list go away unexpectedly.)

     

     

    My customer has a list of numeric ranges that will need to be referenced in an array. The data is static, and it is provided monthly to the customer as a table with ~3000 rows, 2 columns each, like this:27049783822704978382

     

    27049783942704978394

     

    27049783962704978396

     

    27049784652704978465

     

    27049785062704978506

     

    32624734863262473486

     

    32624735443262473544

     

    32624735493262473549

     

    32624735683262473568

     

    32624735833262473583

     

    32624735873262473587

     

    32624736643262473664

     

    32624736893262473689

     

    32624737363262473736

     

    32624750803262475080The tools available for class manipulation are not sufficient for this exercise, and when I try to manupulate a class as if it were an array, I get a TCL error 'can't read "::class_name(xxxx)": variable isn't array while executing "array get $::class_name(xxxx)"'

     

     

    thanks for any insight or alternative approaches

     

    /deb