Post of the Week: HTTP Redirect using Datagroups

[ltwagnon]

In this "Post of the Week" video, we discuss HTTP redirects and how to use datagroups and iRules to do these redirects.  If you have lots of URLs to redirect, then you might want to consider using something like this.  Enjoy!  

Resources

Developing iRules for BIG-IP

[F5_Digger_13600]

Hi John, I think this is a great idea. Instead of building and managing many many of specific redirection iRule or Local Traffic Policy or HTTP class, we simply build one REDIRECT datatroup and corresponding iRule and use it.

One quick question I have is if datagroup size is getting bigger and bigger, say 1000, there is any performance degradation compared to 1000 iRules or Local Traffic Policies? Any criteria when to create another redirection datagroup?

Thank you.

[imeliran]

Hi john, thanks for sharing the post, i have a question regarding the redirect, i can implement the same idea but instead of 1 value of redirection i have like 3-4. i mean when my vip for exmaple is: maincdn.mydomain01.com and my redirection occur between cdn1.mydomain01.com , cdn2.mydomain01.com etc..

and the purpose of working with datagroup that for scalability, once i want to implement same redirection to second domain for example: vip maincdn.mydomain02.com and redirection occur between cdn1.mydomain02.com , cdn2.mydomain02.com etc..

thanks in advance

[ltwagnon]

@F5_Digger, great question! The way that the "class" command works in a datagroup is that it hashes the entire datagroup before it searches, so the number of entries in the datagroup doesn't affect the performance too terribly bad. That said, we've done some testing on performance using datagroups and here's some of the results:

Testing done using 10,000 CPS, 1 HTTP request per TCP connection.

Baseline: TCP + HTTP profile + Blank iRule (when RULE_INIT { } )

Baseline results: Total CPU % used = 23%; TMM CPU % used = 16%; TMM Mem (MB) = 254

Check using iRule that I used in the video against datagroup with 1,000 entries (and no match found, so the entire group is forced to be searched).

Results: Total CPU % used = 25%; TMM CPU % used = 18%; TMM Mem (MB) = 259

Check against datagroup with 2,000 entries (no match found).

Results: Total CPU % used = 25%; TMM CPU % used = 18%; TMM Mem (MB) = 261

Check against datagroup with 10,000 entries (no match found).

Results: Total CPU % used = 26%; TMM CPU % used = 18%; TMM Mem (MB) = 277

So, even with a datagroup with 10,000 entries, the performance hit is minimal compared to a straight baseline with a blank iRule. I hope this helps!

[ltwagnon]

@imeliran, thanks for the question. It's true that you can have more than one entry for the "value" portion of the datagroup entry, but you will need some kind of logic to handle the selection of the specific value entry once you match the datagroup key. For example, if you have a datagroup string that is "; and values for that string ", , ; then you would need something in your iRule to manipulate the values so that you can select the proper value to return. I hope this helps. If you have any other questions, let me know. Thanks!

[dragonflymr]

Hi John,

As usual great article. One thing that makes this solution a bit cumbersome is lack of search/filter in datagroup interface (or I am not aware of such functionality?).

If there are hundreds or thousands of entries in datagroup it's hard to find the right one (to modify/delete) or check if given entry is already there. Any tips how to solve this?

I wonder as well how fast update in datagroup is picked up by iRule. I suspect that datagroup is not somehow cached when TCP connection is created and when HTTP_REQUEST is fired (can be multiple in one TCP connection) actual up-to-date content of datagroup is searched - Am I right?

Piotr

[Stanislas_Piro2]

@piotr, you can use external datagroup!

First time, you have to upload file. Next times, you can edit it in the webui!

[dragonflymr]

Hi Stanislas,

Maybe I am missing something but how using external datagroup will help with checking/modifying entries after import? In GUI there is nothing for searching filtering datagroup entries - is that different for external datagroup?

Piotr

[Stanislas_Piro2]

@Piotr,

when editing external Datagroup, this is a field to edit the whole DG like irules. the DG format is:

"param1" := "value1",
"param2" := "value2",
"param3" := "value3",

So to search a value in thousand values, you can easily copy in text editor or use browser search tool.

[dragonflymr]

Thanks for example and explanation, that is kind of solution but it would be nice to have some tools in GUI to perform such task without external tools 🙂

Piotr

[Reddy1]

Instead a Redirect , If i need to modify the URI/PATH and send it to the pool ?

[dragonflymr]

Hi,

 

Check ProxyPass (https://devcentral.f5.com/s/articles/proxypass-v10-v11), you will find good example of such feature.

 

Piotr

 

[Reddy1]

Sure

[Reddy1]

Can this works?

if [matchclass [HTTP::uri] equal REDIRECT] { set uri [matchclass -value [HTTP::uri] equals REDIRECT] HTTP::uri $uri pool MY_Pool }

Note:- example the data-group has entry , string:/x/y value:/z

[Stanislas_Piro2]

@REddy : some curly brackets are missing

you can use this code

if {[set [matchclass -value [HTTP::uri] equals REWRITE]] ne ""} {
    HTTP::uri $uri
    pool MY_Pool
}

[dragonflymr]

Hi,

@Stanislass: Is it not advised to use class match instead of matchclass?

This code is good, but I still think that it is worth checking ProxyPass and use it as base to create iRule resolving not only URI modification but as well pool selection via Data Group.

Piotr

[Stanislas_Piro2]

Your right, I did not see this!

URI rewriting is never a good idea, because one consequence is response rewriting may be required. And response rewriting is the worst configuration...

[dragonflymr]

Well, I know your approach :-), still as fast remedy I had to work figure out how to create what I need based on ProxyPass - seem to be working quite OK at least at header and cookie rewriting for responses.

Piotr

[jmolchin_47123]

Is it possible to reverse proxy a VS that is using a datagroup to redirect URL's? I don't want clients to proxy behind an F5 interface not access the URL directly. The VS is not using a default pool, the clients destination is in the datagroup.

[Reddy1]

For some reason the below irule not working, if {[class match $uri eq DATAGROUP]} { set new_uri [class match -value $old_uri eq DATAGROUP] HTTP::uri "$new_uri" pool $XYZ }

I need to verify the incomming uri checked against the Data-group before the value is assigned. Do i need to change it to set new_uri "[class match -value $old_uri eq DATAGROUP]" ?

[dragonflymr]

Hi,

Assuming that you using $uri in if and then use $old_uri inside this will not work, so I suggest something like that:

if ( [set new_uri [class match -value [HTTP::uri] eq DATAGROUP]] ne ""} { HTTP::uri "$new_uri" pool $XYZ }

It is assuming that your DG is set like that: old_uri := new_uri

• old_uri - key name you use to match with DG
• new_uri - new URI that should be set if there is match on old_uri

Piotr

[Reddy1]

Thanks Piotr, Actually it was a typo while i was writing the irule here.