Hi Experts Can I use Universal persistence using x-forwarder with i-rule? I would have each x-forwarded IP stick to the same back-end pool member. Will this work? Can you please share code? Any other options to be enabled on the virtual server, I already have type set as standard vs.Regards, Sumanta.

Universal Persistence with X-forwarder

Yann_Desmarest_
Nacreous
Jun 11, 2016
Hi,

You can try this example :

when HTTP_RESPONSE { persist add uie $clientip } when HTTP_REQUEST { set clientip "" if { [HTTP::header exists "X-Forwarded-For"] } { set clientip [HTTP::header "X-Forwarded-For"] } else { set clientip [IP::client_addr] } persist uie $clientip }
- Sumanta_88744
  Cirrus
  Jun 21, 2016
  Hi Yann It appears this doesn't work. Please see error. 01070650:3: FastHTTP profile on virtual server /Common/AFG_BSF_VS can only be used with cookie persistence using insert mode.
- Yann_Desmarest_
  Nacreous
  Jun 21, 2016
  Hi, this is because you are using fasthttp VS instead of standard VS with http profile
- Sumanta_88744
  Cirrus
  Jun 21, 2016
  Hi Yann Thanks. It started working after I applied http profile to a standard virtual server. Is One Connect mandatory in this case?
Yann_Desmarest
Cirrus
Jun 11, 2016
Hi,

You can try this example :

when HTTP_RESPONSE { persist add uie $clientip } when HTTP_REQUEST { set clientip "" if { [HTTP::header exists "X-Forwarded-For"] } { set clientip [HTTP::header "X-Forwarded-For"] } else { set clientip [IP::client_addr] } persist uie $clientip }
- Sumanta_88744
  Cirrus
  Jun 21, 2016
  Hi Yann It appears this doesn't work. Please see error. 01070650:3: FastHTTP profile on virtual server /Common/AFG_BSF_VS can only be used with cookie persistence using insert mode.
- Yann_Desmarest
  Cirrus
  Jun 21, 2016
  Hi, this is because you are using fasthttp VS instead of standard VS with http profile
- Sumanta_88744
  Cirrus
  Jun 21, 2016
  Hi Yann Thanks. It started working after I applied http profile to a standard virtual server. Is One Connect mandatory in this case?

Yann_Desmarest

Cirrus

Jun 22, 2016

Hi,

You should change $xff by $clientip. $xff doesn't exists anywhere.

when HTTP_RESPONSE { 
    persist add uie $clientip 
} 

when HTTP_REQUEST { 
    set clientip "" 
    if { [HTTP::header exists "X-Forwarded-For"] } { 
        set clientip [HTTP::header "X-Forwarded-For"] 
    } else { 
        set clientip [IP::client_addr] 
    } 
    persist uie $clientip 
    log local0. "[IP::client_addr]:[TCP::client_port]: XFF: $clientip" 
}

Sumanta_88744
Cirrus
Jun 22, 2016
Hi Yann

Thanks. Got it. Working now. What is the difference between persist add uie $clientip under HTTP_RESPONSE and persist add uie $clientip under HTTP_REQUEST?

The x-forwarded is supposed to come in http-request packet.

Regards,

Sumanta.
- Yann_Desmarest
  Cirrus
  Jun 22, 2016
  in the request it's just "persist uie" not "persist add uie". in the response you add a persistence record. in the request, you are doing a lookup of the persistence
- Sumanta_88744
  Cirrus
  Jun 23, 2016
  Hi Yann Thanks. I can see under iRule statistics, that HTTP_REQUEST=0 and HTTP_RESPONSE=62. However, there are no failures or aborts. I was hoping that request counter will be more since I am extracting real IP address from x-forwarded value and using it to create persistence. In case it is not present in the http packet, then normal source IP would be used. Is it okay or do I need to swap the "persist uie" and "persist add uie"
- Yann_Desmarest
  Cirrus
  Jun 23, 2016
  Hi, it can be a bug on the statistics. I already noticed that. Do you have logs in the ltm log file regarding the log command you set in the http request event ?
Sumanta_88744
Cirrus
Jul 20, 2016
Hi Yann

One more query is how to rate limit the connections per XFF IP address? Do we need to modify the rule or write something new?

Regards,

Sumanta.
- Yann_Desmarest
  Cirrus
  Jul 20, 2016
  Hi Sumanta,
  
  Please find below a working irule for your need that I already answered here :
  
  iRule - URI/Referer Rate limit per minute
  
  The below iRule allow you to do rate limiting by Client IP per URI. You can change the if condition to match whatever you want :
  
  when RULE_INIT { set static::maxReqs 3; set static::timeout 60; } when HTTP_REQUEST { if { [HTTP::header exists "X-Forwarded-For"] } { set client_IP_addr [getfield [lindex [HTTP::header values X-Forwarded-For] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { whitelist if { [class match [IP::client_addr] equals ips_whitelist] }{ return } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } } persist uie $clientip } when HTTP_RESPONSE { persist add uie $clientip }
- Sumanta_88744
  Cirrus
  Jul 20, 2016
  Hi Yann
  
  Thanks a lot. I actually have two virtual servers, one standard http type with persistence and the other Fast L4 type without persistence. Will the above iRule you wrote, go into the persistence profile of the first vs? And how do I write the second rule for the other vs? Just apply directly to vs and remove the "persist uie $clientip" and "persist add uie $clientip" ?
  
  Regards,
  
  Sumanta.
- Sumanta_88744
  Cirrus
  Jul 20, 2016
  Actually I was wondering whether or not modifying the existing persistence iRule is needed or not? Maybe it is easier to just add a new single iRule and apply it to both virtual servers as needed.

Yann_Desmarest_

Nacreous

Jul 20, 2016

A formatted version of the "Per VS" rate limiting. You can apply the same irule to all standard VS using UIE persistence.

when RULE_INIT {
    set static::maxReqs 3;
    set static::timeout 60;
}

when HTTP_REQUEST {
        set vs [URI::basename [virtual]]
        if { [HTTP::header exists "X-Forwarded-For"] } {
            set client_IP_addr [getfield [lindex  [HTTP::header values "X-Forwarded-For"]  0] "," 1]
        } else {
            set client_IP_addr [IP::client_addr]
        }
        if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with $vs_URI_LIST_TO_LIMIT] ) } {

            whitelist
            if { [class match [IP::client_addr] equals $vs_ips_whitelist] }{
               return
            }
            set getcount [table lookup -notouch "$vs_$client_IP_addr:[HTTP::uri]"]
            if { $getcount equals "" } {
                table set "$vs_$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout
            } else {
                if { $getcount < $static::maxReqs } {
                    table incr -notouch "$vs_$client_IP_addr:[HTTP::uri]"
                } else {
                    reject
                }
            }
        }
        persist uie $clientip 
}

when HTTP_RESPONSE { 
    persist add uie $clientip 
}

Sumanta_88744
Cirrus
Jul 21, 2016
Hi Yann

Thanks a lot. Should I use this i-rule and add it to standard VS settings or just modify the existing i-rule under persistence profile?

I think I am confused with this one. Since the purpose of i-rule under Persistence profile is different from normal i-rule under vs.
Yann_Desmarest_
Nacreous
Jul 21, 2016
Hi,

You should assign this irule to the VS directly. There is no need to assign an Universal persistence profile in this scenario.

Here an extract from the doc : Note: The following persistence methods require a corresponding persistence profile be added to the virtual server: ssl, msrdp, cookie

Yann_Desmarest

Cirrus

Jul 20, 2016

A formatted version of the "Per VS" rate limiting. You can apply the same irule to all standard VS using UIE persistence.

when RULE_INIT {
    set static::maxReqs 3;
    set static::timeout 60;
}

when HTTP_REQUEST {
        set vs [URI::basename [virtual]]
        if { [HTTP::header exists "X-Forwarded-For"] } {
            set client_IP_addr [getfield [lindex  [HTTP::header values "X-Forwarded-For"]  0] "," 1]
        } else {
            set client_IP_addr [IP::client_addr]
        }
        if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with $vs_URI_LIST_TO_LIMIT] ) } {

            whitelist
            if { [class match [IP::client_addr] equals $vs_ips_whitelist] }{
               return
            }
            set getcount [table lookup -notouch "$vs_$client_IP_addr:[HTTP::uri]"]
            if { $getcount equals "" } {
                table set "$vs_$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout
            } else {
                if { $getcount < $static::maxReqs } {
                    table incr -notouch "$vs_$client_IP_addr:[HTTP::uri]"
                } else {
                    reject
                }
            }
        }
        persist uie $clientip 
}

when HTTP_RESPONSE { 
    persist add uie $clientip 
}

Sumanta_88744
Cirrus
Jul 21, 2016
Hi Yann

Thanks a lot. Should I use this i-rule and add it to standard VS settings or just modify the existing i-rule under persistence profile?

I think I am confused with this one. Since the purpose of i-rule under Persistence profile is different from normal i-rule under vs.
Yann_Desmarest
Cirrus
Jul 21, 2016
Hi,

You should assign this irule to the VS directly. There is no need to assign an Universal persistence profile in this scenario.

Here an extract from the doc : Note: The following persistence methods require a corresponding persistence profile be added to the virtual server: ssl, msrdp, cookie

Yann_Desmarest_

Nacreous

Jul 25, 2016

when RULE_INIT {
    set static::maxReqs 20000;
    set static::timeout 1800;
}
when HTTP_REQUEST {
       if { [HTTP::header exists X-Forwarded-For] } {
           set client_IP_addr [getfield [lindex  [HTTP::header values X-Forwarded-For]  0] "," 1]
        } else {
            set client_IP_addr [IP::client_addr]
        }
        if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } {
            if { [class match [IP::client_addr] equals ips_whitelist] }{
               return
            }
            set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"]
            if { $getcount equals "" } {
                table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout
            } else {
                if { $getcount < $static::maxReqs } {
                    table incr -notouch "$client_IP_addr:[HTTP::uri]"
                } else {
                    reject
                }
            }
}

Sumanta_88744
Cirrus
Jul 25, 2016
Thanks Yann, Let me try this out in our network. Will update you. :)

Yann_Desmarest

Cirrus

Jul 25, 2016

when RULE_INIT {
    set static::maxReqs 20000;
    set static::timeout 1800;
}
when HTTP_REQUEST {
       if { [HTTP::header exists X-Forwarded-For] } {
           set client_IP_addr [getfield [lindex  [HTTP::header values X-Forwarded-For]  0] "," 1]
        } else {
            set client_IP_addr [IP::client_addr]
        }
        if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } {
            if { [class match [IP::client_addr] equals ips_whitelist] }{
               return
            }
            set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"]
            if { $getcount equals "" } {
                table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout
            } else {
                if { $getcount < $static::maxReqs } {
                    table incr -notouch "$client_IP_addr:[HTTP::uri]"
                } else {
                    reject
                }
            }
}

Sumanta_88744
Cirrus
Jul 25, 2016
Thanks Yann, Let me try this out in our network. Will update you. :)

Sumanta_88744
Cirrus
Aug 16, 2016
Hi Yann

Getting a whole bunch of errors on this. Can you assist please? I can't debug, not from coding background.

01070151:3: Rule [/Common/iRule_rate_limit] error: /Common/iRule_rate_limit:8: error: [command is not valid in the current scope][if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } ] /Common/iRule_rate_limit:10: error: [undefined procedure: set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } ][{ set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $stat

Regards,

Sumanta.
- Yann_Desmarest
  Cirrus
  Aug 16, 2016
  Hi,
  
  On my side, I just tested the following irule (same as previous but without comments) :
  
  when RULE_INIT { set static::maxReqs 20000; set static::timeout 1800; } when HTTP_REQUEST { set client_IP_addr [IP::client_addr] if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } } }
  
  And it's working. I'm using 12.1.0 but should be fine on older version. Can you repost the irule you try to use ?
- Sumanta_88744
  Cirrus
  Aug 16, 2016
  Hi Yann
  
  Thanks. See this below please. I am using 11.6.0 with HF 5. I removed the whitelist IP group and XFF sections since I am testing with Fast L4 profile on https port.
  
  when RULE_INIT { set static::maxReqs 20000; set static::timeout 1260; } when HTTP_REQUEST { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } }
- Sumanta_88744
  Cirrus
  Aug 16, 2016
  Hi Yann
  
  The last one you posted is working, probably I got some wrong braces. :(
  
  Let me test this with traffic and see if I can perform any DoS attack on the virtual server.
  
  Regards,
  
  Sumanta.

Forum Discussion

Universal Persistence with X-forwarder

Recent Discussions

cpu consumption on HA standby devices is higher

f5 command document

Load Balancing for 3 app

Packet loss between Fortigate and B2250

Retain the URL on Browser

Related Content

XFF Universal Persistence iRule

Extracting X-Forwarded-For in Node.js

TCP Option 28 X-Forwarded-For Header

Restricting Access to Virtual from client IP address in X-Forwarder-For HTTP header

X-Forwarded-For on L4 VS

ABOUT DEVCENTRAL

RESOURCES

SUPPORT

PARTNERS