Forum Discussion
Universal Persistence with X-forwarder
Hi Experts
Can I use Universal persistence using x-forwarder with i-rule? I would have each x-forwarded IP stick to the same back-end pool member. Will this work? Can you please share code? Any other options to be enabled on the virtual server, I already have type set as standard vs.
Regards,
Sumanta.
A formatted version of the "Per VS" rate limiting. You can apply the same irule to all standard VS using UIE persistence.
when RULE_INIT { set static::maxReqs 3; set static::timeout 60; } when HTTP_REQUEST { set vs [URI::basename [virtual]] if { [HTTP::header exists "X-Forwarded-For"] } { set client_IP_addr [getfield [lindex [HTTP::header values "X-Forwarded-For"] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with $vs_URI_LIST_TO_LIMIT] ) } { whitelist if { [class match [IP::client_addr] equals $vs_ips_whitelist] }{ return } set getcount [table lookup -notouch "$vs_$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$vs_$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$vs_$client_IP_addr:[HTTP::uri]" } else { reject } } } persist uie $clientip } when HTTP_RESPONSE { persist add uie $clientip }
- Yann_Desmarest_Nacreous
Hi,
You can try this example :
when HTTP_RESPONSE { persist add uie $clientip } when HTTP_REQUEST { set clientip "" if { [HTTP::header exists "X-Forwarded-For"] } { set clientip [HTTP::header "X-Forwarded-For"] } else { set clientip [IP::client_addr] } persist uie $clientip }
- Hi Yann It appears this doesn't work. Please see error. 01070650:3: FastHTTP profile on virtual server /Common/AFG_BSF_VS can only be used with cookie persistence using insert mode.
- Yann_Desmarest_NacreousHi, this is because you are using fasthttp VS instead of standard VS with http profile
- Hi Yann Thanks. It started working after I applied http profile to a standard virtual server. Is One Connect mandatory in this case?
Hi,
You can try this example :
when HTTP_RESPONSE { persist add uie $clientip } when HTTP_REQUEST { set clientip "" if { [HTTP::header exists "X-Forwarded-For"] } { set clientip [HTTP::header "X-Forwarded-For"] } else { set clientip [IP::client_addr] } persist uie $clientip }
- Hi Yann It appears this doesn't work. Please see error. 01070650:3: FastHTTP profile on virtual server /Common/AFG_BSF_VS can only be used with cookie persistence using insert mode.
- Hi, this is because you are using fasthttp VS instead of standard VS with http profile
- Hi Yann Thanks. It started working after I applied http profile to a standard virtual server. Is One Connect mandatory in this case?
Hi,
You should change $xff by $clientip. $xff doesn't exists anywhere.
when HTTP_RESPONSE { persist add uie $clientip } when HTTP_REQUEST { set clientip "" if { [HTTP::header exists "X-Forwarded-For"] } { set clientip [HTTP::header "X-Forwarded-For"] } else { set clientip [IP::client_addr] } persist uie $clientip log local0. "[IP::client_addr]:[TCP::client_port]: XFF: $clientip" }
Hi Yann
Thanks. Got it. Working now. What is the difference between persist add uie $clientip under HTTP_RESPONSE and persist add uie $clientip under HTTP_REQUEST?
The x-forwarded is supposed to come in http-request packet.
Regards,
Sumanta.
- in the request it's just "persist uie" not "persist add uie". in the response you add a persistence record. in the request, you are doing a lookup of the persistence
- Hi Yann Thanks. I can see under iRule statistics, that HTTP_REQUEST=0 and HTTP_RESPONSE=62. However, there are no failures or aborts. I was hoping that request counter will be more since I am extracting real IP address from x-forwarded value and using it to create persistence. In case it is not present in the http packet, then normal source IP would be used. Is it okay or do I need to swap the "persist uie" and "persist add uie"
- Hi, it can be a bug on the statistics. I already noticed that. Do you have logs in the ltm log file regarding the log command you set in the http request event ?
Hi Yann
One more query is how to rate limit the connections per XFF IP address? Do we need to modify the rule or write something new?
Regards,
Sumanta.
Hi Sumanta,
Please find below a working irule for your need that I already answered here :
iRule - URI/Referer Rate limit per minute
The below iRule allow you to do rate limiting by Client IP per URI. You can change the if condition to match whatever you want :
when RULE_INIT { set static::maxReqs 3; set static::timeout 60; } when HTTP_REQUEST { if { [HTTP::header exists "X-Forwarded-For"] } { set client_IP_addr [getfield [lindex [HTTP::header values X-Forwarded-For] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { whitelist if { [class match [IP::client_addr] equals ips_whitelist] }{ return } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } } persist uie $clientip } when HTTP_RESPONSE { persist add uie $clientip }
Hi Yann
Thanks a lot. I actually have two virtual servers, one standard http type with persistence and the other Fast L4 type without persistence. Will the above iRule you wrote, go into the persistence profile of the first vs? And how do I write the second rule for the other vs? Just apply directly to vs and remove the "persist uie $clientip" and "persist add uie $clientip" ?
Regards,
Sumanta.
Actually I was wondering whether or not modifying the existing persistence iRule is needed or not? Maybe it is easier to just add a new single iRule and apply it to both virtual servers as needed.
- Yann_Desmarest_Nacreous
A formatted version of the "Per VS" rate limiting. You can apply the same irule to all standard VS using UIE persistence.
when RULE_INIT { set static::maxReqs 3; set static::timeout 60; } when HTTP_REQUEST { set vs [URI::basename [virtual]] if { [HTTP::header exists "X-Forwarded-For"] } { set client_IP_addr [getfield [lindex [HTTP::header values "X-Forwarded-For"] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with $vs_URI_LIST_TO_LIMIT] ) } { whitelist if { [class match [IP::client_addr] equals $vs_ips_whitelist] }{ return } set getcount [table lookup -notouch "$vs_$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$vs_$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$vs_$client_IP_addr:[HTTP::uri]" } else { reject } } } persist uie $clientip } when HTTP_RESPONSE { persist add uie $clientip }
Hi Yann
Thanks a lot. Should I use this i-rule and add it to standard VS settings or just modify the existing i-rule under persistence profile?
I think I am confused with this one. Since the purpose of i-rule under Persistence profile is different from normal i-rule under vs.
- Yann_Desmarest_Nacreous
Hi,
You should assign this irule to the VS directly. There is no need to assign an Universal persistence profile in this scenario.
Here an extract from the doc : Note: The following persistence methods require a corresponding persistence profile be added to the virtual server: ssl, msrdp, cookie
A formatted version of the "Per VS" rate limiting. You can apply the same irule to all standard VS using UIE persistence.
when RULE_INIT { set static::maxReqs 3; set static::timeout 60; } when HTTP_REQUEST { set vs [URI::basename [virtual]] if { [HTTP::header exists "X-Forwarded-For"] } { set client_IP_addr [getfield [lindex [HTTP::header values "X-Forwarded-For"] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with $vs_URI_LIST_TO_LIMIT] ) } { whitelist if { [class match [IP::client_addr] equals $vs_ips_whitelist] }{ return } set getcount [table lookup -notouch "$vs_$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$vs_$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$vs_$client_IP_addr:[HTTP::uri]" } else { reject } } } persist uie $clientip } when HTTP_RESPONSE { persist add uie $clientip }
Hi Yann
Thanks a lot. Should I use this i-rule and add it to standard VS settings or just modify the existing i-rule under persistence profile?
I think I am confused with this one. Since the purpose of i-rule under Persistence profile is different from normal i-rule under vs.
Hi,
You should assign this irule to the VS directly. There is no need to assign an Universal persistence profile in this scenario.
Here an extract from the doc : Note: The following persistence methods require a corresponding persistence profile be added to the virtual server: ssl, msrdp, cookie
- Yann_Desmarest_Nacreous
when RULE_INIT { set static::maxReqs 20000; set static::timeout 1800; } when HTTP_REQUEST { if { [HTTP::header exists X-Forwarded-For] } { set client_IP_addr [getfield [lindex [HTTP::header values X-Forwarded-For] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { if { [class match [IP::client_addr] equals ips_whitelist] }{ return } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } }
Thanks Yann, Let me try this out in our network. Will update you. :)
when RULE_INIT { set static::maxReqs 20000; set static::timeout 1800; } when HTTP_REQUEST { if { [HTTP::header exists X-Forwarded-For] } { set client_IP_addr [getfield [lindex [HTTP::header values X-Forwarded-For] 0] "," 1] } else { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { if { [class match [IP::client_addr] equals ips_whitelist] }{ return } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } }
Thanks Yann, Let me try this out in our network. Will update you. :)
Hi Yann
Getting a whole bunch of errors on this. Can you assist please? I can't debug, not from coding background.
01070151:3: Rule [/Common/iRule_rate_limit] error: /Common/iRule_rate_limit:8: error: [command is not valid in the current scope][if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } ] /Common/iRule_rate_limit:10: error: [undefined procedure: set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } ][{ set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $stat
Regards,
Sumanta.
Hi,
On my side, I just tested the following irule (same as previous but without comments) :
when RULE_INIT { set static::maxReqs 20000; set static::timeout 1800; } when HTTP_REQUEST { set client_IP_addr [IP::client_addr] if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } { set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } } }
And it's working. I'm using 12.1.0 but should be fine on older version. Can you repost the irule you try to use ?
Hi Yann
Thanks. See this below please. I am using 11.6.0 with HF 5. I removed the whitelist IP group and XFF sections since I am testing with Fast L4 profile on https port.
when RULE_INIT { set static::maxReqs 20000; set static::timeout 1260; } when HTTP_REQUEST { set client_IP_addr [IP::client_addr] } if { ([HTTP::method] eq "GET") and ([class match [string tolower [HTTP::uri]] ends_with URI_LIST_TO_LIMIT] ) } set getcount [table lookup -notouch "$client_IP_addr:[HTTP::uri]"] if { $getcount equals "" } { table set "$client_IP_addr:[HTTP::uri]" "1" $static::timeout $static::timeout } else { if { $getcount < $static::maxReqs } { table incr -notouch "$client_IP_addr:[HTTP::uri]" } else { reject } } }
Hi Yann
The last one you posted is working, probably I got some wrong braces. :(
Let me test this with traffic and see if I can perform any DoS attack on the virtual server.
Regards,
Sumanta.
Recent Discussions
Related Content
* Getting Started on DevCentral
* Community Guidelines
* Community Terms of Use / EULA
* Community Ranking Explained
* Community Resources
* Contact the DevCentral Team
* Update MFA on account.f5.com