Forum Discussion
cmoates
Nimbostratus
NimbostratusKeeping a hash's size in check
Hey there,
It is entirely possible that I'm approaching this problem the wrong way, and if so, I'll take any guidance on another solution to the problem. So with that said, here's my problem:
We have a farm of inbound SMTP servers. I want to restrict the number of simultaneous connections from a single IP on the internet to 20, no matter which server in the pool they are connected to. If they try to exceed this, we'll just flat out refuse connections from them for N seconds, until they stop trying. This has worked pretty well at keeping certain spamhauses from obliterating us with simultaneous SMTP requests.
So my solution is the iRule pasted below. The problem is that the ::users hash eventually exceeds the 4MB limit and causes TMM to core. F5 support has provided me with that detail after analyzing the core dumps, telling me that right before the crash, the ::users hash had over 785,000 entries.
I'm thinking that I need some way to expire entries after awhile. Right now, we crash about every 24 hours, so if I could expire 12 hour old entries, that would probably be sufficient. But I'm thinking there's got to be some better way that I just haven't thought of.
Ok, now that my rambling is out of the way, here's my iRule:
when RULE_INIT {
set ::maxconnect 20
set ::blocktime 120
array set ::users { }
array set ::spammers { }
}
when CLIENT_ACCEPTED {
if { [matchclass [IP::remote_addr] equals $::smtp_whitelist ] } {
Accept whitelisted hosts
return
}
set clientip [IP::remote_addr]
set now [clock second]
if { [ info exists ::spammers($clientip) ] } {
if { $::blocktime > [expr { $now - $::spammers($clientip) }] } {
If blocked client tries to send email before blocktime ends,
reset time, send TCP respond and drop connection
set ::spammers($clientip) $now
log local0. "Spam Client Dropped, IP: [IP::remote_addr]"
TCP::respond "450 Message Rejected - Too many connections from your IP, try again later\r\n"
drop
return
} else {
If blocktime is over, free host from spammerlist
unset ::spammers($clientip)
}
}
if { [ info exists ::users(nb,$clientip)] } {
if { [expr { $now - $::users(time,$clientip) }] > $::blocktime } {
If last connection is over blocktime ago, reset status
set ::users(nb,$clientip) 1
set ::users(time,$clientip) $now
return
} else {
incr ::users(nb,$clientip)
set ::users(time,$clientip) $now
if { $::users(nb,$clientip) > $::maxconnect } {
If maxconnections is exceeded, put client to spammerlist,
send TCP respond and drop connection
set ::spammers($clientip) $now
set ::users(nb,$clientip) 1
set ::users(time,$clientip) $now
log local0. "Spam Client Dropped, IP: [IP::remote_addr]"
TCP::respond "450 Message Rejected - Too many connections from your IP, try again later\r\n"
drop
return
}
}
} else {
New client
set ::users(nb,$clientip) 1
set ::users(time,$clientip) $now
}
}- The_BhattmanHave you considered creating an when CLIENT_CLOSED event to keep the ::user HASH in check.
For Example:when CLIENT_CLOSED { if { [info exists ::users($clientip)] } { incr ::users($clientip) -1 if { $::users($clientip) <= 0 } { unset ::users($clientip) } } 
cmoatesThanks, I think that's basically what I needed. I'm new to iRules, and was missing the CLIENT_CLOSED hook. I'll adjust the code appropriately and test it out.- hoolio
Cirrostratus
Some other thoughts...
It looks like your current methodology is for non-white listed clients, to check if they're already identified as a spammer. If not, add them to a user array with a timestamp. If they surpass the max TCP connection count within a given timeframe, add them to the spammers array. The only time a spammer is removed from the spammer array is if they make subsequent requests without exceeding the request/timeframe threshold. A user IP is never cleared.
You could use the session table (Click here) instead of arrays. There is a native timeout option when adding entries to the table. I'm not sure what the upper limits are on the number of records the session table can hold though. You could create a single entry per client IP address. I think this would be more efficient than maintaining separate lists of users and the "spammers".
You'd end up with an even smaller number of records if you only tracked current connections--rather than the connections over a period of time. The session table entry would contain the client IP address and a count of current connections. You could check if the count is over the maximum for each request before allowing the request. You could add logic to decrement the count in CLIENT_CLOSED, when the TCP connection is closed.
If you did want to track connections over a period of time and enforce a threshold, I think you'd need to stick with an array. There isn't a method for counting the number of session table entries that match a session key. There has been mention of this, but I haven't seen any concrete info.
An example command to add an entry would be:
session add uie [IP::client_addr] $count $timeout_in_seconds
You can get the count for the particular client using:
set count [session lookup uie [IP::client_addr]]
Aaron
