Instructing BigIP to expire RAM Cache when it receives a URL Query

Hi David,

Can you try this without the redirect? Also, you can parse the URI without the query string using HTTP::path. To get just the query string you can use HTTP::query.

when HTTP_REQUEST {
 Disable cache for a defined set of folders
if { [class match [HTTP::uri] contains folders_to_not_cache] }{
CACHE::disable
return
}

if { [string tolower [HTTP::query]] starts_with "expire" } {
log "Request to expire [HTTP::path] received."
set expire_cache_trigger 1

 Remove the query string from the URI
HTTP::uri [HTTP::path]
}
}

when CACHE_REQUEST {
if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger==1 ) } {
CACHE::expire
log "Cache Expired"
unset expire_cache_trigger
}
}

Aaron

Trying it without the redirect I'm seeing the same behavior. It's working for some computers that send the ?expire, and not for others...

 

 

I wasn't using the URI::path as some of the pages in cache have other query parameters associated with them that still need to be passed to the servers in the pool, such as:

 

 

author.html?page=4?expire

Who/what is appending the "?expire" string? If there is already a query string in the URI, it would be appended with an ampersand like author.html?page=4&expire. If you want to remove expire from the query string you could use string map:

if { [string tolower [HTTP::query]] ends_with "expire" } {
log "Request to expire [HTTP::path] received."
set expire_cache_trigger 1

 Remove expire from the query string
HTTP::uri [string range [HTTP::uri] 0 end-7]
}

Aaron

That's a good point, changes incorporated.

 

 

Here's soemthing that confuses me more...

 

 

From the remote site if we open the url in a browser with ?expire, then http_request sees it and sets the variable, but the cache_request doesn't see that the variable exists.

 

 

But if we use curl from the same computer to open the same url, it works...

Ok, Disregard. Found the issue.

 

 

It was on the browser of the remote computer we were testing... something was preventing it from running, maybe an existing connection or something. If we went to private browsing mode, it worked.

 

 

So, wasn't a problem with the irule that I can see.

 

 

Thank you for the heads up on the string map.

UPDATE:

Below is the updated iRule and a few logs. Basically when a page is modified by an author and then saved, a single request is made to the uri with 'expire' added as a query: uri?expire

The goal is that when the F5 receives the uri?expire it will completely remove the uri from it's existing ram cache and grab the newly updated uri which it will then cache.

What is happening instead is that the F5 continues to serve the old version of the uri that it was supposed to have cleared.

We did a test where we:

1. Updated and saved a page with the content: TEST 1

2. Verified it saved correctly and was being served correctly by accessing the server locally and bypassing the F5

3. Manually requested the URI with uri?expire

4. Used a browser in private mode by deleting all cookies and cache, then requesting the uri to look if the content showed TEST 1

We repeated this process incrementally by repeating the steps above and incrementing the TEST number. What we found in say TEST 4 was that the F5 would sometimes show TEST 3, and even sometimes show TEST 2, but never showed TEST 4.

We could occasionally see the updated version of the uri by requesting uri?expire= or uri?expire?expire

Here are some logs:

13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired
13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired
13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired

And here is the iRule.

iRule-Cache-Processing

when HTTP_REQUEST {

 If the URL contains a folder that we're not supposed to cache, disable the cache
if { [class match [HTTP::uri] contains folders_to_not_cache] }{

CACHE::disable
return

} else {

 Ignore any cache-control headers sent by the user's browser.
if { [HTTP::header exists "Cache-Control"] } { HTTP::header remove "Cache-Control" }

 Turn on caching
CACHE::enable 
}


 Convert the query options to lower case, then look for "expire" in them
if { [string tolower [HTTP::query]] ends_with "expire" } {

 If we found the request, make sure it's coming from a valid admin IP
if { [class match [client_addr] equals admin_ips] } {

 If it's a valid source, then log the request and set the variable to flag the cache to be expired in the CACHE_REQUEST event.
log "Request to expire [getfield [string tolower [HTTP::uri]] "?expire" 1] received from [client_addr]."
log local0. "URI Rule Hit"

 Adding next line to force next request to web1 for when cache is reloaded
pool [LB::server pool] member x.x.x.x 80

set expire_cache_trigger 1
HTTP::uri [string range [HTTP::uri] 0 end-7]

} else {

 If request was not from a valid source, log the attempt
log "Received expire cache directive from non-admin IP [client_addr], ignoring"

 redirect them to the page without the expire command.
HTTP::uri [string range [HTTP::uri] 0 end-7]

}
}
}

when CACHE_REQUEST {

 Check if the variable has been set instructing this event to expire the cache
if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger ne "" ) } {

CACHE::expire
log "Cache Expired"

 Unset the variable so we don't keep calling this function needlessly
unset expire_cache_trigger

}
}

Any ideas?

Thanks,

Caleb

Caleb

 

 

Im looking to so something very similar,so tjis looks like a very good start.

 

 

 

a couple of changes i'd make:

 

1) give the expire trigger variable a defaiult value of 0. this will simplify the if check in the cache_request block.

 

2) add a log line to the cache_request block outside of the if block. This will show if the request is getting that far...

 

 

 

hope that helps.

 

 

 

Gav

 

Caleb

I've done some testing here, and think I've got a working iRule...

Apologies, i've striped out some of your logic, however it should be easy enough to put back in...

Anyways, here's a copy of my iRule...

 when RULE_INIT {

 Debug variable
set static::ExpireRAMCacheDebug 1

}

when HTTP_REQUEST {

if { $static::ExpireRAMCacheDebug } { log local0. "Processing HTTP_REQUEST at Priority 500." }

 Set default cache trigger value. 
set expire_cache_trigger 0

 Check if from authorized IP address. 
if { [class match [IP::client_addr] equals Authorized_IPs] } {

 Check HTTP::query for the expire param.
if { [string tolower [HTTP::query]] contains "expire" } {
 Param present in HTTP::query
if { $static::ExpireRAMCacheDebug } { log "Request to expire [HTTP::uri] received." }

 Set the expire trigger.
set expire_cache_trigger 1

 Save a copy of the URI with the param and value removed
set uri [string map [list "expire" ""] [HTTP::uri]]

 Replace && with & in the updated query string if && is present and set the URI to this
set uri [string map "&& &" $uri]

 Check if there are no other query items
if { $uri contains "&" } {
 Another query param exists
set uri [string map [list "?&" "?"] $uri]
 Log the cached and updated values for the URI
log local0. "Other Query params present. Original URI: [HTTP::uri], updated URI: $uri"
HTTP::uri $uri

} else {
set uri [string map [list "?" ""] $uri]
 Log the cached and updated values for the URI
log local0. "No Query params present. Original URI: [HTTP::uri], updated URI: $uri"
HTTP::uri $uri
}
}
}
}

when CACHE_REQUEST {

if { $static::ExpireRAMCacheDebug } { log local0. "Processing CACHE_REQUEST at Priority 500." }

 Check for expire_cache_trigger
if { $expire_cache_trigger } {

 Expire the CACHE entry.
if { $static::ExpireRAMCacheDebug } { log local0. "Expiring CACHE item $uri." }
CACHE::expire

 Record the item expiry. 
log "Cache Item $host$uri Expired"

 Tidy up
unset expire_cache_trigger
}
} 

It seems to work for me on some quick testing... Will give it a better run-down tomorrow...

Comments/Improvements welcome...

Cheers

Gavin