cancel
Showing results for 
Search instead for 
Did you mean: 
Login & Join the DevCentral Connects Group to watch the Recorded LiveStream (May 12) on Basic iControl Security - show notes included.

Instructing BigIP to expire RAM Cache when it receives a URL Query

David_Patino_20
Nimbostratus
Nimbostratus
Howdy Everyone.

 

 

I've an issue I've been working on.

 

 

I'm trying to develop a irule that will check the URI::query for an expire entry. If it finds one, it will instruct the system to do a CACHE::expire.

 

 

It's sort of working, I can't seem to pin down the proper order of operations.

 

 

Here's what I have so far:

 

 

when HTTP_REQUEST { Disable cache for a defined set of folders if { [class match [HTTP::uri] contains folders_to_not_cache] }{ CACHE::disable return } if { [string tolower [HTTP::uri]] contains "?expire" } { log "Request to expire [getfield [string tolower [HTTP::uri]] "?expire" 1] received." set expire_cache_trigger 1 HTTP::redirect "http://[HTTP::host][getfield [string tolower [HTTP::uri]] "?expire" 1]" } } when CACHE_REQUEST { if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger==1 ) } { CACHE::expire log "Cache Expired" unset expire_cache_trigger } }

 

When I try this command on a system that's local to the external interface of the unit, it works.

 

 

Log output sample:

 

May 4 14:10:58 local/tmm info tmm[4853]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /test.html received. May 4 14:10:58 local/tmm info tmm[4853]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired

 

 

And indeed doing a tmsh show profile ramcache uri /test.html shows that the 'received' and 'expires' timestamps have updated.

 

 

However, if I execute a url?expire from a remote site the variable is no longer set when CACHE_REQUEST runs, so the actual CACHE::expire command never executes.

 

 

I've tried running the CACHE::expire inside the CACHE_RESPONSE event, but at that time it seems to have no effect. The command produces no errors in the log, but when viewing the cache details for the URI the received and expire timestamps do not update.

 

 

Any thoughts?

 

 

thanks.

 

 

David

 

8 REPLIES 8

hooleylist
Cirrostratus
Cirrostratus
Hi David,

 

 

Can you try this without the redirect? Also, you can parse the URI without the query string using HTTP::path. To get just the query string you can use HTTP::query.

 

 

when HTTP_REQUEST { Disable cache for a defined set of folders if { [class match [HTTP::uri] contains folders_to_not_cache] }{ CACHE::disable return } if { [string tolower [HTTP::query]] starts_with "expire" } { log "Request to expire [HTTP::path] received." set expire_cache_trigger 1 Remove the query string from the URI HTTP::uri [HTTP::path] } } when CACHE_REQUEST { if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger==1 ) } { CACHE::expire log "Cache Expired" unset expire_cache_trigger } }

 

 

Aaron

David_Patino_20
Nimbostratus
Nimbostratus
Trying it without the redirect I'm seeing the same behavior. It's working for some computers that send the ?expire, and not for others...

 

 

I wasn't using the URI::path as some of the pages in cache have other query parameters associated with them that still need to be passed to the servers in the pool, such as:

 

 

author.html?page=4?expire

hooleylist
Cirrostratus
Cirrostratus
Who/what is appending the "?expire" string? If there is already a query string in the URI, it would be appended with an ampersand like author.html?page=4&expire. If you want to remove expire from the query string you could use string map:

 

 

if { [string tolower [HTTP::query]] ends_with "expire" } { log "Request to expire [HTTP::path] received." set expire_cache_trigger 1 Remove expire from the query string HTTP::uri [string range [HTTP::uri] 0 end-7] }

 

 

Aaron

David_Patino_20
Nimbostratus
Nimbostratus
That's a good point, changes incorporated.

 

 

Here's soemthing that confuses me more...

 

 

From the remote site if we open the url in a browser with ?expire, then http_request sees it and sets the variable, but the cache_request doesn't see that the variable exists.

 

 

But if we use curl from the same computer to open the same url, it works...

David_Patino_20
Nimbostratus
Nimbostratus
Ok, Disregard. Found the issue.

 

 

It was on the browser of the remote computer we were testing... something was preventing it from running, maybe an existing connection or something. If we went to private browsing mode, it worked.

 

 

So, wasn't a problem with the irule that I can see.

 

 

Thank you for the heads up on the string map.

Caleb_ISC_13156
Nimbostratus
Nimbostratus
UPDATE:

 

 

Below is the updated iRule and a few logs. Basically when a page is modified by an author and then saved, a single request is made to the uri with 'expire' added as a query: uri?expire

 

 

The goal is that when the F5 receives the uri?expire it will completely remove the uri from it's existing ram cache and grab the newly updated uri which it will then cache.

 

 

What is happening instead is that the F5 continues to serve the old version of the uri that it was supposed to have cleared.

 

 

We did a test where we:

 

 

1. Updated and saved a page with the content: TEST 1

 

2. Verified it saved correctly and was being served correctly by accessing the server locally and bypassing the F5

 

3. Manually requested the URI with uri?expire

 

4. Used a browser in private mode by deleting all cookies and cache, then requesting the uri to look if the content showed TEST 1

 

 

We repeated this process incrementally by repeating the steps above and incrementing the TEST number. What we found in say TEST 4 was that the F5 would sometimes show TEST 3, and even sometimes show TEST 2, but never showed TEST 4.

 

 

We could occasionally see the updated version of the uri by requesting uri?expire= or uri?expire?expire

 

 

Here are some logs:

 

13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x. 13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired 13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x. 13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired 13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x. 13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired

 

 

And here is the iRule.

 

 

iRule-Cache-Processing

 

when HTTP_REQUEST { If the URL contains a folder that we're not supposed to cache, disable the cache if { [class match [HTTP::uri] contains folders_to_not_cache] }{ CACHE::disable return } else { Ignore any cache-control headers sent by the user's browser. if { [HTTP::header exists "Cache-Control"] } { HTTP::header remove "Cache-Control" } Turn on caching CACHE::enable } Convert the query options to lower case, then look for "expire" in them if { [string tolower [HTTP::query]] ends_with "expire" } { If we found the request, make sure it's coming from a valid admin IP if { [class match [client_addr] equals admin_ips] } { If it's a valid source, then log the request and set the variable to flag the cache to be expired in the CACHE_REQUEST event. log "Request to expire [getfield [string tolower [HTTP::uri]] "?expire" 1] received from [client_addr]." log local0. "URI Rule Hit" Adding next line to force next request to web1 for when cache is reloaded pool [LB::server pool] member x.x.x.x 80 set expire_cache_trigger 1 HTTP::uri [string range [HTTP::uri] 0 end-7] } else { If request was not from a valid source, log the attempt log "Received expire cache directive from non-admin IP [client_addr], ignoring" redirect them to the page without the expire command. HTTP::uri [string range [HTTP::uri] 0 end-7] } } } when CACHE_REQUEST { Check if the variable has been set instructing this event to expire the cache if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger ne "" ) } { CACHE::expire log "Cache Expired" Unset the variable so we don't keep calling this function needlessly unset expire_cache_trigger } }

 

 

 

 

Any ideas?

 

 

Thanks,

 

 

Caleb

GavinW_29074
Nimbostratus
Nimbostratus
Caleb

 

 

Im looking to so something very similar,so tjis looks like a very good start.

 

 

 

a couple of changes i'd make:

 

1) give the expire trigger variable a defaiult value of 0. this will simplify the if check in the cache_request block.

 

2) add a log line to the cache_request block outside of the if block. This will show if the request is getting that far...

 

 

 

hope that helps.

 

 

 

Gav

 

GavinW_29074
Nimbostratus
Nimbostratus
Caleb

 

 

I've done some testing here, and think I've got a working iRule...

 

 

Apologies, i've striped out some of your logic, however it should be easy enough to put back in...

 

 

Anyways, here's a copy of my iRule...

 

 

when RULE_INIT { Debug variable set static::ExpireRAMCacheDebug 1 } when HTTP_REQUEST { if { $static::ExpireRAMCacheDebug } { log local0. "Processing HTTP_REQUEST at Priority 500." } Set default cache trigger value. set expire_cache_trigger 0 Check if from authorized IP address. if { [class match [IP::client_addr] equals Authorized_IPs] } { Check HTTP::query for the expire param. if { [string tolower [HTTP::query]] contains "expire" } { Param present in HTTP::query if { $static::ExpireRAMCacheDebug } { log "Request to expire [HTTP::uri] received." } Set the expire trigger. set expire_cache_trigger 1 Save a copy of the URI with the param and value removed set uri [string map [list "expire" ""] [HTTP::uri]] Replace && with & in the updated query string if && is present and set the URI to this set uri [string map "&& &" $uri] Check if there are no other query items if { $uri contains "&" } { Another query param exists set uri [string map [list "?&" "?"] $uri] Log the cached and updated values for the URI log local0. "Other Query params present. Original URI: [HTTP::uri], updated URI: $uri" HTTP::uri $uri } else { set uri [string map [list "?" ""] $uri] Log the cached and updated values for the URI log local0. "No Query params present. Original URI: [HTTP::uri], updated URI: $uri" HTTP::uri $uri } } } } when CACHE_REQUEST { if { $static::ExpireRAMCacheDebug } { log local0. "Processing CACHE_REQUEST at Priority 500." } Check for expire_cache_trigger if { $expire_cache_trigger } { Expire the CACHE entry. if { $static::ExpireRAMCacheDebug } { log local0. "Expiring CACHE item $uri." } CACHE::expire Record the item expiry. log "Cache Item $host$uri Expired" Tidy up unset expire_cache_trigger } }

 

 

It seems to work for me on some quick testing... Will give it a better run-down tomorrow...

 

 

Comments/Improvements welcome...

 

 

Cheers

 

Gavin