Forum Discussion

David_Patino_20's avatar
David_Patino_20
Icon for Nimbostratus rankNimbostratus
May 04, 2012

Instructing BigIP to expire RAM Cache when it receives a URL Query

Howdy Everyone.

I've an issue I've been working on.

I'm trying to develop a irule that will check the URI::query for an expire entry. If it finds one, it will instruct the system to do a CACHE::expire.

It's sort of working, I can't seem to pin down the proper order of operations.

Here's what I have so far:

when HTTP_REQUEST {
 Disable cache for a defined set of folders
if { [class match [HTTP::uri] contains folders_to_not_cache] }{
CACHE::disable
return
}

if { [string tolower [HTTP::uri]] contains "?expire" } {
log "Request to expire [getfield [string tolower [HTTP::uri]] "?expire" 1] received."
set expire_cache_trigger 1
HTTP::redirect "http://[HTTP::host][getfield [string tolower [HTTP::uri]] "?expire" 1]"
}
}

when CACHE_REQUEST {
if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger==1 ) } {
CACHE::expire
log "Cache Expired"
unset expire_cache_trigger
}
}

When I try this command on a system that's local to the external interface of the unit, it works.

Log output sample:

May 4 14:10:58 local/tmm info tmm[4853]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /test.html received. May 4 14:10:58 local/tmm info tmm[4853]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired

And indeed doing a tmsh show profile ramcache uri /test.html shows that the 'received' and 'expires' timestamps have updated.

However, if I execute a url?expire from a remote site the variable is no longer set when CACHE_REQUEST runs, so the actual CACHE::expire command never executes.

I've tried running the CACHE::expire inside the CACHE_RESPONSE event, but at that time it seems to have no effect. The command produces no errors in the log, but when viewing the cache details for the URI the received and expire timestamps do not update.

Any thoughts?

thanks.

David

8 Replies

  • Hi David,

    Can you try this without the redirect? Also, you can parse the URI without the query string using HTTP::path. To get just the query string you can use HTTP::query.

    when HTTP_REQUEST {
     Disable cache for a defined set of folders
    if { [class match [HTTP::uri] contains folders_to_not_cache] }{
    CACHE::disable
    return
    }
    
    if { [string tolower [HTTP::query]] starts_with "expire" } {
    log "Request to expire [HTTP::path] received."
    set expire_cache_trigger 1
    
     Remove the query string from the URI
    HTTP::uri [HTTP::path]
    }
    }
    
    when CACHE_REQUEST {
    if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger==1 ) } {
    CACHE::expire
    log "Cache Expired"
    unset expire_cache_trigger
    }
    }
    

    Aaron
  • Trying it without the redirect I'm seeing the same behavior. It's working for some computers that send the ?expire, and not for others...

     

     

    I wasn't using the URI::path as some of the pages in cache have other query parameters associated with them that still need to be passed to the servers in the pool, such as:

     

     

    author.html?page=4?expire
  • Who/what is appending the "?expire" string? If there is already a query string in the URI, it would be appended with an ampersand like author.html?page=4&expire. If you want to remove expire from the query string you could use string map:

    
    if { [string tolower [HTTP::query]] ends_with "expire" } {
    log "Request to expire [HTTP::path] received."
    set expire_cache_trigger 1
    
     Remove expire from the query string
    HTTP::uri [string range [HTTP::uri] 0 end-7]
    }
    

    Aaron
  • That's a good point, changes incorporated.

     

     

    Here's soemthing that confuses me more...

     

     

    From the remote site if we open the url in a browser with ?expire, then http_request sees it and sets the variable, but the cache_request doesn't see that the variable exists.

     

     

    But if we use curl from the same computer to open the same url, it works...
  • Ok, Disregard. Found the issue.

     

     

    It was on the browser of the remote computer we were testing... something was preventing it from running, maybe an existing connection or something. If we went to private browsing mode, it worked.

     

     

    So, wasn't a problem with the irule that I can see.

     

     

    Thank you for the heads up on the string map.
  • UPDATE:

    Below is the updated iRule and a few logs. Basically when a page is modified by an author and then saved, a single request is made to the uri with 'expire' added as a query: uri?expire

    The goal is that when the F5 receives the uri?expire it will completely remove the uri from it's existing ram cache and grab the newly updated uri which it will then cache.

    What is happening instead is that the F5 continues to serve the old version of the uri that it was supposed to have cleared.

    We did a test where we:

    1. Updated and saved a page with the content: TEST 1

    2. Verified it saved correctly and was being served correctly by accessing the server locally and bypassing the F5

    3. Manually requested the URI with uri?expire

    4. Used a browser in private mode by deleting all cookies and cache, then requesting the uri to look if the content showed TEST 1

    We repeated this process incrementally by repeating the steps above and incrementing the TEST number. What we found in say TEST 4 was that the F5 would sometimes show TEST 3, and even sometimes show TEST 2, but never showed TEST 4.

    We could occasionally see the updated version of the uri by requesting uri?expire= or uri?expire?expire

    Here are some logs:

    13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
    13:03:16 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired
    13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
    13:03:18 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired
    13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Request to expire /anything/ received from x.x.x.x.
    13:03:27 tmm1[5121]: 01220002:6: Rule iRule-Cache-Processing : Cache Expired

    And here is the iRule.

    iRule-Cache-Processing

    when HTTP_REQUEST {
    
     If the URL contains a folder that we're not supposed to cache, disable the cache
    if { [class match [HTTP::uri] contains folders_to_not_cache] }{
    
    CACHE::disable
    return
    
    } else {
    
     Ignore any cache-control headers sent by the user's browser.
    if { [HTTP::header exists "Cache-Control"] } { HTTP::header remove "Cache-Control" }
    
     Turn on caching
    CACHE::enable 
    }
    
    
     Convert the query options to lower case, then look for "expire" in them
    if { [string tolower [HTTP::query]] ends_with "expire" } {
    
     If we found the request, make sure it's coming from a valid admin IP
    if { [class match [client_addr] equals admin_ips] } {
    
     If it's a valid source, then log the request and set the variable to flag the cache to be expired in the CACHE_REQUEST event.
    log "Request to expire [getfield [string tolower [HTTP::uri]] "?expire" 1] received from [client_addr]."
    log local0. "URI Rule Hit"
    
     Adding next line to force next request to web1 for when cache is reloaded
    pool [LB::server pool] member x.x.x.x 80
    
    set expire_cache_trigger 1
    HTTP::uri [string range [HTTP::uri] 0 end-7]
    
    } else {
    
     If request was not from a valid source, log the attempt
    log "Received expire cache directive from non-admin IP [client_addr], ignoring"
    
     redirect them to the page without the expire command.
    HTTP::uri [string range [HTTP::uri] 0 end-7]
    
    }
    }
    }
    
    when CACHE_REQUEST {
    
     Check if the variable has been set instructing this event to expire the cache
    if { ( [info exists expire_cache_trigger] ) && ( $expire_cache_trigger ne "" ) } {
    
    CACHE::expire
    log "Cache Expired"
    
     Unset the variable so we don't keep calling this function needlessly
    unset expire_cache_trigger
    
    }
    }

    Any ideas?

    Thanks,

    Caleb
  • Caleb

     

     

    Im looking to so something very similar,so tjis looks like a very good start.

     

     

     

    a couple of changes i'd make:

     

    1) give the expire trigger variable a defaiult value of 0. this will simplify the if check in the cache_request block.

     

    2) add a log line to the cache_request block outside of the if block. This will show if the request is getting that far...

     

     

     

    hope that helps.

     

     

     

    Gav

     

  • Caleb

    I've done some testing here, and think I've got a working iRule...

    Apologies, i've striped out some of your logic, however it should be easy enough to put back in...

    Anyways, here's a copy of my iRule...

     when RULE_INIT {
    
     Debug variable
    set static::ExpireRAMCacheDebug 1
    
    }
    
    when HTTP_REQUEST {
    
    if { $static::ExpireRAMCacheDebug } { log local0. "Processing HTTP_REQUEST at Priority 500." }
    
     Set default cache trigger value. 
    set expire_cache_trigger 0
    
     Check if from authorized IP address. 
    if { [class match [IP::client_addr] equals Authorized_IPs] } {
    
     Check HTTP::query for the expire param.
    if { [string tolower [HTTP::query]] contains "expire" } {
     Param present in HTTP::query
    if { $static::ExpireRAMCacheDebug } { log "Request to expire [HTTP::uri] received." }
    
     Set the expire trigger.
    set expire_cache_trigger 1
    
     Save a copy of the URI with the param and value removed
    set uri [string map [list "expire" ""] [HTTP::uri]]
    
     Replace && with & in the updated query string if && is present and set the URI to this
    set uri [string map "&& &" $uri]
    
     Check if there are no other query items
    if { $uri contains "&" } {
     Another query param exists
    set uri [string map [list "?&" "?"] $uri]
     Log the cached and updated values for the URI
    log local0. "Other Query params present. Original URI: [HTTP::uri], updated URI: $uri"
    HTTP::uri $uri
    
    } else {
    set uri [string map [list "?" ""] $uri]
     Log the cached and updated values for the URI
    log local0. "No Query params present. Original URI: [HTTP::uri], updated URI: $uri"
    HTTP::uri $uri
    }
    }
    }
    }
    
    when CACHE_REQUEST {
    
    if { $static::ExpireRAMCacheDebug } { log local0. "Processing CACHE_REQUEST at Priority 500." }
    
     Check for expire_cache_trigger
    if { $expire_cache_trigger } {
    
     Expire the CACHE entry.
    if { $static::ExpireRAMCacheDebug } { log local0. "Expiring CACHE item $uri." }
    CACHE::expire
    
     Record the item expiry. 
    log "Cache Item $host$uri Expired"
    
     Tidy up
    unset expire_cache_trigger
    }
    } 

    It seems to work for me on some quick testing... Will give it a better run-down tomorrow...

    Comments/Improvements welcome...

    Cheers

    Gavin