How does WA do ETags?

I should add that Etags are not being passed up by the origin servers, they are sending Expires but WAM is converting Expires in to Etags+Cache-Control. I assume that is by design. This makes troubleshooting very difficult.

Generally speaking, WA generates the same e-tag for the exact same object residing on different servers at same path. For example if blah.gif exists on /image/blah.gif on all the servers being load-balanced, WA would generate the same etag for the object when it sends the response to the browser. The e-tag generated will be consistent across all servers. Thus negating the possibility of the browser downloading the same object multiple times because the e-tag generated on each servers are different.

 

 

However, if the same object reside on a different path then WA would respond with a different etag for that same object. So path/location of the object does come into consideration in WA etag generation. In addition, WA will generate a unique etag for normalized objects. So the same object at the same path on each server if normalized will end-up having different e-tags.

 

 

WA will always generate e-tags for objects it serves. the tags generated also plays an important role in IBR. WA converts origin server Expires to its own as defined in the policy. This setting can be changed.

 

 

Would you please explain as to why this makes it difficult for you to troubleshoot? What exactly is the problem that you are trying to troubleshoot?

Posted By s.isa@f5.com on 01/10/2009 1:03 PM

 

 

WA will always generate e-tags for objects it serves. the tags generated also plays an important role in IBR. WA converts origin server Expires to its own as defined in the policy. This setting can be changed.

 

 

Would you please explain as to why this makes it difficult for you to troubleshoot? What exactly is the problem that you are trying to troubleshoot?

 

 

 

The reason why it is difficult for us to troubleshoot, is because our origin servers are sending specific Date+Expires but we are seeing cases where it seems like those are not being honored, despite having the policy configured to use HTTP lifetime headers. The documentation only mentions Cache-Control and not Expires which leads me to suspect it is maybe ignoring those and falling back to the Maximum Age on the node.

 

 

I am curious about what you mean by "This setting can be changed"? Can you elaborate on that a bit?

 

 

The other reason this is a problem for us has to do with the fact that we are using Akamai up-stream and they do not honor Cache-Control: Private.

 

 

What are the date + expires values? I have seen issues when the values are set significantly in the future (greater than 1 year) that WA will ignore them. The pvac log will show that an invalid expires was encountered and it is being set to a different value.

 

 

The way WA sets the max-age or expires can be modified in build 9.4.5 HF 2 and later via he 'Ignore no-cache headers in the response' check box. In 9.4.5 HF 2 and later if that box is selected WA will ignore all cache-control headers. If any of the cache-control headers need to be preserved for use with upstream devices de-selecting this item will enable WA to pass on and use all cache-control settings from the origin servers. The exception here is the above issue, if the date or expires is considered invalid WA will compute that based on the values in the policy. When using the origin server cache control headers WA will use the s-maxage for its lifetime setting and the max-age value will be used for the client lifetime settings. If one of these is missing the values from the policy will be inserted.

 

 

Thanks, Dawn. That's a good explaination but doesn't seem to jive with what I am seeing:

 

 

Here is what a response from the origin looks like:

 

 

HTTP/1.1 200 OK

 

Date: Tue, 23 Dec 2008 20:45:18 GMT

 

Server: Apache

 

Expires: Tue, 23 Dec 2008 23:45:27 GMT

 

Content-Length: 60359

 

Connection: close

 

Content-Type: text/html; charset=utf-8

 

 

 

Now, here is the response for the same request thru the WA:

 

 

HTTP/1.1 200 OK

 

Server: Apache

 

Content-Type: text/html; charset=utf-8

 

Date: Tue, 30 Dec 2008 20:49:57 GMT

 

Content-Length: 9929

 

Content-Encoding: gzip

 

ETag: "pvf9e3703b808734df7f09fc19af3ca4b7"

 

X-WAM-ORIG-CC:private, max-age=10800

 

Cache-Control: private

 

X-WAM-RC-MASQ: On

 

X-PvInfo: [S10201.C17445.A28661.RA0.G706F.U5315C814].[OT/html.OG/pages]

 

Connection: Keep-Alive

 

Vary: Accept-Encoding

 

Accept-Ranges: none

 

X-PVia: webacc04

 

 

 

Where did the Expires header go? Where is the max-age and/or s-maxage?

 

 

On that node of my policy I have it set to preserve cc headers (box is un-checked for response). I went ahead and opened a case on this, since it doesn't seem to make sense.

 

 

Our main problem with this is we have Akamai sitting in front of WA (WA is Akamai's origin) and when they get a "Cache-Control: private" they cache it with a default (and too long) time.

 

Here is the response I received from Akamai about this issue:

 

 

Your configuration is set to honor CC header however, "private" is not a value that we would honor.

 

Instead of Private, would it be possible for you to set your load webaccelerator devices to send the "no-store" value?

 

There is a note regarding the private directive that says: "This usage of the word private only controls where the response may be cached, and cannot ensure the privacy of the message content." So basically private doesn't allow caching of the content but it doesn't say we cannot store it. Our server are cache servers but when content is not reused after a while it does get stored on the local disk (moved from memory to disk).

 

Therefore the private directive is not precise enough to ensure the content won't be cached. Private is more directed to browsers and local proxies but doesn't necessarily apply to our infrastructure. This is why we only honor "no-store".

 

Considering how our server work, you'll see that the no-store directive matches much more.

 

 

 

 

 

What makes troubleshooting even harder is that Akamai caches the "miss" with the X-PvInfo: S10201 header so all Akamai responses look like WA misses.

 

It seems like if we could modify the WA to send no-store on a miss, that would do the trick, but we'll see what support comes back with.

Thanks for opening a case on this. It defnitely doesn't seem to be working as it should for HTML pages. I also find Akamai's response and interpretation of the rfc interesting. According to rfc 2616 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.htmlsec14.9.1(Click here)

 

 

"private

 

Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache. This allows an origin server to state that the specified parts of the response are intended for only one user and are not a valid response for requests by other users. "

 

 

If Akamai doesn't honor the private header and given that there appears to be a discrepency in how max-age values from the server is handled for HTML pages I would recommend an iRule to add the no-store header to the cache-control header.

I did notice some bizarre LTM behaviour in regards to setting Maximum Cache vs original Etags-Cache-control, duplicate zero content length objects being returned to browser, furthermore huge bandwidth consumption utilization spikes.

 

 

 

After reading This http://devcentral.f5.com/Default.aspx?tabid=63&articleType=ArticleView&articleId=163 doc

 

 

 

which recommends to leave Lifetime rules at their default settings, and allow WebAccelerator's Lifetime Heuristic to refresh objects that appear to be regularly modified before Max Age is reached.

 

It also states that Enabling both "Use HTTP lifetime headers if present"and Maximum Age can result in conflicting cache headers sent to the client.

 

 

So I simply set the Max Age, disabled Insert Age Header, and syncronised the LTM with NTP and performance improved significanly.

 

yeah, there's definitely some conflicts when it comes to "Use HTTP lifetime headers if present" and Max Age.

 

Look at this fine example:

 

 

original origin response:

 

HTTP/1.1 200 OK

 

Date: Fri, 23 Jan 2009 19:58:16 GMT

 

Server: Apache

 

Expires: Fri, 23 Jan 2009 19:58:16 GMT

 

Pragma: no-cache

 

Cache-control: no-cache

 

Connection: close

 

Content-Type: text/javascript; charset=UTF-8

 

 

 

 

same request thru WA with 1 hr max age set on node:

 

HTTP/1.1 200 OK

 

Server: Apache

 

Content-Type: text/javascript; charset=UTF-8

 

Date: Fri, 23 Jan 2009 19:41:33 GMT

 

*Content-Length: 0

 

Content-Encoding: gzip

 

ETag: "pvca0c79c28b4c890bf503eade4820ee58"

 

X-WAM-ORIG-CC: private, max-age=3600

 

Expires: Fri, 23 Jan 2009 20:41:33 GMT

 

*Cache-Control: max-age=3600, no-cache

 

X-PvInfo: [S10201.C17445.A62836.RA62791.GF651.U6A20CD3B].[OT/all.OG/includes]

 

X-Cnection: Close

 

Vary: Accept-Encoding

 

Accept-Ranges: bytes

 

 

 

so the official Recommendation on this is:

 

Configure only one of these options, or coordinate the value of WebAccelerator's "Max Age" setting with the cache headers set by the webserver.

 

 

Um, yeah but then what's the point of "Use HTTP lifetime headers if present"? If I had that off and the origin sent a no-cache it would get ignored and cached, no?

 

And conversely, if I don't configure a max age, I don't get any Expires or Last-Modified or max-age info (as seen in my previous post).

 

 

I hope they fix this soon, it's driving me nuts.

yes, this is still a "problem" although there are workarounds. PS: this is still not fixed in 10.x code.