Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researching a new form of HTTP caching optimization

blog.phusion.nl

Post reply on HN
Print view View original

11–20 of 50 posts

Re: Researching a new form of HTTP caching optimization

#11

Post by Argorak »

samarudge wrote: post #6
FooBarWidget wrote: post #5

Earlier quoted context omitted.

Good to know. Using edge side includes may be easier than trying to change the app to a semi single page app. But that only solves half of the problem. The other half is varying the response based on the value of a specific cookie. I've updated the blog post with information regarding edge side include.

Varnish also supports custom cache keys via the VCL function `vcl_hash` as documented in https://www.varnish-software.com/static/book/VCL_functions.h... I couldn't (quickly) find documentation on how to get the value of a specific cookie, but the server could send a user ID in a header or something Varnish can easily access to be used in the above function.

You will probably have to write a VCL plugin if you want to parse the cookie, but that's not a huge hassle.

Re: Researching a new form of HTTP caching optimization

#13

Post by WimLeers »

> What if the cache can parse cookies and vary the cached response by the value of a specific cookie, not the entire header?

> This is caching that Varnish and other “normal” HTTP caches (including CloudFlare) could not have done.

This is false. I'm not at all very familiar with Varnish, but I know this is easily possible, and has been used for many, many years.

E.g. for Drupal + Varnish, i.e. to only keep Drupal's session cookie, I found these examples, in less than a minute of googling:

- https://www.varnish-cache.org/trac/wiki/VarnishAndDrupal

- https://www.lullabot.com/blog/article/configuring-varnish-hi... (grep for "inclusion")

Everything in this article has been well-known for at least half a decade, yet is being presented as major technical breakthroughs. Too much marketing, IMO.

Re: Researching a new form of HTTP caching optimization

#14

Post by hannibalhorn »

Even ignoring the fact that edge side includes are the way to go when confronted with this problem, it sounds like they're basically saying they'd like to include the userid in the Vary header, but cannot because the Cookie header includes a bunch of other stuff.

Instead of parsing the Cookie header and doing a bunch of additional work in the cache layer, why not just add a separate custom header (X-User-Id, say) and Vary on that?

Re: Researching a new form of HTTP caching optimization

#15

Post by WimLeers »

hannibalhorn wrote: post #14

Even ignoring the fact that edge side includes are the way to go when confronted with this problem, it sounds like they're basically saying they'd like to include the userid in the Vary header, but cannot because the Cookie header includes a bunch of other stuff. Instead of parsing the Cookie header and doing a bunch of additional work in the cache layer, why not just add a separate custom header (X-User-Id, say) and…

You're talking about response headers. But this deals with sending responses, you only get requests. Request headers can't contain a custom header that you control (unless your JS is making the requests of course). Hence you have to transform the Cookie header into whatever you need.

Re: Researching a new form of HTTP caching optimization

#16

Post by michaelmior »

rcarmo wrote: post #7

Hmmm. I might be missing something here, but I routinely clean out cookies in "public"/unauthed URLs in Varnish, as well as hashing the cache based on _part_ of a specific cookie (the bit that defines, say, the site's theme, or a generic user role). They do mention that they didn't investigate how to do this in Varnish, but I recall having picked up the basic technique from one of the author's posts.

This was my first thought too. You can vary by any subset of cookies in Varnish if you simply normalize the Cookie header by stripping out irrelevant cookies. Another trick I've used is to parse the cookie to look for the user ID and set a User-Id header on the backend response and then set Vary: User-Id.

Re: Researching a new form of HTTP caching optimization

#17

Post by jgrahamc »

WimLeers wrote: post #13

> What if the cache can parse cookies and vary the cached response by the value of a specific cookie, not the entire header? > This is caching that Varnish and other “normal” HTTP caches (including CloudFlare) could not have done. This is false. I'm not at all very familiar with Varnish, but I know this is easily possible, and has been used for many, many years. E.g. for Drupal + Varnish, i.e. to only keep Drupal's s…

Yes, and we (CloudFlare) offer different caching based on cookies.

Re: Researching a new form of HTTP caching optimization

#19

Post by megaman821 »

chris_overseas wrote: post #8

I had a contract at a company about 13 years ago where I was working on a web-based CMS that had been built entirely in-house. Because each rendered page's content was built up in a hierarchical manner, I added a caching layer that allowed arbitrary portions of the rendered page "tree" to be cached all the way up to the entire page and HTTP response if possible. Each cached portion could be located quickly based on i…

I have seen this referred to as the "Russian doll" caching strategy. You can find several examples using Rail's partials or Django's cache template tag.

Re: Researching a new form of HTTP caching optimization

#20

Post by FooBarWidget »

jgrahamc wrote: post #17
WimLeers wrote: post #13

> What if the cache can parse cookies and vary the cached response by the value of a specific cookie, not the entire header? > This is caching that Varnish and other “normal” HTTP caches (including CloudFlare) could not have done. This is false. I'm not at all very familiar with Varnish, but I know this is easily possible, and has been used for many, many years. E.g. for Drupal + Varnish, i.e. to only keep Drupal's s…

Yes, and we (CloudFlare) offer different caching based on cookies.

Can you link to the relevant documentation?
Post reply on HN