ETag and HTTP Caching • Hacker News Board

Re: ETag and HTTP Caching

#81

Post by lstamour » Fri, Apr 12, 2024, 1:35 AM UTC

JaggedJax wrote: post #77

lstamour wrote: post #44

Earlier quoted context omitted.

I could probably write one pretty quick, under the assumption that we are storing only JS-compatible JSON with no encoding hiccups (JSON sadly isn’t as standard as it appears at first glance…) just hash(JSON.stringify(JSON.parse(fileText))) and you’re done. This assumes the same parse and serialize methods are expected to be used at both ends, that they only normalize formatting and that you don’t have to worry about…

I'm kind of thinking now, why bother with generating a weak ETag at all? Unless your backend is doing things that would commonly cause differences in JSON formatting for the same data, this is probably a rare case and not worth the extra effort or processing. Figure it out when you're at a scale that it actually matters, and stick with strong ETags for now. It's good to know about this option for handling in the fron…

Yeah, I’ve never really heard of “weak etags” before in any sort of common usage of the term. Honestly, most people tend to skip etags by embedding hashes in filenames directly, this way you can avoid any bad proxies serving up stale content or dropping headers. It’s rare these days to be an issue given the use of TLS end-to-end encryption, but I’m sure it still occasionally happens. And yes, the more serious approach to possibly poorly formatted JSON is to “normalize it” into the expected format. It’s less about caching and more about ensuring what you serve to your front end is consistent, even if you are liberal in what inputs you can handle. E.g. if someone gives you XML, rather than write a front end that can handle both XML and JSON, pull the data out of both and make your own JSON later.

Re: ETag and HTTP Caching

#82

Post by sa46 » Fri, Apr 12, 2024, 3:39 AM UTC

infogulch wrote: post #80

sa46 wrote: post #76

Earlier quoted context omitted.

I prefer to treat HTTP as a transport layer. A 200 means the transport succeeded but doesn’t indicate that the RPC succeeded.

This makes no sense at all. "you got a response" means the transport succeeded, "it timed out" means the transport failed.

That’s a strong assertion. There’s plenty of status codes that indicate something bad happened at the HTTP level but don’t convey information about the RPC.

- 404 might mean the URL was wrong

- 405 might mean swapping POST with PUT

Re: ETag and HTTP Caching

#83

Post by jbverschoor » Fri, Apr 12, 2024, 7:02 AM UTC

sa46 wrote: post #82

infogulch wrote: post #80

Earlier quoted context omitted.

This makes no sense at all. "you got a response" means the transport succeeded, "it timed out" means the transport failed.

That’s a strong assertion. There’s plenty of status codes that indicate something bad happened at the HTTP level but don’t convey information about the RPC. - 404 might mean the URL was wrong - 405 might mean swapping POST with PUT

Not the http level… at the level above, the user of your api.

Both of these are simple programming errors. You might as well have used the wrong host or protocol for that matter

Re: ETag and HTTP Caching

#84

Post by derkades » Fri, Apr 12, 2024, 9:14 PM UTC

johneth wrote: post #51

joosters wrote: post #43

I've not seen a very convincing use-case for ETags vs Last-Modified date caching. In the example request, the server still has to do all of the work generating the page, in order to calculate the ETag and then determine whether or not the page has changed. In most situations, it's simpler to have timestamps to compare against, because that gives the server a faster way to spot unmodified data. e.g. you get a HTTP req…

When I use ETags, I'll generate the ETag value once (on startup), then cache and serve that value. When the resource changes, regenerate the ETag value. Obviously doesn't work if you're using ETags on dynamic resources, but works well for non-dynamic, but unpredictably frequently changing resources.

And couldn't you do the same thing with a timestamp instead of a hash?

Re: ETag and HTTP Caching

#85

Post by nikolayasdf123 » Mon, Apr 22, 2024, 12:04 PM UTC

rednafi wrote: post #52

nikolayasdf123 wrote: post #45

if only browsers respected this. none of browsers use ETag and If-None-Match mechanism. instead they do their own wizzardy caching...

Hmm... Chrome seems to handle this quite well, at least in my experience. I'm curious about what sort of issue you encountered.

same as above

Re: ETag and HTTP Caching

#86

Post by nikolayasdf123 » Mon, Apr 22, 2024, 12:05 PM UTC

demurgos wrote: post #46

nikolayasdf123 wrote: post #45

if only browsers respected this. none of browsers use ETag and If-None-Match mechanism. instead they do their own wizzardy caching...

I was refactoring a project serving user uploaded files yesterday, and had the occasion to test caching. Both Firefox and Chrome used ETag and If-None-Match properly to check and cache queries. Which problems did you encounter? There was still one thing that surprised me a bit (but also makes sense). Images are fetched only once per page load in my testing. If an image with 60sec of cache is loaded, then removed by J…

the issue I got was:

1) I (AWS CloudFront) supply ETag and If-None-Match header; I can see that header in responses.

2) browsers sometimes do respect that (once) I see 304 in responses, but 99% of the time they don't include ETag/If-None-Match in requests and thus I never get 304 responses (albeit AWS CloudFront, resource, data — nothing changed) and instead they perform some other caching and reload whole resource again with TTL that does not seem to come from my headers, totally disregarding ETags/If-None-Match logic.

for videos it is even worse. unless you set `preload=none` in html, Safari, Firefox, Chrome will have all different policies trying to preload all videos on screen ignoring lazyload html tags. worse of all, caching does not work well, and videos will be attempted to be loaded almost every time and ETags/If-None-Match totally ignored.

Re: ETag and HTTP Caching

#87

Post by nikolayasdf123 » Mon, Apr 22, 2024, 12:05 PM UTC

withinboredom wrote: post #61

nikolayasdf123 wrote: post #45

if only browsers respected this. none of browsers use ETag and If-None-Match mechanism. instead they do their own wizzardy caching...

It works correctly. What do you mean?

look above

Re: ETag and HTTP Caching

#88

Post by withinboredom » Mon, Apr 22, 2024, 12:17 PM UTC

nikolayasdf123 wrote: post #86

demurgos wrote: post #46

Earlier quoted context omitted.

I was refactoring a project serving user uploaded files yesterday, and had the occasion to test caching. Both Firefox and Chrome used ETag and If-None-Match properly to check and cache queries. Which problems did you encounter? There was still one thing that surprised me a bit (but also makes sense). Images are fetched only once per page load in my testing. If an image with 60sec of cache is loaded, then removed by J…

the issue I got was: 1) I (AWS CloudFront) supply ETag and If-None-Match header; I can see that header in responses. 2) browsers sometimes do respect that (once) I see 304 in responses, but 99% of the time they don't include ETag/If-None-Match in requests and thus I never get 304 responses (albeit AWS CloudFront, resource, data — nothing changed) and instead they perform some other caching and reload whole resource a…

It sounds like you have caching disabled in your browser, either in dev-tools itself or through an extension. This is NOT normal behavior.

Re: ETag and HTTP Caching

#89

Post by nikolayasdf123 » Mon, Apr 22, 2024, 12:49 PM UTC

withinboredom wrote: post #88

nikolayasdf123 wrote: post #86

Earlier quoted context omitted.

the issue I got was: 1) I (AWS CloudFront) supply ETag and If-None-Match header; I can see that header in responses. 2) browsers sometimes do respect that (once) I see 304 in responses, but 99% of the time they don't include ETag/If-None-Match in requests and thus I never get 304 responses (albeit AWS CloudFront, resource, data — nothing changed) and instead they perform some other caching and reload whole resource a…

It sounds like you have caching disabled in your browser, either in dev-tools itself or through an extension. This is NOT normal behavior.

the same happens for all browsers, Chrome, Safari, Firefox with default settings. but do agree this is not normal behavior :/

actually caching is happening, but it does not follow ETags or Caching policy headers that backends return, instead some in-browser internal caching policy being run

Re: ETag and HTTP Caching

#90

Post by withinboredom » Mon, Apr 22, 2024, 12:54 PM UTC

nikolayasdf123 wrote: post #89

withinboredom wrote: post #88

Earlier quoted context omitted.

It sounds like you have caching disabled in your browser, either in dev-tools itself or through an extension. This is NOT normal behavior.

the same happens for all browsers, Chrome, Safari, Firefox with default settings. but do agree this is not normal behavior :/ actually caching is happening, but it does not follow ETags or Caching policy headers that backends return, instead some in-browser internal caching policy being run

I’ve never seen the behavior you are describing. I would check that you’re sending the headers correctly: https://neilmadden.blog/2020/05/18/a-flowchart-for-cache-con...

I’ve used that flowchart too many times.