Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ETag and HTTP Caching

rednafi.com

Post reply on HN
Print view View original

71–80 of 92 posts

Re: ETag and HTTP Caching

#71

Post by dragonwriter »

jerf wrote: post #70
stingraycharles wrote: post #32

Earlier quoted context omitted.

My coworkers insisted on always returning 200 and having the status code in JSON. At least at that point it’s clearly not HTTP anymore, and it’s better than pretending to be compliant like your Bob. But something dies inside me whenever I have to work with it.

I've taken a very operational view of HTTP errors, which is "What do I want things receiving this error to do?" Unfortunately, that's not a clean question since there's no list you can simple consult to get all behaviors that all HTTP error messages cause. The most important of these is, if this is being accessed by a browser, what will the error code make it do? Fortunately, for a lot of my API-type work, I also get…

400 you screwed up vs 500 I screwed up is always better than 200 OK not really.

You can get more specific, and for REST style APIs the correct specific HTTP status code is usually apparent for both successful and unsuccessful requests, but 2xx/4xx/5xx is simple and should be trivial to determine for anything you are using HTTP for even if it's not REST-like.

Re: ETag and HTTP Caching

#72

Post by alganet »

jerf wrote: post #70
stingraycharles wrote: post #32

Earlier quoted context omitted.

My coworkers insisted on always returning 200 and having the status code in JSON. At least at that point it’s clearly not HTTP anymore, and it’s better than pretending to be compliant like your Bob. But something dies inside me whenever I have to work with it.

I've taken a very operational view of HTTP errors, which is "What do I want things receiving this error to do?" Unfortunately, that's not a clean question since there's no list you can simple consult to get all behaviors that all HTTP error messages cause. The most important of these is, if this is being accessed by a browser, what will the error code make it do? Fortunately, for a lot of my API-type work, I also get…

You're right about being safe rather than unecessarily fancy.

A lot of codes get real usage though. CDNs use ETags extensively to save bandwidth. It can save a lot of server time.

  - https://developers.cloudflare.com/cache/reference/etag-headers/
  - https://techdocs.akamai.com/property-mgr/docs/val-entity-tag
Varnish, HAProxy, Apache mod-proxy, nginx all can do similar things. Some of them can do this even if you always return 200 (by having rewrite rules and so on). It is often better to leave this kind of work to some upper abstract layer. Some of thse codes are only applicable in a layered system (502, for example, often seen when nginx can't reach a backend application), so they seem useless to developers, but they're not.

For APIs, other stuff uses those codes. Tools like DataDog and NewRelic will get better if you use generic 400 and generic 500 for client and server errors respectively. You can make them work with 200s and a little configuration though, but it's extra work.

If you never needed any of this, it's better not to use it.

Re: ETag and HTTP Caching

#73

Post by alganet »

HdS84 wrote: post #56
alganet wrote: post #25

This is nice. It reminds of how miserable my life is. — Which HTTP code I should return for my API? I already used 404, 403, but I need another one. Damn, HTTP is so old and it makes no sense. — You can't use HTTP codes like that Bob, they're not a free choice. They're for the protocol, not for your app. — Let's look at the list. Hm... "412 Precondition Failed". Hey, it sounds nice. It fits to my use case. I'm gonna…

The problem ist that there are multiple status codes which should be used (400, 422, 500 etc). But others are massive footguns. Would have been better to have a "technical" status code and a "application status code", but hey that would have required better engineering.

RTFM. It's all there. Not a blogpost, not a cheat sheet, the manual (in this case, the spec). Yes, it takes a while.

Anything can be a footgun if misused.

Re: ETag and HTTP Caching

#74

Post by jerf »

dragonwriter wrote: post #71
jerf wrote: post #70

Earlier quoted context omitted.

I've taken a very operational view of HTTP errors, which is "What do I want things receiving this error to do?" Unfortunately, that's not a clean question since there's no list you can simple consult to get all behaviors that all HTTP error messages cause. The most important of these is, if this is being accessed by a browser, what will the error code make it do? Fortunately, for a lot of my API-type work, I also get…

400 you screwed up vs 500 I screwed up is always better than 200 OK not really. You can get more specific, and for REST style APIs the correct specific HTTP status code is usually apparent for both successful and unsuccessful requests, but 2xx/4xx/5xx is simple and should be trivial to determine for anything you are using HTTP for even if it's not REST-like.

I do tend to use those in exactly that way, yes.

However, while your mileage may vary, I end up getting the same complaint from the users either way. Even when my 400 contains an exact reason why the input is incorrect.

Granted, on the one hand, this can be fixed on the individual level, but on the other hand, it's the same effect writ small that when writ large makes the response codes nearly useless, so this post is maybe half cathartic grousing. I can't push caring about response codes. I can document it, I can yield detailed errors, and I can be as careful as I like, but this is a "it takes two to tango" situation and at scale, on average, the other end doesn't want to tango.

Re: ETag and HTTP Caching

#75

Post by jerf »

alganet wrote: post #72
jerf wrote: post #70

Earlier quoted context omitted.

I've taken a very operational view of HTTP errors, which is "What do I want things receiving this error to do?" Unfortunately, that's not a clean question since there's no list you can simple consult to get all behaviors that all HTTP error messages cause. The most important of these is, if this is being accessed by a browser, what will the error code make it do? Fortunately, for a lot of my API-type work, I also get…

You're right about being safe rather than unecessarily fancy. A lot of codes get real usage though. CDNs use ETags extensively to save bandwidth. It can save a lot of server time. - https://developers.cloudflare.com/cache/reference/etag-headers/ - https://techdocs.akamai.com/property-mgr/docs/val-entity-tag Varnish, HAProxy, Apache mod-proxy, nginx all can do similar things. Some of them can do this even if you alway…

I should indeed have clarified that the browser web has a much richer set of headers and response codes in use, and they are truly useful, and anyone serving web pages at scale should indeed learn about them. IIRC it's still about 1/3rd to 1/4th of the nominally defined HTTP response codes that are useful, but it's still something.

The non-browser web, they approach useless. Which I'm not happy about and not celebrating or advocating for. It's just how it is.

Re: ETag and HTTP Caching

#76

Post by sa46 »

alganet wrote: post #36
stingraycharles wrote: post #32

Earlier quoted context omitted.

My coworkers insisted on always returning 200 and having the status code in JSON. At least at that point it’s clearly not HTTP anymore, and it’s better than pretending to be compliant like your Bob. But something dies inside me whenever I have to work with it.

I'd say returning 200 for all successes is reasonable if the responses are simple. Returning 200 for an error makes no sense. Having 400s and 500s is the simplest way to have observability over protocol behavior (think logs, error rates, etc). If you use all 200s, you'd have to re-implement observability by yourself, so you lose simplicity that you gained by ignoring those statuses. It's the same thing with caching s…

I prefer to treat HTTP as a transport layer. A 200 means the transport succeeded but doesn’t indicate that the RPC succeeded.

Re: ETag and HTTP Caching

#77

Post by JaggedJax »

lstamour wrote: post #44
clessg wrote: post #7

Earlier quoted context omitted.

That's because it isn't really generating a weak ETag. From the article: > You could make the `calculateETag` function format-agnostic, so the hash stays the same if the JSON format changes but the content does not. The current `calculateETag` implementation is susceptible to format changes, and I kept it that way to keep the code shorter. They seem to agree, a true weak ETag implementation would probably be trickier…

I could probably write one pretty quick, under the assumption that we are storing only JS-compatible JSON with no encoding hiccups (JSON sadly isn’t as standard as it appears at first glance…) just hash(JSON.stringify(JSON.parse(fileText))) and you’re done. This assumes the same parse and serialize methods are expected to be used at both ends, that they only normalize formatting and that you don’t have to worry about…

I'm kind of thinking now, why bother with generating a weak ETag at all? Unless your backend is doing things that would commonly cause differences in JSON formatting for the same data, this is probably a rare case and not worth the extra effort or processing. Figure it out when you're at a scale that it actually matters, and stick with strong ETags for now.

It's good to know about this option for handling in the frontend if a system returns one though.

Re: ETag and HTTP Caching

#78

Post by ahepp »

stingraycharles wrote: post #32
alganet wrote: post #25

This is nice. It reminds of how miserable my life is. — Which HTTP code I should return for my API? I already used 404, 403, but I need another one. Damn, HTTP is so old and it makes no sense. — You can't use HTTP codes like that Bob, they're not a free choice. They're for the protocol, not for your app. — Let's look at the list. Hm... "412 Precondition Failed". Hey, it sounds nice. It fits to my use case. I'm gonna…

My coworkers insisted on always returning 200 and having the status code in JSON. At least at that point it’s clearly not HTTP anymore, and it’s better than pretending to be compliant like your Bob. But something dies inside me whenever I have to work with it.

I think that's one of the key differences between REST and other kinds of RPC architectures.

I've used SOAP and JSON-RPC, both of which (at least in many implementations) send RPCs as HTTP POST requests and receive 200 responses with any error messages in the body. They're just tunneling over HTTP. It's not necessarily wrong, although I'm convinced that leveraging the HTTP verbs and error codes with REST is a fundamentally better design for the use cases I've seen.

Re: ETag and HTTP Caching

#79

Post by yencabulator »

quectophoton wrote: post #40
alganet wrote: post #25

This is nice. It reminds of how miserable my life is. — Which HTTP code I should return for my API? I already used 404, 403, but I need another one. Damn, HTTP is so old and it makes no sense. — You can't use HTTP codes like that Bob, they're not a free choice. They're for the protocol, not for your app. — Let's look at the list. Hm... "412 Precondition Failed". Hey, it sounds nice. It fits to my use case. I'm gonna…

To be fair, to me HTTP looks like: - The first line of an HTTP request has its own format (space-separated-ish), and mixes method and URL path. - The URL path in that first line has its own format and weird escaping, and mixes one path with zero-or-more key value pairs. - The headers have their own format. - The body has its own format. Most (all?) HTTP libraries for clients and servers abstract all that mess away in…

HTTP/2 or HTTP/3 don't look like that, anymore. The HEADERS frame is just a key-value map where the reserved keys :method, :scheme, :path etc are used for the previously top-level message elements.

Re: ETag and HTTP Caching

#80

Post by infogulch »

sa46 wrote: post #76
alganet wrote: post #36

Earlier quoted context omitted.

I'd say returning 200 for all successes is reasonable if the responses are simple. Returning 200 for an error makes no sense. Having 400s and 500s is the simplest way to have observability over protocol behavior (think logs, error rates, etc). If you use all 200s, you'd have to re-implement observability by yourself, so you lose simplicity that you gained by ignoring those statuses. It's the same thing with caching s…

I prefer to treat HTTP as a transport layer. A 200 means the transport succeeded but doesn’t indicate that the RPC succeeded.

This makes no sense at all. "you got a response" means the transport succeeded, "it timed out" means the transport failed.
Post reply on HN