Live data from Hacker News

GitHub is aggressively caching raw.github, breaking many use cases

github.com

101–110 of 145 posts

Re: GitHub is aggressively caching raw.github, breaking many use cases

#101
post #57
post #37

Earlier quoted context omitted.

So we are back to file_v1, file_v1.1, file_v1.2 etc?

No, we are where we should have always been: > GET /file/latest HTTP/1.1

Sure... but their CDN is caching the results for /file/latest, so they'd either have to handle cache expiry differently or the same bug would happen.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#102
post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

There are actually only two hard problems in computer science: 0) Cache invalidation 1) Naming things 5) Asynchronous callbacks 2) Off-by-one errors 3) Scope creep 6) Bounds checking

Haven't seen that variant before! Thanks.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#103
post #24
post #13

Earlier quoted context omitted.

That doesn't solve cache invalidation; that just means you're always invalidating the cache even in cases where you don't actually want to.

Content addressing causes the name to change only when the content changes, which also means the name doesn't change if the content doesn't change, this by definition you don't have spurious cache invalidations

And how do you check what the new name of the content you haven't seen is?

Re: GitHub is aggressively caching raw.github, breaking many use cases

#104
post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

Github raw seems like the simplest system for solving cache invalidation: invalidate the cache of a changed file when it’s pushed. They have access to both GitHub and the raw service. I know there are usually all sorts of layers between that make interconnectivity logistically complicated, but am I wrong that at the top-level it’s that simple?

GitHub responded that it was simply a bug in that Cache-Control was set to a day instead of 5 minutes. It's already been fixed.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#106

Earlier quoted context omitted.

There are actually only two hard problems in computer science: 0) Cache invalidation 1) Naming things 5) Asynchronous callbacks 2) Off-by-one errors 3) Scope creep 6) Bounds checking

You forgot the 'Segmentation Fault' at the last line

It is on the list, but was never written to stdout because of a NULL pointer exception.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#107
post #84

I don't think raw was supposed to be used as your CDN

It doesn't matter what it was supposed to be used for. It's bad data in any context, even ones where you can't think of a way to blame the victim for holding it wrong.

It's not bad data though, it's just old and cached, which seems fine IMO even if the stale period is longer than you might expect.

It's not like it's serving the wrong content for a specific commit ref, it's when you request the file as on a branch, and that branch has recently changed.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#108

Earlier quoted context omitted.

Isn’t this where stuff like ETAGs are supposed to help? Not completely solve, but at least help reduce the problem a bit more?

Yes, but cache invalidation is a hard problem, so most services side-step etags (+ if-none-match), only to hit caches elsewhere.

[flagged]

Re: GitHub is aggressively caching raw.github, breaking many use cases

#110
post #63
post #46

Earlier quoted context omitted.

The commit ID seems to be part of the URL you can use https://raw.githubusercontent.com/burekasKodi/repository.bur... I'm fine with them caching "latest" tbh.

I’m not fine with latest not actually being latest. That defeats the point of the URL.

I get the issue but I do strongly feel that you should expect http resources to be cached. It's so common of a thing expecting it to never be cached is unreasonable as a design.

I would expect the headers to make this clear, however.

Post reply on HN