Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub is aggressively caching raw.github, breaking many use cases

github.com

Post reply on HN
Print view View original

71–80 of 145 posts

Re: GitHub is aggressively caching raw.github, breaking many use cases

#71

Post by endymi0n »

ruuda wrote: post #7
TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

You can eliminate the cache invalidation problem by not reusing old names for new things.

Sounds a little bit like the standard recipe of any functional programming enthusiast: Just use pure functions!

Except that they‘re colliding all the time with that tiny problem that the world isn‘t pure... People expect the same page at the same URL, with different content, tomorrow, so good luck choosing a different one than the one they have bookmarked and that ranks on Google.

Caching doesn‘t get any less hard by trying to define the problem away.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#72

Post by usr1106 »

toastal wrote: post #67
usr1106 wrote: post #40

A bug: See https://github.com/orgs/community/discussions/46691#discussi... (Well, if we believe the statement "github engineer here". Of course every clown could write that, too)

[flagged]

Personally I prefer gitlab. Just wanted to look at a bug I noted in their product this week (European weeks start on Monday :) ). Gave up because I couldn't find my way around, whether it's already reported. But at least with unlimited time I could probably do it.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#74

Post by joshspankit »

TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

Github raw seems like the simplest system for solving cache invalidation: invalidate the cache of a changed file when it’s pushed.

They have access to both GitHub and the raw service. I know there are usually all sorts of layers between that make interconnectivity logistically complicated, but am I wrong that at the top-level it’s that simple?

Re: GitHub is aggressively caching raw.github, breaking many use cases

#75

Post by nikeee »

amadeuspagel wrote: post #33

I'm guessing that a lot of people embed these files in their website in one way or another, making aggressive caching necessary.

GitHub recommends using GH Pages for that. Raw links cause a higher server load and aren't even being served with the correct mime type.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#77

Post by nix23 »

croes wrote: post #37
Laaas wrote: post #20

This seems reasonable to me. Caches should last longer; if you want to be sure of you get the latest version, rename the file. a good trick is to include the hash of the file in the name to get content-addressing à la IPFS.

So we are back to file_v1, file_v1.1, file_v1.2 etc?

Like OpenVMS makes it on Filesystem level where the highest version is the "real" one:

Blupblub;1 Blupblub;2 --> Blupblub;2 = Blupblub

Actually a good idea ;)

Re: GitHub is aggressively caching raw.github, breaking many use cases

#78

Post by throwanem »

ruuda wrote: post #7
TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

You can eliminate the cache invalidation problem by not reusing old names for new things.

Not using old names for new things in this case means abandoning the concept of Git branch tags.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#79

Post by throwanem »

EdwardDiego wrote: post #54
iLoveOncall wrote: post #48

Earlier quoted context omitted.

> if you want to be sure of you get the latest version, rename the file. This is a beyond ridiculous statement. It is a BUG that you do not get the latest version of the file when viewing raw, not an error you made that you should address by having a filename driven versioning system. If I'm using Git and GitHub, it's specifically to NOT have to deal with v1, v1.1, final, final_for_real, final_of_the_finalest, this_t…

Ridiculous is expecting a service you're not paying for to serve files in a way that fits your use case when you've never entered into a contract that guarantees the behaviour you're relying on. You can use Git just fine without GitHub.

Who says I'm not paying for Github? But a Git porcelain that fails to show the version of a file it claims it's showing is a Git porcelain with a bug in it, regardless.

You don't have to take my word for it! https://github.com/orgs/community/discussions/46691#discussi...

Re: GitHub is aggressively caching raw.github, breaking many use cases

#80

Post by roundandround »

joshspankit wrote: post #74
TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

Github raw seems like the simplest system for solving cache invalidation: invalidate the cache of a changed file when it’s pushed. They have access to both GitHub and the raw service. I know there are usually all sorts of layers between that make interconnectivity logistically complicated, but am I wrong that at the top-level it’s that simple?

That is too simple for the feature they are using. The client itself has its own cache and the only way to fully prevent traffic from a client is to tell it content it caches will remain valid for some amount of time into the future.

For URLs that return the latest entry there is no valid amount of time known in advance by GitHub unless they want to introduce mandatory publication delays. For URLs of specific change sets, they should never be corrected again and an infinite cache is pretty much valid unless a user overrides good git practices.

I think GitHub frequently misidentifies which scenario they are in and when they return 1 day for a current state URL users notice, while when they return 5 minutes for a permanent change set that gets a lot if traffic they lost network capacity.

Post reply on HN