Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub is aggressively caching raw.github, breaking many use cases

github.com

Post reply on HN
Print view View original

141–145 of 145 posts

Re: GitHub is aggressively caching raw.github, breaking many use cases

#141

Post by joshspankit »

remram wrote: post #126
ilyt wrote: post #117

Earlier quoted context omitted.

Tag it with etag = hash, done. The client side isn't hard part. The server side would require pushing any invalidation to (I imagine) whole tree of caches, which isn't exactly that hard if you plan for it from the start and have some way of upstream telling downstream file changes, but, well, they probably don't as I'd imagine they didn't expected people to pin their infrastructure to some binary blob on github that…

Etag doesn't let you "fully prevent traffic from a client" (GP's exact words). They'll still send a request to which you need to reply with a 304 after checking the resource.

They wouldn’t even need to check the resource. If the hash (which you get for free since it’s part of the git commit) was the etag then they can quickly reply to that request from an edge.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#142

Post by remram »

joshspankit wrote: post #141
remram wrote: post #126

Earlier quoted context omitted.

Etag doesn't let you "fully prevent traffic from a client" (GP's exact words). They'll still send a request to which you need to reply with a 304 after checking the resource.

They wouldn’t even need to check the resource. If the hash (which you get for free since it’s part of the git commit) was the etag then they can quickly reply to that request from an edge.

You get it "for free" if you load up the repo and check some files. That's not free at all.

In fact, loading a file by name from a Git repo is rather expensive, and is definitely not the way their CDN should be keeping things in cache (gotta load ref, uncompress and parse commit object, uncompress and read tree object(s), just to get the blob's hash. Every one of those objects is both deflated and delta-encoded.)

Re: GitHub is aggressively caching raw.github, breaking many use cases

#143

Post by joshspankit »

remram wrote: post #142
joshspankit wrote: post #141

Earlier quoted context omitted.

They wouldn’t even need to check the resource. If the hash (which you get for free since it’s part of the git commit) was the etag then they can quickly reply to that request from an edge.

You get it "for free" if you load up the repo and check some files. That's not free at all. In fact, loading a file by name from a Git repo is rather expensive, and is definitely not the way their CDN should be keeping things in cache (gotta load ref, uncompress and parse commit object, uncompress and read tree object(s), just to get the blob's hash. Every one of those objects is both deflated and delta-encoded.)

Nono: don't do it at lookup, set the etag when the repo is pushed.

I'm open to the idea that it's less computation to simply hash the files instead of deflate and decide, but my point was the hash is already calculated on the client when the changed file is added to the repo.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#144

Post by smashah »

pjc50 wrote: post #93
smashah wrote: post #82

I hit this problem for my project. On every launch of my lib, it would grab a patches file from the main github repo. I was seeing people having to wait 5 minutes+ and multiple restarts to get the latest pushed file. The solution was to run a custom BunnyCDN instance where I can easily invalidate the caches via a basic API request which happens on a github push of that file. I was surprised to see that sometimes that…

> On every launch of my lib, it would grab a patches file from the main github repo. Was this a project that interacts with github otherwise? Because this seems very .. brittle.

Yes of course it is wholly based in GitHub.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#145

Post by CommitSyn »

onlypositive wrote: post #55
CommitSyn wrote: post #42

Earlier quoted context omitted.

> That's why the two things are together in that maxim. I think you missed the joke.

Did they?

> There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

That's three things with an off-by-one error.

Post reply on HN