Live data from Hacker News

GitHub is aggressively caching raw.github, breaking many use cases

github.com

81–90 of 145 posts

Re: GitHub is aggressively caching raw.github, breaking many use cases

#81

Earlier quoted context omitted.

Github did layoffs, yes. We'll see how much longer the site can stay up. This is a death rattle.

GitHub is among many tech companies currently shedding fat from hiring during the pandemic boom. The morality of that is another discussion, but I'd hardly call this a death rattle.

Let's see you apply that same logic in this thread:

https://news.ycombinator.com/item?id=34715890

Re: GitHub is aggressively caching raw.github, breaking many use cases

#82
I hit this problem for my project. On every launch of my lib, it would grab a patches file from the main github repo. I was seeing people having to wait 5 minutes+ and multiple restarts to get the latest pushed file. The solution was to run a custom BunnyCDN instance where I can easily invalidate the caches via a basic API request which happens on a github push of that file.

I was surprised to see that sometimes that file was being requested millions of times a day due to attempts to loading it too early. With some optimizations I was able to greatly reduce that by 99%.

Having seen all those requests, I understand why github aggressively caches these files.

I agree that repo owners should be allowed to invalidate those caches upon an API request - or it should happen automatically on every change.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#85

Earlier quoted context omitted.

Github raw seems like the simplest system for solving cache invalidation: invalidate the cache of a changed file when it’s pushed. They have access to both GitHub and the raw service. I know there are usually all sorts of layers between that make interconnectivity logistically complicated, but am I wrong that at the top-level it’s that simple?

That is too simple for the feature they are using. The client itself has its own cache and the only way to fully prevent traffic from a client is to tell it content it caches will remain valid for some amount of time into the future. For URLs that return the latest entry there is no valid amount of time known in advance by GitHub unless they want to introduce mandatory publication delays. For URLs of specific change…

Isn’t this where stuff like ETAGs are supposed to help? Not completely solve, but at least help reduce the problem a bit more?

Re: GitHub is aggressively caching raw.github, breaking many use cases

#86

Earlier quoted context omitted.

Github raw seems like the simplest system for solving cache invalidation: invalidate the cache of a changed file when it’s pushed. They have access to both GitHub and the raw service. I know there are usually all sorts of layers between that make interconnectivity logistically complicated, but am I wrong that at the top-level it’s that simple?

That is too simple for the feature they are using. The client itself has its own cache and the only way to fully prevent traffic from a client is to tell it content it caches will remain valid for some amount of time into the future. For URLs that return the latest entry there is no valid amount of time known in advance by GitHub unless they want to introduce mandatory publication delays. For URLs of specific change…

> The client itself has its own cache

Let’s leave this aside since it both has known client-side mitigations, and is not the cause of the issue that was posted.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#87
post #84

I don't think raw was supposed to be used as your CDN

It doesn't matter what it was supposed to be used for. It's bad data in any context, even ones where you can't think of a way to blame the victim for holding it wrong.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#88
post #44

Earlier quoted context omitted.

How do you see that? At leat in my mobile browser nothing jumps into my eye. This user has contributions to @github, but very few. Edit: Found it. Need to click on one of the achievements. Then the layout changes and it appears in the lower left corner under Organizations.

It doesn't appear to be visible on mobile, only desktop.

Ah, good catch. We'll try to see if we can fix that in the future so the mobile site shows our staff badges.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#89

Earlier quoted context omitted.

Ridiculous is expecting a service you're not paying for to serve files in a way that fits your use case when you've never entered into a contract that guarantees the behaviour you're relying on. You can use Git just fine without GitHub.

So expecting that a raw file isn't outdated is a special usecase now? And I don't see where you see that I'm not a paying user? This issue affects every repository, including ones where the user pays.

Welp, is it in your SLA?
Post reply on HN