Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub is aggressively caching raw.github, breaking many use cases

github.com

Post reply on HN
Print view View original

91–100 of 145 posts

Re: GitHub is aggressively caching raw.github, breaking many use cases

#91

Post by nkozyra »

the_mitsuhiko wrote: post #17
InvaderFizz wrote: post #9

Reading the comment threads on GitHub, some files get a TTL of 300, some get a TTL of 86400. The "why" is certainly an interesting question.

> Reading the comment threads on GitHub, some files get a TTL of 300, some get a TTL of 86400. The "why" is certainly an interesting question. I would guess GitHub is slowly cutting down on people (ab)using it for free file hosting. Files that are hit a lot probably get significantly longer cache timeouts.

I don't think this would impact this too much for free file hosting unless there's a use case where people are rewriting the same file very often.

With sounds more like standard git use case than file hosting abuse

Re: GitHub is aggressively caching raw.github, breaking many use cases

#92

Post by sourcecodeplz »

_cenw wrote: post #32
kzrdude wrote: post #28

What level of service should we expect from github? Many of its services are free, can we ask for more when not paying?

raw.github has seen some extensive (ab)use for bandwidth heavy applications and/or pointing a few thousand clients at it concurrently (including entire IoT fleets), without ever being communicated as your globally consistent free CDN. I'm actually more amazed GitHub will give you a response in the 2xx range at all for these use-cases.

This. I know I've used it as a static file hosting service before. The cache problem was real then too. Can't complain though.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#93

Post by pjc50 »

smashah wrote: post #82

I hit this problem for my project. On every launch of my lib, it would grab a patches file from the main github repo. I was seeing people having to wait 5 minutes+ and multiple restarts to get the latest pushed file. The solution was to run a custom BunnyCDN instance where I can easily invalidate the caches via a basic API request which happens on a github push of that file. I was surprised to see that sometimes that…

> On every launch of my lib, it would grab a patches file from the main github repo.

Was this a project that interacts with github otherwise? Because this seems very .. brittle.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#94

Post by roundandround »

joshspankit wrote: post #86
roundandround wrote: post #80

Earlier quoted context omitted.

That is too simple for the feature they are using. The client itself has its own cache and the only way to fully prevent traffic from a client is to tell it content it caches will remain valid for some amount of time into the future. For URLs that return the latest entry there is no valid amount of time known in advance by GitHub unless they want to introduce mandatory publication delays. For URLs of specific change…

> The client itself has its own cache Let’s leave this aside since it both has known client-side mitigations, and is not the cause of the issue that was posted.

Once you are using one style of caching its usually a mistake to introduce another even if the style is marginal. They very clearly have an issue with max age with clients/CDNs in the second half of the thread and probably have similar problems on internal transparent proxies, etc.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#95

Post by airstrike »

TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

There are actually only two hard problems in computer science:

0) Cache invalidation

1) Naming things

5) Asynchronous callbacks

2) Off-by-one errors

3) Scope creep

6) Bounds checking

Re: GitHub is aggressively caching raw.github, breaking many use cases

#96

Post by _cenw »

rgbrenner wrote: post #56
_cenw wrote: post #38

Earlier quoted context omitted.

That's a terrible idea. Source: Implemented global TTL in our own caching DNS in front of kube-dns (which is horrible if you, among other things, have node containers with no DNS caching; i still have a pcap with 20000+ queries for A in s3.amazonaws.com in a 0.2s span) before coredns was a thing. The CPU spikes were huge, but remained hidden for a long time due to metrics resolution. But eventually it got bad enough…

There are circumstances where that’s the right strategy. For example, GitHub may be using it to ensure two requests for two different files in a repo receive the same version of the repo. Not saying they’re doing that.. just explaining the cache strategy. An explanation isn’t a recommendation for you to go out and apply it to everything. Source: founded and operated a cdn for 5 years of my life.

As long as you key the expiration to something, be that the client IP or the repository, you're probably fine. Having your entire customer base do their cache expiration request within the same small time span is not amazing though.

You can already get consistent views of raw.github by looking up the HEAD commit and requesting your files from that commit directly, though.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#98

Post by raverbashing »

airstrike wrote: post #95
TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

There are actually only two hard problems in computer science: 0) Cache invalidation 1) Naming things 5) Asynchronous callbacks 2) Off-by-one errors 3) Scope creep 6) Bounds checking

You forgot the 'Segmentation Fault' at the last line

Re: GitHub is aggressively caching raw.github, breaking many use cases

#99

Post by Retric »

thih9 wrote: post #97
TobTobXX wrote: post #3

There are two hard problems in IT: cache invalidation, naming things and off-by-one errors.

You multithreading have forgotten.

Cache invalidation is a large class of multi threading problems, that’s just as much an issue between threads as between servers.

Re: GitHub is aggressively caching raw.github, breaking many use cases

#100

Post by ignoramous »

stingraycharles wrote: post #85
roundandround wrote: post #80

Earlier quoted context omitted.

That is too simple for the feature they are using. The client itself has its own cache and the only way to fully prevent traffic from a client is to tell it content it caches will remain valid for some amount of time into the future. For URLs that return the latest entry there is no valid amount of time known in advance by GitHub unless they want to introduce mandatory publication delays. For URLs of specific change…

Isn’t this where stuff like ETAGs are supposed to help? Not completely solve, but at least help reduce the problem a bit more?

Yes, but cache invalidation is a hard problem, so most services side-step etags (+ if-none-match), only to hit caches elsewhere.
Post reply on HN