Earlier quoted context omitted.
This absolutely can happen in large systems. If some part of the system is at capacity, then slightly increasing the load can cause it to fall behind and start accumulating a backlog. These backlogs can cause clients to make more retries, exacerbating the problem. Potentially further cascading through the system.
I believe their point is that "system is at capacity" is something they ought to start fixing before the capacity is exceeded
The August 17 outage
51–60 of 804 posts
Re: The August 17 outage
#52"Since April, monthly commits have grown from 1.4 billion to 2.9 billion. " Wow, that is some incredible growth in a really short time.
It really is. I know I've gone from tens a month to thousands a month. They have to be projecting >100B/month in the next year or two.
Re: The August 17 outage
#53Earlier quoted context omitted.
They had $1b revenue in 2023 and now probably more than $2b in revenue... do you have cost figures showing what their expenses are?
> We have since added more than 3 million CPU cores, 120 petabytes of high-speed storage, and significant network capacity. We installed as much hardware as available power allowed in our existing data centers while accelerating our migration to Azure. That can't be cheap.
Re: The August 17 outage
#54Sorry to suggest this but if they charged everyone say $1/mo. it would absolutely help the massive surge from AI coding they seem to have had. I don't like paying for free stuff but gh certainly worth it.
Might force people to review their slop before pushing it.
Re: The August 17 outage
#55Re: The August 17 outage
#56Has to be one of the most vague outage summaries of the year
It is specific and technical.
Re: The August 17 outage
#57Has to be one of the most vague outage summaries of the year
Re: The August 17 outage
#58Sorry to suggest this but if they charged everyone say $1/mo. it would absolutely help the massive surge from AI coding they seem to have had. I don't like paying for free stuff but gh certainly worth it.
Re: The August 17 outage
#59We need to have a package of FLOSsoftware that you could run on the cloud of your choice that offers most of what GitHub does (niceties on top of Git) without the centralization. GitLab was close last I remember but there was some sort of enterprise tier when I tried hosting stuff on a local server years ago. I want true FLOSS, not another SaaS equivalent of the coke dealer giving clients the good uncut stuff when th…
https://forgejo.org/ promises to be this, have only lightly used it on https://codeberg.org/ but it seems nice?
I self-host it and its pretty easy to have uptime higher than github when you have 3 users total.
Absolutely do not recommend using GitLab.
Re: The August 17 outage
#60Github down, no hard drives available, no memory available, thanks AI! Seems like we are headed for Tech Gridlock.
This stuff is good! This is what a booming economy looks like. There are people out there competing with you for resources because they have cool ideas they want to implement.