Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: GitHub employees what's going on? Why?

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 280 posts

Re: Ask HN: GitHub employees what's going on? Why?

#131

Post by thex10 »

0xbadcafebee wrote: post #48
tristanj wrote: post #8

Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating. The site is struggling to keep up. Github's COO confirms it here: https://x.com/kdaigle/status/2040164759836778878 Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth rema…

If that's the superficial cause, then the root cause is a lack of limits. Any production system in the world will go down from too much load if you don't limit the load. You have to do performance testing to determine what the system's peak utilization is, and put limits in place to prevent exceeding that limit. There are multiple kinds of limits you need. First you need the raw i/o limits on things like networking,…

I really enjoyed this read, as I am starting to grapple with some of this now. Do you have any resources for learning more about this stuff and these tradeoffs?

Re: Ask HN: GitHub employees what's going on? Why?

#132

Post by nrr »

fallingbananna wrote: post #113
herpdyderp wrote: post #2

non-helpful answer: the "Microsoft Acquires GitHub" line in this graph answers all questions https://damrnelson.github.io/github-historical-uptime/

To be fair, before the aquisition, GitHub was super stable, but also it felt like they rarely changed or added anything. After Microsoft acquired them, they started rolling out new features at an incredible pace. And if you look at the breakdown of the downtime, most of the downtime is in features that didn't exist before the acquisition. Though not all of the downtime is in new features. The core is also less stable…

My (actuarial) read on this is that pre-Microsoft GitHub acknowledged that the complexity inherent in their technology stack imparted a lot of change risk, and that probably informed their decisions around shipping new features.

My (not-so-actuarial 1990's-experience-colored) feeling is that this is Microsoft doing what Microsoft does best, namely including a lot of seemingly compelling features in a product to get people hooked and to keep competitors in a position of trying to keep up.

Re: Ask HN: GitHub employees what's going on? Why?

#133

Post by geodel »

roughly wrote: post #111
srmatto wrote: post #88

Earlier quoted context omitted.

Infra is a cost center for a business unless your a PaaS and even then maybe still. Cost centers are usually ferreted out for savings, not investment.

Infra is a cost center right up until its absence begins to undercut your value proposition to customers, and then it becomes an investment. GitHub seems well on the wrong side of that equation right now.

In general true. But most of paying customers, including enterprise have made a bet that they cut down expense and resources on operational experience, system management knowledge to minimum because cloud .

So even if these cloud/SAAS solutions perform poorly it is not like customers have wherewithal to create alternate solutions.

Re: Ask HN: GitHub employees what's going on? Why?

#134

Post by andyjohnson0 »

Github used to be built on mysql / redis / ruby on rails / C / shell, running on dedicated hardware. Microsoft left it like that after they acquired the company.

Eventually, though, they decided to migrate the whole thing to Azure. And they were far enough through that to be basically committed... when AI coding started hitting them with much higher workloads.

I personally think the reliability problems are more to do with the reliability of the Azure migration. But both factors are likely relevant.

Re: Ask HN: GitHub employees what's going on? Why?

#135

Post by layer8 »

tristanj wrote: post #8

Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating. The site is struggling to keep up. Github's COO confirms it here: https://x.com/kdaigle/status/2040164759836778878 Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth rema…

Turns out that the cloud doesn’t scale that easily by itself after all?

Re: Ask HN: GitHub employees what's going on? Why?

#136

Post by etc-hosts »

logicchains wrote: post #31
tekno45 wrote: post #22

Earlier quoted context omitted.

OpenAI and Claude don't accept commits and host PRs? How are they the same?

They're undergoing the same kind of load. Almost every AI commit that's putting pressure on GitHub's servers was written on OpenAI and Anthropic's servers.

Every response from GitHub must be deterministic. OpenAI and Anthropic do not have this problem.

Re: Ask HN: GitHub employees what's going on? Why?

#137

Post by aleph_minus_one »

fooey wrote: post #122
maccard wrote: post #114

Earlier quoted context omitted.

GitHub’s reliability issues started earlier than the boom in agentic coding they talk about. I have comments on here from 2025 talking about massive outages with actions and issues.

When you're the uncontested market leader, you don't budget operations to be perfectly reliable, you budget to not be too unreliable. All those 9's are expensive after all. The AI boom caught them napping though, and they're slipping well into the too unreliable category

> The AI boom caught them napping though

Microsoft, who invested eyewatering sums into AI, even near to betting the whole company on an AI boom, is surprised about the AI boom?

Re: Ask HN: GitHub employees what's going on? Why?

#138

Post by Pseudocrat »

qrush wrote: post #60

I got an "it is unacceptable" from their CPO on 8/7, and that they are "working around the clock on it". https://x.com/mariorod1/status/2085800861469495465 I really think something deeper is going wrong there, and they're not being honest with their paying customers (and enterprises) about it.

I mean they may be working around the clock, but after 5 or 6 layoffs in 3 years… half their engineers are probably still learning the code base.

Re: Ask HN: GitHub employees what's going on? Why?

#139

Post by aleph_minus_one »

layer8 wrote: post #135
tristanj wrote: post #8

Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating. The site is struggling to keep up. Github's COO confirms it here: https://x.com/kdaigle/status/2040164759836778878 Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth rema…

Turns out that the cloud doesn’t scale that easily by itself after all?

> Turns out that the cloud doesn’t scale that easily by itself after all?

The cloud does scale that easily, the software doesn't. :-)

Re: Ask HN: GitHub employees what's going on? Why?

#140

Post by lluisantoni »

tristanj wrote: post #8

Github is struggling because AI-boosted coding increased the number of commits 14x in the past year, and the pace is still accelerating. The site is struggling to keep up. Github's COO confirms it here: https://x.com/kdaigle/status/2040164759836778878 Platform activity is surging. There were 1 billion commits in 2025. As of three months ago, it was 275 million per week, on pace for 14 billion this year if growth rema…

That is a good point. And this issue looks analogous to what is happening in the research publication world (arxiv, peer-review, etc). In particular, I wonder whether code pushed is of lesser quality, CI pipelines break more often too, and this saturates github servers.
Post reply on HN