Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

781–790 of 1001 posts

Re: Incident with Github.com [resolved]

#781
post #29

I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.

I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding. But maybe I’m wrong?

[dead]

Re: Incident with Github.com [resolved]

#782
post #29

I had a lot of goodwill for GitHub but I think today is the tipping point. Looking at a unicorn page, I feel this lingering hope that it's transient (like it usually was in the old days) but my mind reassures me it's probably going to be a long full outage again. The hope is dead.

I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding. But maybe I’m wrong?

That excuse was valid a few months ago or back in 2025. Not anymore.

Re: Incident with Github.com [resolved]

#783

Earlier quoted context omitted.

It is interesting, how much money is being lost during this outage? My significant other was just let go from their job as a scapegoat for an organizational error: 3 layers of failure - IC, manager, director, and the IC was let go. The error caused a 7 figure loss for the company that has 10 figures of revenue per year. The manager and director may not see any consequences, though the director will probably be forced…

Is responding calmly a privilege to have, or a kindness to yourself?

don't do this

Re: Incident with Github.com [resolved]

#784
post #485

Earlier quoted context omitted.

> Luckily we already have a distributed version control system that can take over github Surprised that this wasn't just a link to Git. The biggest benefit of Git over predecessors was that it enabled decentralization. Github re-centralized the common case for the sake of convenience.

The biggest benefits of Git over predecessors were branching, 3-way merge, and history rewrites (in particular, rebase and squash). As much as Git fans wanted decentralization to be a major selling point, decentralization has never been a major selling point.

I think you understate what a big deal the decentralization was when Git came out. Back in the early 2000s, getting internet while out of the office was hard. 3G was in its infancy and expensive so you were at the mercy of finding a WiFi hotspot. Git's predecessors, such as SVN, were unusable without an internet connection to the server. Git (and Mercurial) allowed doing version control without an internet connection for the first time, which is a massive benefit for some types of development or doing work on the road. This aspect of Git definitely became less important as always-on internet connections became ubiquitous, but it's still super helpful sometimes. I can't imagine going back.

Re: Incident with Github.com [resolved]

#785
Wow they just went into "another" outage

> Update 16:59 UTC - The degradation affecting API Requests, Actions, Git Operations, Issues, Pages, Pull Requests and Webhooks has been mitigated. We are monitoring to ensure stability.

> Update 17:30 UTC - Git Operations is experiencing degraded performance. We are continuing to investigate.

> Update 17:36 UTC - Issues is experiencing degraded performance. We are continuing to investigate.

> Update 18:48 UTC - API Requests is experiencing degraded availability. We are continuing to investigate.

Re: Incident with Github.com [resolved]

#787
I think for Github to redeem itself in the community eyes, it should come out with a very detailed technical analysis (short of proprietary details) what went wrong technically and what went wrong from a management perspective. I think it is hard to believe the line about the LLM influx at this point. I think nothing short of a full postmortem break down will restore trust in Github at this point. Even then it Github has probably lost nearly all trust and ability to turn around its reputation in the short to medium term.

Re: Incident with Github.com [resolved]

#788

I think for Github to redeem itself in the community eyes, it should come out with a very detailed technical analysis (short of proprietary details) what went wrong technically and what went wrong from a management perspective. I think it is hard to believe the line about the LLM influx at this point. I think nothing short of a full postmortem break down will restore trust in Github at this point. Even then it Github…

I absolutely agree. I don't feel that I have sufficient evidence that they even really care about these reliability problems.

Re: Incident with Github.com [resolved]

#789

Does anyone get the feeling that Microsoft is trying to kill GitHub in favor of ADO?

Less conspiratorially; they may care a lot less about it than ADO, and not give it the priority/resources they would otherwise. Neglect can often do the work of malign intention.

Re: Incident with Github.com [resolved]

#790

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

ultimately, folks need to stop writing massive YAML files to define their builds. CI should schedule jobs, your build system should define them. my workflow files look like this: steps: name: Do the thing run: nix build .# at any point, if my CI is down or i need to switch or i need to run builds locally, i can do so trivially.

My CI runner just calls make targets, that can easily be run locally if needed.
Post reply on HN