Azure infra rock solid as always.
GitHub is down again
201–210 of 425 posts
Re: GitHub is down again
#202The biggest thing tying my team to GitHub right now is that we use Graphite to manage stacked diffs, and as far as I can tell, Graphite doesn't support anything but GitHub. What other tools are people using for stacked-diff workflows (especially code review)? Gerrit is the other option I'm aware of but it seems like it might require significant work to administer.
I use git town. Fits my brain a lot better.
Re: GitHub is down again
#203Earlier quoted context omitted.
They're in the process of moving from "legacy" infra to Azure, so there's a ton of churn happening behind the scenes. That's probably why things keep exploding.
I don't know jack about shit here, but genuinely: why migrate a live production system piecewise? Wouldn't it be far more sane to start building a shadow copy on Azure and let that blow up in isolation while real users keep using the real service on """legacy""" systems that still work?
A few reasons:
1. Stateful systems (databases, message brokers) are hard to switch back-and-forth; you often want to migrate each one as few times as possible.
2. If something goes sideways -- especially performance-wise -- it can be hard to tell the reason if everything changed.
3. It takes a long time (months/years) to complete the migration. By doing it incrementally, you can reap the advantages of the new infra, and avoid maintaining two things.
---
All that said, GitHub is doing something wrong.
Re: GitHub is down again
#204Earlier quoted context omitted.
Migrations of Actions and Copilot to Azure completed in 2024. Pages and Packages completed in 2025. Core platform and databases began in October 2025 and are in progress, with traffic split between the legacy Github data center and Azure.
That's probably partly why things have got increasingly flaky - until they finish there'll be constant background cognitive load and surface area for bugs from the fact everything (especially the data) is half-migrated
You'd think so, and we don't know about today's incident yet, but recent Github incidents have been attributed specifically to Azure, and Azure itself has had a lot of downtime recently that lasts for many hours.
Re: GitHub is down again
#205What are good alternatives to GitHub for private repos + actions? I'm considering moving my company off of it because of reliablity issues.
At my last job I ran a GitLab instance on a tiny AWS server and ran workers on old desktop PCs in the corner of the office. It's pretty nice if you don't mind it being some of the heaviest software you've ever seen. I also tried gitea, but uninstalled it when I encountered nonsense restrictions with the rationale "that's how GitHub does it". It was okay, pretty lightweight, but locking out features purely because "th…
One thing that always bothered me about gitea is they wouldn't even dog food for a long time. GitLab has been developing on GitLab since forever, basically.
Re: GitHub is down again
#206Earlier quoted context omitted.
They're in the process of moving from "legacy" infra to Azure, so there's a ton of churn happening behind the scenes. That's probably why things keep exploding.
I don't know jack about shit here, but genuinely: why migrate a live production system piecewise? Wouldn't it be far more sane to start building a shadow copy on Azure and let that blow up in isolation while real users keep using the real service on """legacy""" systems that still work?
Because it's significantly harder to isolate problems and you'll end up in this loop
* Deploy everything * It explodes * Rollback everything * Spend two weeks finding problem in one system and then fix it * Deploy everything * It explodes * Rollback everything * Spend two weeks finding a new problem that was created while you were fixing the last problem * Repeat ad nauseum
Migrating iteratively gives you a foundation to build upon with each component
Re: GitHub is down again
#207You can literally watch GitHub explode bit by bit. Take a look at the GitHub Status History; it's hilarious: https://www.githubstatus.com/history .
You know what I think would reverse the trend? More vibe coding!
All the cool kids move fast and break things. Why not the same for core infrastructure providers? Let's replace our engineers with markdown files named after them.
Re: GitHub is down again
#208What are good alternatives to GitHub for private repos + actions? I'm considering moving my company off of it because of reliablity issues.
We self-host Gitlab at work and it's amazing. CI/CD is great and it has never once gone down.
Re: GitHub is down again
#209GitHub has had customer visible incidents large enough to warrant status page updates almost every day this year ( https://www.githubstatus.com/history ). This should not be normal for any service, even at GitHub's size. There's a joke that your workday usually stops around 4pm, because that's when GitHub Actions goes down every day. I wish someone inside the house cared to comment why the services barely stay up and…
It's 100% because the number of operations happening on Github has likely 100x'd since the introduction of coding agents. They built Github for one kind of scale, and the problem is that they've all of a sudden found themselves with a new kind of scale.
That doesn't normally happen to platforms of this size.
Re: GitHub is down again
#210Earlier quoted context omitted.
Fucking REALLY?!
Migrations of Actions and Copilot to Azure completed in 2024. Pages and Packages completed in 2025. Core platform and databases began in October 2025 and are in progress, with traffic split between the legacy Github data center and Azure.
This has those Hotmail migration vibes off the early 2000s.