Funny to see this closely match contribution graphs with effectively no downtime on weekends.
Red Squares – GitHub outages as contributions
21–30 of 175 posts
Re: Red Squares – GitHub outages as contributions
#22I don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?
See previous days articles. Agentic coding. Going from 1b annual commits to estimated 14b or more from one year to another.
Re: Red Squares – GitHub outages as contributions
#23I wonder how well this corolates with azure incidents. Especially for the US regions.
I also bet my money on Azure. Someone who allegedly worked there recently posted an article here on the numerous problems with Azure. Sadly I didn’t bookmark it.
Re: Red Squares – GitHub outages as contributions
#24I don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?
The faster you move, the more you screw up, almost no company producing software have figured out how to move fast and not screw up. It's so hard, that companies even used to boast about how much they didn't care about screwing up, as long as they moved fast.
Add in new "productivity" tools that help you move even faster, with even less regards for how much you screw up (even though the tool could be used for you to move at the same speed, but with less screw ups), and an engineering culture which boils down to "Why not?", and you get platforms run by Microsoft that are unable to achieve two nines of reliability.
Re: Red Squares – GitHub outages as contributions
#25Would be interesting to see if this correlated with their release cycles.
Well, outages seem to be distributed across all days except weekends. So this seems like people fucking around with stuff being a major factor.
Most outages are caused by changes by humans ("actors"?), very rarely are things "People just dig our stuff so much we can't keep up" but more often "We didn't think about this performance drawback when we built thing X, now it's hurting us", and of course, more outages when you try to fix those issues without fully considering the scope and impact.
Re: Red Squares – GitHub outages as contributions
#26Earlier quoted context omitted.
Well, outages seem to be distributed across all days except weekends. So this seems like people fucking around with stuff being a major factor.
Surely it just means more people working, resulting in more load, resulting in more outages?
Or even both. In any kind of continuous deployment, you'd expect outages at the point of deployment, or shortly thereafter as the unintended consequences ripple.
Then the load during the working days makes those ripples larger and into outages.
Re: Red Squares – GitHub outages as contributions
#27I don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?
ai
Re: Red Squares – GitHub outages as contributions
#28A graph I have to question is even accurate.
> Across 170 days with at least one incident · worst day Thu, Nov 20, 2025 (1.1 days)
1.1 days total how is that possible? Scrolling over that day doesn't indicate the math behind the scenes - 1.3 hours single bullet point.
Also Nov 19 has a bullet point 1.3 day outage but total is 8.1 hours
Re: Red Squares – GitHub outages as contributions
#29Another reminder that a self hosted git repository would have more uptime than GitHub and centralizing everything to GitHub was a very bad idea. [0]
Re: Red Squares – GitHub outages as contributions
#30I don't really understand why this is happening at this scale, it's not like they just became broke and can't afford a proper server... can someone explain?
Agents are shipping code faster all over the world and in some cases 24 hours a day. Additionally, some significant number of non-developers are now developers i.e. they are also shipping to github regularly.
This is not limited to just pushing code but all the bells and whistles that github added as features under the assumption of some predictable growth are now exceeding the original plans.
I suspect a lot of their existing systems have to be re-architected for unanticipated scale, and it won't happen overnight for sure.