Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Another GitHub outage in the same day

githubstatus.com

Post reply on HN
Print view View original

261–270 of 298 posts

Re: Another GitHub outage in the same day

#261

Post by falloutx »

reactordev wrote: post #208
falloutx wrote: post #185

Earlier quoted context omitted.

No its not. 121M repos added on github in 2025, and overall they have 630 million now. There is probably at best 2x increased in output (mostly trash output), but no where near 100x https://github.blog/news-insights/octoverse/octoverse-a-new-...

Published in Oct 2025... I think your estimate is off. Note the hockey stick growth in the graph they showed in Oct. Here we are in February. It's gotten way worse now with additional Claude's, Claw's, Ralph's, and such. It may not be 100x as was told to me but it's definitely putting the strain on the entire org.

> It may not be 100x as was told to me but it's definitely putting the strain on the entire org.

But thats not even the top 5 strain on github, their main issue is the forced adoption of Azure. I can guarantee you that about 99% of repos are still cold, as in very few pulls and no pushes and that hasn't changed in 3 months. Storage itself doesn't add that much strain on the system if the data is accessed rarely.

Re: Another GitHub outage in the same day

#262

Post by [deleted] »

amarant wrote: post #216
Andrex wrote: post #207

Earlier quoted context omitted.

Very weird it happened twice! But that's a kind of a cool factoid to tell people haha. Even with devs and publishers that don't die or are killed, they still lay hundreds off when a game is done. Then the studio limps along in pre-production mode on their next game for 4-5 years it seems like... Maybe the only job stability in the industry is with indies, and... Nintendo?

I'd add the hugely successful studios to that list. Even after ms acquisition, to the best of my knowledge neither of the 2 studios I worked at had any layoffs. But they boast the most sold video game in the history of videogames (Tetris a close-ish second), and most downloaded free mobile game, respectively. Each have player bases larger than the population of the country they're from! Here's to hoping ms is hesitan…

[deleted]

Re: Another GitHub outage in the same day

#263

Post by close04 »

amarant wrote: post #216
Andrex wrote: post #207

Earlier quoted context omitted.

Very weird it happened twice! But that's a kind of a cool factoid to tell people haha. Even with devs and publishers that don't die or are killed, they still lay hundreds off when a game is done. Then the studio limps along in pre-production mode on their next game for 4-5 years it seems like... Maybe the only job stability in the industry is with indies, and... Nintendo?

I'd add the hugely successful studios to that list. Even after ms acquisition, to the best of my knowledge neither of the 2 studios I worked at had any layoffs. But they boast the most sold video game in the history of videogames (Tetris a close-ish second), and most downloaded free mobile game, respectively. Each have player bases larger than the population of the country they're from! Here's to hoping ms is hesitan…

> But they boast the most sold video game in the history of videogames (Tetris a close-ish second), and most downloaded free mobile game, respectively.

Just out of curiosity, I guessed Minecraft which tracks, and Subway Surfers respectively, rather than Candy Crush Saga. Is CCS actually the most downloaded free mobile game ever?

Re: Another GitHub outage in the same day

#264

Post by tibbar »

akshayshah wrote: post #245
tibbar wrote: post #71

Earlier quoted context omitted.

Github used to publish some pretty interesting postmortems. Maybe they still do. IIRC that they were struggling with scaling their SQL db and were starting to hit the limits. It's a tough position to be in because you have to either to a massive migration to a data layer with much different semantics, or you have to keep desperately squeezing performance and skirting on the edge of outages with a DB that wasn't reall…

I’d be surprised by this: GitHub pretty famously used Vitess, and I’d be surprised if each shard were too big for modern hardware. Based on previous reporting [0], they’re running out of space in the main data center and new management is determined to move to Azure in a hurry. I’d bet that these outages are a combination of a worsening capacity crunch in the old data center and…well, Azure. [0]: https://thenewstack.…

Here’s a post from 2021 about the migration! [0]

I guess 2021 is a long time ago now. How did that happen…

[0] https://github.blog/engineering/infrastructure/partitioning-...

Re: Another GitHub outage in the same day

#265

Post by alextingle »

TacticalCoder wrote: post #133
heliumtera wrote: post #98

Remember the other day when a bunch of yous were making fun of zig moving away from GitHub? Now suddenly you all say this is not the future you wanted. Everyday you opt in to get wrecked by Microsoft. You all do realize you all could, for a change, learn something and never again touch anything Microsoft related? Fool me once...

> You all do realize you all could, for a change, learn something and never again touch anything Microsoft related? I learned that lesson in the 90s and became an "ABM" (Anything But Microsoft). People sadly shall never learn: Windows 12 is going to come out and shall suck more than any previous version of Windows except Windows 11, so they'll see it as progress. Then Windows 13 is going to be an abysmal piece of cra…

Windows XP was a disaster. Resistance to adopting it was industry wide, and lasted years. It was only with Service Pack 2 (which was an enormous rewrite of basically the whole system) that they started to turn the ship around.

Re: Another GitHub outage in the same day

#266

Post by tosti »

palata wrote: post #256
tosti wrote: post #241

Earlier quoted context omitted.

IDK what you're smoking, but he was born in Germany so can't be the next potus.

> IDK what you're smoking Or maybe it's a problem with your English? (Note: I'm being offensive just because you are :-) ). > when Trump croaks Thiel will be the president in all but name. This means that he will have the power, but not the title.

Ok I get it, sorry bout that and thanks for the laugh!

Re: Another GitHub outage in the same day

#268

Post by vampiregrey »

neilv wrote: post #76
vampiregrey wrote: post #2

At this point, GitHub outages feel closer to cloud provider outages than a SaaS blip. Curious how many people here still run self-hosted Git (GitLab / Gitea) vs fully outsourcing version control.

Yay for GitLab and Forgejo/Gitea. My previous two startups used GitLab successfully. The smaller startup used paid-tier hosted by gitlab.com. The bigger startup (with strategic cutting-edge IP, and multinational security sensitivity) used the expensive on-prem enterprise GitLab. (The latter startup, I spent some principal engineer political capital to move us to GitLab, after our software team was crippled by the Mic…

I just checked out Forgejo. I think i start with it, looks clean and lightweight. For my homelab I don’t have very large requirements. Might be a good starting point for me.

Re: Another GitHub outage in the same day

#269

Post by adamcharnock »

dizhn wrote: post #193
adamcharnock wrote: post #82

Earlier quoted context omitted.

We can run a Forgejo instance for you with Firecracker VM runners on bare metal. We can also support it and provide an SLA. We're running it internally and it is very solid. We're running the runners on bare metal, with a whole lot of large CI/CD jobs (mostly Rust compilation). The down side is that the starting price is kinda high, so the math probably only works out if you also have a number of other workloads to r…

You've got any docs for firecracker as forgejo runners?

Ping me an email, adam@ domain.

If you're interested I'll see about getting the PR created sooner rather than later.

Re: Another GitHub outage in the same day

#270

Post by disgruntledphd2 »

close04 wrote: post #263
amarant wrote: post #216

Earlier quoted context omitted.

I'd add the hugely successful studios to that list. Even after ms acquisition, to the best of my knowledge neither of the 2 studios I worked at had any layoffs. But they boast the most sold video game in the history of videogames (Tetris a close-ish second), and most downloaded free mobile game, respectively. Each have player bases larger than the population of the country they're from! Here's to hoping ms is hesitan…

> But they boast the most sold video game in the history of videogames (Tetris a close-ish second), and most downloaded free mobile game, respectively. Just out of curiosity, I guessed Minecraft which tracks, and Subway Surfers respectively, rather than Candy Crush Saga. Is CCS actually the most downloaded free mobile game ever?

Call of Duty and Candy Crush I would say (if you count Farm Heroes as CCS because it's just a reskin for APAC then it's probably not even close).
Post reply on HN