Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident with multple GitHub services

githubstatus.com

Post reply on HN
Print view View original

121–130 of 136 posts

Re: Incident with multple GitHub services

#121

Post by oefrha »

koyote wrote: post #85
embedding-shape wrote: post #17

I've been on a somewhat binge to move a bunch of stuff to self-hosting at home. Yesterday I finally completed my self-hosted Forgejo instance at home, together with Linux, Windows (via VM) and macOS (via Mac Mini) runners/workers for CI/CD, so everything finally lives in-house (literally), instead of all source code + Actions being on GitHub but the infrastructure actually living locally. This is probably the first t…

And once you start self-hosting, you realise how slow the 'modern' web actually is. I host forgejo on a single NUC with a bunch of other stuff in Proxmox, the page loads in 6ms! Immich is not quite as fast but still a ton faster than Google photos.

That’s not a given. Self-hosted GitLab on my pretty good hardware is still slow. I just opened a very small repo with ~20 files and ~5 commits. The page spun for 5s+ before showing me the directory listing and readme. Subsequent loads are faster (~1s) but still not instant.

Re: Incident with multple GitHub services

#123

Post by eddyg »

LorenDB wrote: post #27

https://mrshu.github.io/github-statuses/ says they are down to 88.15% uptime. Even when you consider uptime of individual components, their best is 99.78%, so two nines.

The scale of growth they’re dealing with is insane.

“There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.)

GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week.”

Source: GitHub COO on April 3, 2026. https://x.com/kdaigle/status/2040164759836778878

Re: Incident with multple GitHub services

#124

Post by embedding-shape »

ryandrake wrote: post #89
skydhash wrote: post #84

Earlier quoted context omitted.

Maybe my needs are simpler. But I just made do with systemd services and apt (debian). I've also setup Incus for the occasional software testing and playing around. After using OpenBSD as a daily driver, I'm more keen with creating a native package for the OS/Distro than wrangling docker compose files.

Yea, it's always weird to see people say "I'm simplifying my life and reducing my cloud dependencies by running a homelab and self-hosting!" and then they list the dozens of alphabet soup software they're running on it that they're now relying/depending on. "Oh I run 20 VMs and containers and Docker orchestration and Nextcloud and Syncthing and Jellyfin and Plex and Forgejo and Komodo and Home Assistant and Immich an…

Depends on your requirements, I'm jealous you can get away with something so simple, I cannot, and I also have poor memory so having it described in code been most helpful, if I ssh into a server after months of not touching it I barely remember what's on it anymore.

I think the most important thing for me is that I chose when I have time to upgrade, it's no longer forced upon me, that's why I prefer to depend on myself rather than 3rd party services for things that are essential. Been so many times I've had to put other (more important) things on hold because some service somewhere decided that they're gonna change something, and to get stuff working again you need to migrate something. Just got so tired of not being in control of that schedule.

Re: Incident with multple GitHub services

#126

Post by CodesInChaos »

eddyg wrote: post #123
LorenDB wrote: post #27

https://mrshu.github.io/github-statuses/ says they are down to 88.15% uptime. Even when you consider uptime of individual components, their best is 99.78%, so two nines.

The scale of growth they’re dealing with is insane . “There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.) GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week.” Source: GitHub COO on April 3, 2026. https://x.com/kdaigle/status/2040164759836778878

What's causing that growth? AI bots spamming commits?

Re: Incident with multple GitHub services

#127

Post by rpmisms »

eddyg wrote: post #123
LorenDB wrote: post #27

https://mrshu.github.io/github-statuses/ says they are down to 88.15% uptime. Even when you consider uptime of individual components, their best is 99.78%, so two nines.

The scale of growth they’re dealing with is insane . “There were 1 billion commits in 2025. Now, it's 275 million per week, on pace for 14 billion this year if growth remains linear (spoiler: it won't.) GitHub Actions has grown from 500M minutes/week in 2023 to 1B minutes/week in 2025, and now 2.1B minutes so far this week.” Source: GitHub COO on April 3, 2026. https://x.com/kdaigle/status/2040164759836778878

That really does put it in perspective. Wow.

Re: Incident with multple GitHub services

#128

Post by olingern »

agartner wrote: post #82

There was another really bad incident today: https://www.githubstatus.com/incidents/zsg1lk7w13cf > We have resolved a regression present when using merge queue with either squash merges or rebases. If you use merge queue in this configuration, some pull requests may have been merged incorrectly between 2026-04-23 16:05-20:43 UTC. We had ~8 commits get entirely reverted on our default branch during this time. I've nev…

Same, we got an email with a PDF with the affected commits and how to fix. What an absolute clown show.

Re: Incident with multple GitHub services

#129

Post by alper »

hedayet wrote: post #87

Is Github losing any significant business from all these outages? Curious because for a long time we as an industry maintained that reliability and brand value are business critical; but seems like they are cared very little now a days. Happy to be corrected about my perception too.

From Github to Gitlab? Out of the frying pan into the fire.

It'd be nice if there was a good alternative for anybody working at scale.

Re: Incident with multple GitHub services

#130

Post by k4rli »

alper wrote: post #129
hedayet wrote: post #87

Is Github losing any significant business from all these outages? Curious because for a long time we as an industry maintained that reliability and brand value are business critical; but seems like they are cared very little now a days. Happy to be corrected about my perception too.

From Github to Gitlab? Out of the frying pan into the fire. It'd be nice if there was a good alternative for anybody working at scale.

Codeberg or self-hosted Gitea seems a perfect replacement.
Post reply on HN