Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub appears to be struggling with measly three nines availability

theregister.com

Post reply on HN
Print view View original

231–240 of 255 posts

Re: GitHub appears to be struggling with measly three nines availability

#231

Post by noir_lord »

llimllib wrote: post #79
a-french-anon wrote: post #43

Earlier quoted context omitted.

If you want more ammo for your ranting (no offense meant, I also rant): an issue as massive as https://github.com/orgs/community/discussions/142308 lingering for years should do the trick.

It really feels like Firefox is not a supported browser on GitHub, I hit this and also find that much of the time the commit message is not correctly pulled from the PR description when that setting is enabled

Given I get "Too Many Requests" if I try and search for a repo from FF logged out and have done for at least a year, I'd say so.

Works fine on Chrome/Edge.

Re: GitHub appears to be struggling with measly three nines availability

#232

Post by lijok »

teaearlgraycold wrote: post #222
franktankbank wrote: post #167

Earlier quoted context omitted.

Doesn't a single compromised action in the chain cause the whole to be fucked? Pinning the top level doesn't prevent any spread.

Might want to vendor everything?

That’s the way to go indeed. We’ve done it, not difficult, just a bit of gruntwork to keep them updated when needed

Re: GitHub appears to be struggling with measly three nines availability

#233

Post by collingreen »

shimman wrote: post #147
827a wrote: post #113

I don't want to give too much credit to Github, because their uptime is truly horrendous and they need to fix it. But: I've felt like its a little unfair to judge the uptime of company platforms like this; by saying "if any feature at all is down, its all down" and then translating that into 9s for the platform. I never use Github Copilot; it does go down a lot, if their status page is to be believed; I don't really…

This company is part of the portfolio of a $trillion+ transnational corporation. The idea that we can't judge them, when they clearly have more resources than 99% of other companies on this planet, doesn't hold up to any scrutiny. Why defend a company that clearly doesn't care about its customers and see them as a money spigot to suck dry?

You had an opportunity to use three or more nines in your comparison to other companies and you just left it on the table and went with two nines. ;p

Re: GitHub appears to be struggling with measly three nines availability

#234

Post by siren2026 »

ARandomerDude wrote: post #134
lucideer wrote: post #117

Earlier quoted context omitted.

You're right that labelling any outage as "Github is down" is an overgeneralisation, & we should focus on bottlenecks that impact teams in a time sensitive matter, but that isn't the case here. Their most stable service (API) has only two 9s (99.69%). They're not even struggling to get their average to three 9s, they're struggling to get ANY service to three 9s. They're struggling to get many services to two 9s. Copi…

I love multiple 9s as much as the next guy but that's only 27 hours per year of downtime. For a mostly free (for me) service, I'm thankful.

The issue is also that those 27 hours don't happen at once, They happen in small chunks of a couple minutes which makes it happen almost everyday and has a ton of downstream build and retry issues. The resulting downtime is probably 2 orders of magnitudes higher at least.

Re: GitHub appears to be struggling with measly three nines availability

#235

Post by SahAssar »

graphememes wrote: post #209
alfalfasprout wrote: post #206

Earlier quoted context omitted.

It's not though... it's sitting a hair above 90%

My comment is that I expect it to be more frequent and less given the scale and usage. Quite honestly, we have the most insane expectations about uptime and availability in my opinion. I'm glad that we can meet them. But still, insane.

If something is unfixably broken 1 out of 10 times I use it I will just consider it broken. 1 out of 100 is unreliable but usable.

That goes for most things physical and digital.

Re: GitHub appears to be struggling with measly three nines availability

#236

Post by MeetingsBrowser »

graphememes wrote: post #220
MeetingsBrowser wrote: post #217

Earlier quoted context omitted.

At the risk of going recursive, how do you define not bad? Could be worse?

Yes, and honestly, for it's usage, similar feelings for AWS, not bad. Although, I guess I could ask, are you looking for a rubrick that I follow? Not trying to be inflammatory, just trying to understand what exactly you're looking for. I am mostly going off of overall usage vs downtime.

How often can a service be down and still be considered to have incredible uptime.

Personally, I would define incredible uptime as roughly 5 minutes down per month, or around 99.99% uptime.

GitHub is currently sitting at about 70 hours of downtime per month, on average across all services.

Re: GitHub appears to be struggling with measly three nines availability

#237

Post by _heimdall »

MoreQARespect wrote: post #35
_heimdall wrote: post #17

Earlier quoted context omitted.

He was never truly independent though. The org structure was such that the GitHub CEO reported up through a Microsoft VP and Satya. He was never really a CEO after the acquisition, it was in name only. Also of note is that the Microsoft org chart always showed GitHub in that structure while the org chart available to GitHub stopped at their CEO. Its not that they were finally rolled into Microsoft's org chart so much…

I never said he was "truly independent" nor meant to imply it. Nonetheless it looks like he was both willing and able to push back on a good deal of the AI stupidity raining down from above and then he was removed and then, well, this...

You said he was independent, I didn't include "truly" intending to make a distinction there. How could one be an independent CEO while reporting to a VP who reports to another CEO?

I don't personally know him and wouldn't begin to assume on what, or how, he pushed back. Though Microsoft had AI in the GitHub org well before the leadership change - the AI leader now in charge of GitHub was previously in charge of an AI org that was moved over in the org chart to dotted line report as embedded employees, or whatever they would have been called.

Re: GitHub appears to be struggling with measly three nines availability

#238

Post by fragmede »

phyzome wrote: post #225
BenjiWiebe wrote: post #204

Earlier quoted context omitted.

Native browser in-page search is working for me, on Firefox. Is this a browser-specific change or is it a staged rollout coming my way soon?

Firefox for me as well. It's dodgy on some pages, but on larger PRs it now breaks.

Yeah for large files/PRs it's complicated because RAM is limited and bandwidth is limited so optimizing for them is tricky.

Re: GitHub appears to be struggling with measly three nines availability

#239

Post by lmm »

braiamp wrote: post #14
embedding-shape wrote: post #3

From GitHub CTO in 2025 when they announced they're moving everything to Azure instead of letting GitHub's infrastructure remain independent: > For us, availability is job #1, and this migration ensures GitHub remains the fast, reliable platform developers depend on That went about as well as everyone thought back then. Does anyone else remember back in ~2014-2015 sometime, when half the community was screaming at Gi…

> I wish we could get back to platforms (or OSes for that matter) focusing in reliability and stability That's only a valid sentiment if you only use the big players. Both of those have medium/smaller competitors that have shown (for decades) that they are extremely boring, therefore stable.

Like who? HN loves to rave about GitLab but they're even less stable than GitHub.

Re: GitHub appears to be struggling with measly three nines availability

#240

Post by hawaiianbrah »

swisniewski wrote: post #170

To be honest, I’m not surprised that GitHub has been having issues. If you have ever operated GitHub Enterprise Server, it’s a nightmare. It doesn’t support active-active. It only supports passive standbys. Minor version upgrades can’t be done without downtime, and don’t support rollbacks. If you deploy an update, and it has a bug, the only thing you can do is restore from backup leading to data loss. This is the sof…

I worked on GHES for a couple of years. Before that, it sounded like it was a sort of volunteer rotation, there wasn’t durable funding for a team when we joined. Mind boggling that the money maker of the company was staffed like that.

It is a complicated project. Thankfully their durable funding story has improved in recent years and they are staffing GHES up at levels they haven’t for at least 7 years. Hopefully it improves. I’m not there anymore, I was laid off last year.

Post reply on HN