How many 9s is GitHub at now? 2?
Github proudly boasts an industry-leading seven 9s of uptime. 49.999999%
GitHub having issues [resolved]
41–50 of 150 posts
Re: GitHub having issues [resolved]
#42How many 9s is GitHub at now? 2?
90 day non-degraded uptime of Github Actions is 98.8% if the official numbers can be believed
Re: GitHub having issues [resolved]
#43Happening very often lately
and we all know why
Re: GitHub having issues [resolved]
#44I swear this is my fault. I can go weeks without doing infra work. Github does fine, I don't see any hiccups, status page is all green. But the day comes that I need to tweak a deploy flow, or update our testing infra and about halfway through the task I take the whole thing down. It's gotten to the point where when there's an outage I'm the first person people ask what I'm doing...and it's pretty dang consistent....
do you know the Pauli-Effect? https://en.wikipedia.org/wiki/Pauli_effect
Re: GitHub having issues [resolved]
#45Earlier quoted context omitted.
Top-down mandates to use AI as much as possible, and to rip up their infrastructure and move everything to Azure. https://www.windowscentral.com/microsoft/using-ai-is-no-long... https://thenewstack.io/github-will-prioritize-migrating-to-a...
I figured that it would be something like that. But it's been so frequent that I expect the leadership to act decisively towards a long-term reliability plan. Unfortunately they have near monopoly in this space, so I guess there's not enough incentive to fix the situation.
How frequent? I think the obsession with uptime is annoying. If GitHub is down, if there’s something so critical, then you need some more control of the system. Otherwise take a couple hours and get a coffee or an early lunch.
Re: GitHub having issues [resolved]
#46I have a bug bash in an hour and fixes that need to go in beforehand. So of course GitHub is down.
Re: GitHub having issues [resolved]
#47In moments like this, it's useful to have a "break glass" mode in your CI tooling: a way to run a production CI pipeline from scratch, when your production CI infrastructure is down. Otherwise, if your CI downtime coincides with other production downtime, you might find yourself with a "bricked" platform. I've seen it happen and it is not fun.
It can be a pain to setup a break-glass, especially if you have a lot of legacy CI cruft to deal with. But it pays off in spades during outages.
I'm biased because we (dagger.io) provide tooling that makes this break-glass setup easier, by decoupling the CI logic from CI infrastructure. But it doesn't matter what tools you use: just make sure you can run a bootstrap CI pipeline from your local machine. You'll thank me later.
Re: GitHub having issues [resolved]
#48Are we serious?
Re: GitHub having issues [resolved]
#49Earlier quoted context omitted.
I figured that it would be something like that. But it's been so frequent that I expect the leadership to act decisively towards a long-term reliability plan. Unfortunately they have near monopoly in this space, so I guess there's not enough incentive to fix the situation.
How frequent? I think the obsession with uptime is annoying. If GitHub is down, if there’s something so critical, then you need some more control of the system. Otherwise take a couple hours and get a coffee or an early lunch.
Frequent enough to interrupt the flow of an entire organization, wasting thousands of hours. Take a look: