GitHub incident: ongoing issues with Actions, Issues and other Git operations • Hacker News Board

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#111

Post by _flux » Tue, May 09, 2023, 2:57 PM UTC

solotronics wrote: post #105

hospitalJail wrote: post #93

I'm the lone person at my team that still believes in keeping most of our stuff local, with online versions as primarily backup. Every time some global service goes down, or internal internet/intranet goes down, there is a security breach, or a WFH person has a power outage I'm reminded I'm right. I'm no luddite, these services make you dependent on them. The worst thing I'm dependent on here is a bad computer. We ha…

GIT is actually a great protocol for keeping distributed copies of code. You can pretty easily with bash cycle through a list of backup urls for a git repo, looking for updates.

Git itself is nice, but then there's the issue tracker and CI that are more difficult to setup.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#112

Post by hospitalJail » Tue, May 09, 2023, 3:19 PM UTC

goodoldneon wrote: post #99

hospitalJail wrote: post #93

I'm the lone person at my team that still believes in keeping most of our stuff local, with online versions as primarily backup. Every time some global service goes down, or internal internet/intranet goes down, there is a security breach, or a WFH person has a power outage I'm reminded I'm right. I'm no luddite, these services make you dependent on them. The worst thing I'm dependent on here is a bad computer. We ha…

I don't know about you, but the stuff I self-manage usually has worse uptime than SaaS products

Depends.

Having our programs offline mean I can run them, even if the internet isnt working.

Instead of getting 0 data from downtime, I can still get the data, run the programs, and give it to the person who needs it.

If we are fully online, if the servers are down, we basically lose the entire time.

Not to mention, I think 'uptime' is a pretty optimistic number, unusable slow service doesnt seem to hit any metrics I'm aware of.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#113

Post by dijit » Tue, May 09, 2023, 3:25 PM UTC

goodoldneon wrote: post #99

hospitalJail wrote: post #93

I'm the lone person at my team that still believes in keeping most of our stuff local, with online versions as primarily backup. Every time some global service goes down, or internal internet/intranet goes down, there is a security breach, or a WFH person has a power outage I'm reminded I'm right. I'm no luddite, these services make you dependent on them. The worst thing I'm dependent on here is a bad computer. We ha…

I don't know about you, but the stuff I self-manage usually has worse uptime than SaaS products

Doesn't matter if your uptime is 80% as long as that 20% of downtime is happening when nobody is working

Additionally an 80% uptime architecture is really simple to maintain and restore and so on.

Complexity increases exponentially the more 9's you add.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#114

Post by that_guy_iain » Tue, May 09, 2023, 4:18 PM UTC

manojlds wrote: post #78

that_guy_iain wrote: post #39

Earlier quoted context omitted.

About 10 years ago someone said we should move to self hosting because bitbucket who we used were unreliable. I looked at the status page and say 2 hours of downtime over 3-months, while we have 3-4 days of downtime on our self hosted jenkins during the same time. I always think of that when I see people complain about services being unreliable. Often we see one or two problems in short span and forget about the mont…

No it isn't. It's been very unstable in recent times ( 1 year or so)

I‘ve personally only experience 1 outage that effected me. And I’ve been using it heavily for the past year and moderately for years before that.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#115

Post by that_guy_iain » Tue, May 09, 2023, 4:19 PM UTC

darkwater wrote: post #96

that_guy_iain wrote: post #39

Earlier quoted context omitted.

About 10 years ago someone said we should move to self hosting because bitbucket who we used were unreliable. I looked at the status page and say 2 hours of downtime over 3-months, while we have 3-4 days of downtime on our self hosted jenkins during the same time. I always think of that when I see people complain about services being unreliable. Often we see one or two problems in short span and forget about the mont…

> while we have 3-4 days of downtime on our self hosted jenkins during the same time Wow. This sounds like a broken organization?

Sounds like a startup with no devops and before a lot of the tooling to make things better.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#116

Post by ishanjain28 » Tue, May 09, 2023, 4:41 PM UTC

subtra3t wrote: post #75

ishanjain28 wrote: post #71

Earlier quoted context omitted.

And how often that has happened? Seems a little harsh

Just like calling GitHub "unreliable garbage"...

Github has been down hundreds of times this year alone. They have reported outages 72 times this year and there are multiple times when services are unavailable and they don't report it on the status page.

I don't see how the two situations are comparable

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#117

Post by PCloud » Tue, May 09, 2023, 4:59 PM UTC

wun0ne wrote: post #3

Does GitHub provide uptime stats? Seems very unreliable recently.

You can checkout https://github.com/GitHub-Incident-History/GitHub-Incident-H... which provides all recorded GitHub incidents.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#118

Post by Sohcahtoa82 » Tue, May 09, 2023, 5:46 PM UTC

capableweb wrote: post #79

candiddevmike wrote: post #51

Earlier quoted context omitted.

This is why I don't touch other people's keyboards.

Little bit of bacteria helps build up your immune system for when you really need it :)

You jest, but it's actually a legit hypothesis.

https://en.wikipedia.org/wiki/Hygiene_hypothesis

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#119

Post by smcleod » Tue, May 09, 2023, 8:26 PM UTC

ta988 wrote: post #91

smcleod wrote: post #57

Earlier quoted context omitted.

You're kidding right? People / companies rely on git being available for deployments, builds, config changes, developer workflow etc

Youa are confusing git and github.

No I'm not. This outage affects Github, not git itself - but if you're storing your git repos (and automation) on Github then you cannot git clone, push etc... from or to them - all of which are critical to CI/CD.

Re: GitHub incident: ongoing issues with Actions, Issues and other Git operations

#120

Post by bombolo » Tue, May 09, 2023, 10:12 PM UTC

electroly wrote: post #104

Kelteseth wrote: post #5

Why don't we all collectively mirror our repos to gitlab and switch development to there during the monthly outage?

GitHub outages aren't nearly long or often enough to consider this. Git is distributed, just keep working locally until GitHub is back up. GitHub outages are nowhere near the threshold of pain I'd require to introduce a second Git hosting provider to the mix. Really, GitHub outages barely hurt at all. It's not like an AWS or Cloudflare outage which is more likely to be a production disaster. Every outage a bunch of p…

Depends how your company is set up. Some people can't run tests locally and just push commits to have some magic run the tests online.