Live data from Hacker News

GitHub partial outage

githubstatus.com

21–30 of 88 posts

Re: GitHub partial outage

#21

Anyone using GitLab have any insight on how well their operations are running these days? We originally left GitLab for GitHub after being bit by a major outage that resulted in data loss. Our code was saved, but we lost everything else. But that was almost 10 years ago at this point.

No issues on GitLab.

Haven't seen any outage from GitLab in like, ever.

Re: GitHub partial outage

#24

Anyone using GitLab have any insight on how well their operations are running these days? We originally left GitLab for GitHub after being bit by a major outage that resulted in data loss. Our code was saved, but we lost everything else. But that was almost 10 years ago at this point.

Not sure what specific operational services are of interest - but here's a link to their historical service status [0]

[0] https://status.gitlab.com/pages/history/5b36dc6502d06804c083...

Re: GitHub partial outage

#27

Anyone using GitLab have any insight on how well their operations are running these days? We originally left GitLab for GitHub after being bit by a major outage that resulted in data loss. Our code was saved, but we lost everything else. But that was almost 10 years ago at this point.

We use GitLab on the daily. Roughly 200 repos pushing to ~20 on any given day. There have been a few small, unpublished outages that we determined were server side since we have a geo-distributed team, but as a platform seems far more stable than 5-6 years ago.

My only real current complaint is that the webhooks that are supposed to fire in repo activity have been a little flaky for us over the past 6-8 months. We have a pretty robust chatops system in play, so these things are highly noticeable to our team. It’s generally consistent, but we’ve had hooks fail to post to our systems on a few different occasions which forced us to chase up threads until we determined our operator ingestion service never even received the hooks.

That aside, we’re relatively happy customers.

Re: GitHub partial outage

#28
post #5

Looking forward to the postmortem. Are they using AI agents this time to resolve the outage? Probably not. But this time, there is no CEO of GitHub to contact and good luck contacting Satya to solve the outage.

The postmortem will be simple since Github goes down so consistently every week you can almost use it as an alternative timekeeping system.

Re: GitHub partial outage

#29

Github is owned by Microsoft, so this is a pretty small time indie operation, you need to give them a break.

Just your casual $3.8T company.

There were so many severe Github Actions outages (10+ ?) in the past year. Cause: Migration to the disaster zone also known as Azure, I assume. Most of them happened during (morning) CET working hours, as to not inconvenience the americans and/or make headlines.

Money doesn't buy competency. It's a long-term culture thing. You can never let go on maintaining competency in your organization. It rots if you do. I guess Microsoft did let go.

Re: GitHub partial outage

#30
post #13

Earlier quoted context omitted.

I can't read the entirety of this article[1] because it's paywalled, but it looks like they ran their own servers: > GitHub is currently hosted on the company’s own hardware, centrally located in Virginia I imagine this predates their acquisition from Microsoft. Honestly, given how often Github seems to be down compared to the level of dependency people have on it, this might be one of the few cases where I might hav…

Well… https://www.reuters.com/technology/microsoft-azure-down-thou...

Fair enough, my Azure experience is minimal enough that maybe I shouldn't make assumptions about whether this would improve things. That being said, I do think there's merit in the idea that if Microsoft is going to be able to solve this problem, they probably should try to solve it just once, and in a general way, rather than just for Github?
Post reply on HN