Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub partial outage

githubstatus.com

Post reply on HN
Print view View original

41–50 of 88 posts

Re: GitHub partial outage

#41

Post by lysace »

conception wrote: post #40
lysace wrote: post #29

Earlier quoted context omitted.

Just your casual $3.8T company. There were so many severe Github Actions outages (10+ ?) in the past year. Cause: Migration to the disaster zone also known as Azure, I assume. Most of them happened during (morning) CET working hours, as to not inconvenience the americans and/or make headlines. Money doesn't buy competency. It's a long-term culture thing. You can never let go on maintaining competency in your organiza…

“guess Microsoft did let go” - are we thinking of the same Microsoft here?

I am thinking of the atrophying one. Not MikeRoweSoft.

Re: GitHub partial outage

#42

Post by grepfru_it »

wavemode wrote: post #34

It's possible that Microsoft buying GitHub was a large-scale psyop intended to reduce the productivity of the competition. Any time their startup competitors are making too much progress they can just push the "GitHub incident" button and slow everyone down.

We used to obsessively care about 500s. Like I would make a change that caused a 0.1% spike in 500s and I would silently say I'm sorry to the folks who got the unicorn page.

I'm not sure the new school cares nearly as much. But then again this is how companies change as they mature. I saw this with StubHub as well.. The people who care the most are the initial employees, employee #7291 usually dgaf

Re: GitHub partial outage

#43

Post by gen220 »

kaishiro wrote: post #27
numbsafari wrote: post #20

Anyone using GitLab have any insight on how well their operations are running these days? We originally left GitLab for GitHub after being bit by a major outage that resulted in data loss. Our code was saved, but we lost everything else. But that was almost 10 years ago at this point.

We use GitLab on the daily. Roughly 200 repos pushing to ~20 on any given day. There have been a few small, unpublished outages that we determined were server side since we have a geo-distributed team, but as a platform seems far more stable than 5-6 years ago. My only real current complaint is that the webhooks that are supposed to fire in repo activity have been a little flaky for us over the past 6-8 months. We ha…

FWIW, GitHub is also unreliable with webhooks. Many recent GH outages have affected webhooks.

They are pretty good, in my experience, at *eventually* delivering all updates. The outages take the form of a "pause" in delivery, every so often... maybe once every 5 weeks?

Usually the outages are pretty brief but sometimes it can be up to a few hours. Basically I'm unaware of any provider whose webhooks are as reliable as their primary API. If you're obsessive about maintaining SLAs around timely state, you can't really get around maintaining some sort of fall-back poll.

Re: GitHub partial outage

#44

Post by isodev »

ecshafer wrote: post #23

Github is owned by Microsoft, so this is a pretty small time indie operation, you need to give them a break.

I bet Microsoft is sad not because people can’t push, but because the training data for Copilot has slowed down.

PS: None of our 40+ engineers felt anything, our self hosted Forgejo is as snappy as ever.

Re: GitHub partial outage

#45

Post by blibble »

gkoberger wrote: post #36
dustfinger wrote: post #35

Why does the main page show all green when there is an ongoing incident? All green here -> https://www.githubstatus.com/

It's marked as resolved for some reason

because then some mid-level manager gets a telling off

and/or has to pay the SLA out of their budget

Re: GitHub partial outage

#47

Post by philipwhiuk »

colesantiago wrote: post #21
numbsafari wrote: post #20

Anyone using GitLab have any insight on how well their operations are running these days? We originally left GitLab for GitHub after being bit by a major outage that resulted in data loss. Our code was saved, but we lost everything else. But that was almost 10 years ago at this point.

No issues on GitLab. Haven't seen any outage from GitLab in like, ever.

https://status.gitlab.com/pages/history/5b36dc6502d06804c083...

Re: GitHub partial outage

#48

Post by kaishiro »

gen220 wrote: post #43
kaishiro wrote: post #27

Earlier quoted context omitted.

We use GitLab on the daily. Roughly 200 repos pushing to ~20 on any given day. There have been a few small, unpublished outages that we determined were server side since we have a geo-distributed team, but as a platform seems far more stable than 5-6 years ago. My only real current complaint is that the webhooks that are supposed to fire in repo activity have been a little flaky for us over the past 6-8 months. We ha…

FWIW, GitHub is also unreliable with webhooks. Many recent GH outages have affected webhooks. They are pretty good, in my experience, at *eventually* delivering all updates. The outages take the form of a "pause" in delivery, every so often... maybe once every 5 weeks? Usually the outages are pretty brief but sometimes it can be up to a few hours. Basically I'm unaware of any provider whose webhooks are as reliable a…

Completely agree on all points. We've had dual remotes running on a few high traffic repos pushing to both GitLab and GitHub simultaneously as a debug mechanism and our experiences mirror yours.
Post reply on HN