Your weekly reminder to take a break
I still can't pull new branches even though the incident says it's resolved. I don't think my boss would be happy with me taking a break this long... but what else can I do when our business uses GH?
GitHub partial outage
71–80 of 88 posts
Re: GitHub partial outage
#72Earlier quoted context omitted.
Just your casual $3.8T company. There were so many severe Github Actions outages (10+ ?) in the past year. Cause: Migration to the disaster zone also known as Azure, I assume. Most of them happened during (morning) CET working hours, as to not inconvenience the americans and/or make headlines. Money doesn't buy competency. It's a long-term culture thing. You can never let go on maintaining competency in your organiza…
I thought GitHub Actions (in particular; not the rest of GitHub) was always Azure, because it was initially a fork of Azure Pipelines? GitHub as a whole, including the previously non-Azure bits, does seem flakier than a few years ago though, for sure.
It's possible that, even though the Actions part was always on Azure, migrating the other parts to Azure broke some connectivity between the pieces....
Re: GitHub partial outage
#73Coincidentally, Azure Devops was also missing the ssh keys earlier today, both in the web ui and for ssh login.
Well, github is moving to Azure and they are consolidating systems. No surprise there.
Re: GitHub partial outage
#74Earlier quoted context omitted.
I thought GitHub Actions (in particular; not the rest of GitHub) was always Azure, because it was initially a fork of Azure Pipelines? GitHub as a whole, including the previously non-Azure bits, does seem flakier than a few years ago though, for sure.
You seem to be correct. Not that much visible from the outside, but yes it seems like they always ran on Azure, from the 2018 launch. (Apologies for the disinfo, although I qualified it with the "I assume".)
Pre-launch I seem to recall using an entirely different product with the same name, that supported CUE or HCL and had a better gui editor. I think post acquisition they scrapped it for the current (and IMO) worse reskin.
Re: GitHub partial outage
#75Earlier quoted context omitted.
I bet Microsoft is sad not because people can’t push, but because the training data for Copilot has slowed down. PS: None of our 40+ engineers felt anything, our self hosted Forgejo is as snappy as ever.
until your hardware fails! Or your VPS provider goes down! Or whatever else, software services going down is going to happen in some capacity, eventually. Real question is what is acceptable
When you self host you also learn how to backup. It’s not complicated actually, you should look into it.
Re: GitHub partial outage
#76Speaking of GitHub issues if you go to Insight->Traffic in your repo you’ll most likely see this banner: “ Referring sites and popular content are temporarily unavailable or may not display accurately. We're actively working to resolve the issue.” It’s been like that for months now with no sign of anyone working on it. They just don’t care about user experience anymore. https://github.com/orgs/community/discussions/1…
Speaking of "temporarily unavailable but it's actually forever", I've been wanting to get into Fallout and Starfield modding, so been waiting for their official wiki to come out of maintenance mode. I think I first tried to access it when Starfield launched (September 2023), and still today it is "currently down for backend maintenance". https://wiki.bethesda.net/
Re: GitHub partial outage
#77[dead]
Re: GitHub partial outage
#78Speaking of GitHub issues if you go to Insight->Traffic in your repo you’ll most likely see this banner: “ Referring sites and popular content are temporarily unavailable or may not display accurately. We're actively working to resolve the issue.” It’s been like that for months now with no sign of anyone working on it. They just don’t care about user experience anymore. https://github.com/orgs/community/discussions/1…
Hey, I'm the GitHub employee who's working on fixing that right now. The service powering those stats is _ancient_, and it fell over back in September. It's taken longer than I hoped to get a replacement working, but it should be fixed within the next couple of weeks, fingers crossed.
Re: GitHub partial outage
#79It's possible that Microsoft buying GitHub was a large-scale psyop intended to reduce the productivity of the competition. Any time their startup competitors are making too much progress they can just push the "GitHub incident" button and slow everyone down.
We used to obsessively care about 500s. Like I would make a change that caused a 0.1% spike in 500s and I would silently say I'm sorry to the folks who got the unicorn page. I'm not sure the new school cares nearly as much. But then again this is how companies change as they mature. I saw this with StubHub as well.. The people who care the most are the initial employees, employee #7291 usually dgaf
A lot of downvoters seem to have not realized that my comment was a joke.
Though yeah, for startups who depend on GitHub for CI and CD, it's been noticeable how absurdly unreliable GitHub has become over the years.
Re: GitHub partial outage
#80I’m old enough to remember when GitHub was on main page due to a cool feature they added, now they just end up here when it stops working
If I remember it well, every once in a while a new cool feature was also breaking stuff, doubling the chances of getting to the top page here. But truth being told, GitHub was fixing those at light speed too and it was very interesting to follow their progress. Their delivery pipeline (per branch, deliver when ready, etc.) sounded very much innovative by then and I think inspired many people.