Live data from Hacker News

GitHub Actions and Pages are experiencing degraded availability

githubstatus.com

331–340 of 449 posts

Re: GitHub Actions and Pages are experiencing degraded availability

#331

Earlier quoted context omitted.

We have set up a new enterprise and are migrating to GHE. For us this was a no-brainer; we get our own isolated environment, Enterprise Managed Users with Entra OIDC provisioning and Github Copilot in EU data residency. https://eu.githubstatus.com/posts/dashboard looks pretty good to me. We will still keep our github.com enterprise around for public repositories. I’m genuinely curious to know why a company would pref…

I wasn't directly involved, but working in a couple of shops that used GH, I think they really steered people away from GHE, my impression was it's a product they don't really want to support - I vaguely recall hearing that upgrades were very painful on GHE, and the scaling / failover story was not great. Can anyone else confirm?

To clarify, we are on Github Enterprise Cloud, not Server. The option for enterprise managed users with your own ghe.com subdomain and data residency options is still prominently shown in the enterprise signup flow. https://docs.github.com/en/enterprise-cloud@latest/enterpris...

Re: GitHub Actions and Pages are experiencing degraded availability

#333

I don't think this portents anything great for software in general. We're a good year+ into the use LLMs for all major bits of software that we all rely upon and GitHub here is down to one 9 of uptime. I've been using GitHub for a _long_ time, my first commits there go back to August 2009!, and I honestly don't recall GitHub going down as much as it has in the last year. I'm sure there's other things happening in the…

Are these outages caused by introduced bugs, though, or by load issues? As someone who has spent many years working in high load environments, this is not an uncommon pattern. You design a system and it works great. It can handle failures, load spikes, it is horizontally scalable, things are great. You think you figured it out. And then load keeps increasing and you suddenly hit a tipping point where everything keeps…

But that's just a perfectly fitting excuse for GitHub degradation.

Web search, steaming, high frequency trading, and many other systems are resource demanding, and keep scaling. Whether to handle the surge due to bots or wider adoption. But GitHub can't scale git? It isn't even git failing.

Incompetence in leadership is what makes a tech business technically unable to meet growing demand.

Re: GitHub Actions and Pages are experiencing degraded availability

#334
post #204

Earlier quoted context omitted.

There was a great article awhile back that shed some light on just how dysfunctional azure is as a platform: https://news.ycombinator.com/item?id=47616242 If I had to guess it's because Github is sitting on top on infrastructure held up by toothpicks and duct tape

Which is somewhat humorous because it was arguably more stable when they ran on their own hosted colo infra before moving to Azure. This was a choice versus keeping the infra compartmentalized and using Azure for elastic overflow compute needs. I'm sure marketing and bonuses rest on throwing it all on the Azure quicksand though. GitHub Will Prioritize Migrating to Azure Over Feature Development - https://news.ycombin…

It pleases shareholders more to hear about exponential Cloud offering adoption than SLA availability for a developer platform.

Re: GitHub Actions and Pages are experiencing degraded availability

#335
This incident made me look at the SLAs we have with GitHub, since our company is on the Enterprise plan. Interestingly, it seems that only failed GHA invocations are counted towards an SLA breach. We informed engineering to stop using GHA while the incident was ongoing, but it seems what we should have done was to hammer them without remorse to get the whole incident period (10+ hours) counted towards an SLA breach. What a scummy SLA design.

Re: GitHub Actions and Pages are experiencing degraded availability

#336

I don't think this portents anything great for software in general. We're a good year+ into the use LLMs for all major bits of software that we all rely upon and GitHub here is down to one 9 of uptime. I've been using GitHub for a _long_ time, my first commits there go back to August 2009!, and I honestly don't recall GitHub going down as much as it has in the last year. I'm sure there's other things happening in the…

Are these outages caused by introduced bugs, though, or by load issues? As someone who has spent many years working in high load environments, this is not an uncommon pattern. You design a system and it works great. It can handle failures, load spikes, it is horizontally scalable, things are great. You think you figured it out. And then load keeps increasing and you suddenly hit a tipping point where everything keeps…

Actions is one thing, that seems like a difficult, dynamic and bursty thing to host, even before the Vibe Cambrian Explosion. And _nothing_ at scale is easy... But Pages? The static sites? Down for so many hours? Oof.

Re: GitHub Actions and Pages are experiencing degraded availability

#337

Earlier quoted context omitted.

Are these outages caused by introduced bugs, though, or by load issues? As someone who has spent many years working in high load environments, this is not an uncommon pattern. You design a system and it works great. It can handle failures, load spikes, it is horizontally scalable, things are great. You think you figured it out. And then load keeps increasing and you suddenly hit a tipping point where everything keeps…

But that's just a perfectly fitting excuse for GitHub degradation. Web search, steaming, high frequency trading, and many other systems are resource demanding, and keep scaling. Whether to handle the surge due to bots or wider adoption. But GitHub can't scale git? It isn't even git failing. Incompetence in leadership is what makes a tech business technically unable to meet growing demand.

> Incompetence in leadership is what makes a tech business technically unable to meet growing demand.

I don't think this is true. Take LLMs for example, you need GPUs to serve them and these are in short supply at the moment making it hard to meet demand. I don't think this is necessarily the fault of incompetent leadership.

Re: GitHub Actions and Pages are experiencing degraded availability

#339

Earlier quoted context omitted.

Are these outages caused by introduced bugs, though, or by load issues? As someone who has spent many years working in high load environments, this is not an uncommon pattern. You design a system and it works great. It can handle failures, load spikes, it is horizontally scalable, things are great. You think you figured it out. And then load keeps increasing and you suddenly hit a tipping point where everything keeps…

But that's just a perfectly fitting excuse for GitHub degradation. Web search, steaming, high frequency trading, and many other systems are resource demanding, and keep scaling. Whether to handle the surge due to bots or wider adoption. But GitHub can't scale git? It isn't even git failing. Incompetence in leadership is what makes a tech business technically unable to meet growing demand.

GitHub is different because they need to scale writes, that’s fundamentally a much much harder problem

Re: GitHub Actions and Pages are experiencing degraded availability

#340

Earlier quoted context omitted.

To me this correlates more to them being bought by Microsoft, a company known for being seemingly incapable of creating quality software to the point that it's not even funny anymore, and also known for sloppifying all the products they touch.

Microsoft is the General Motors of software.

at present I'd almost put them at Stellantis levels of bad
Post reply on HN