Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

391–400 of 1001 posts

Re: Incident with Github.com [resolved]

#391

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

If you're seriously looking at Gitea/Forgejo, I run Fjord, dedicated managed Forgejo, so you don't have to operate it yourself. We can help with migration and runners too. We also run a free public Forgejo instance if you just want to kick the tires first.

Re: Incident with Github.com [resolved]

#392
post #164

Earlier quoted context omitted.

> Github’s servers are constantly on fire as their usage increased something like 50x due to LLM sloppers pushing large amounts of trash code I sympathized with them when they said this a handful of months ago, but then I saw this [0] page that shows how it's been shot for years prior (which tracks with my memory). I feel for the GH engineers that have to deal with this, especially the SREs. I also don't hate the dow…

They _are_ the LLM coding agent vendor, and are _owned_ by MS, the ~majority~ biggest shareholder of OpenAI. How can you NOT consider that 20x+ scaling in your capacity roadmap projections, where you are trying to get everyone to use these agents as part of your core OKRs? Or, maybe your #1 IT priority was moving everything to Azure instead ;)

> the majority owner of OpenAI

Largest shareholder (27%), not majority.

Re: Incident with Github.com [resolved]

#393

Earlier quoted context omitted.

what the hell does having american born engineers have anything to do with software quality. Man. This site is filled with weird racism.

outsourced cost center

yeah, that's called capitalism. What does it have to do with "foreign".

If Capitalists could, and are doing, they'd make alabama be engineers if they could afford it.

It's so weird watching the tech work force go ham into the "tey terk ow jobz" rather than look at capitalism.

Re: Incident with Github.com [resolved]

#394
post #110

It's been a downward spiral since covid. Enterprise works ok-ish, but the standard version has daily issue ever since.

I don't have data for it, but I have been a Github user since 2012 and have found it to be down often for pretty much that entire time. I always figured it's cultural to a degree–new features always seem pretty buggy/underbaked, and often continue to long term but remain unloved and incomplete.

I has never been so bad as in the couple last years. And it's getting worse.

I never had to wonder if Steam is going to be working today, so I could play my game after work, but it's been an issue with GitHub since covid. At least for me.

Fortunately, due the nature of the service, I can sit out most downtimes. Most of the time at least.

Re: Incident with Github.com [resolved]

#395

Earlier quoted context omitted.

Yes, but GitHub enterprise has different uptime ( https://us.githubstatus.com/posts/dashboard ) and actually does have an SLA. Edit: it seems GHE is also affected by the outage and isn't much better.

Since upgrading to Cloud Enterprise... why didn't my GH uptime get any better? Guess on-prem GHE server is the only way to do that.

[dead]

Re: Incident with Github.com [resolved]

#396
post #361
post #99

This is starting to feel like Twitter. For a variety of reasons, we had a centralized place where everyone of a particular set of persuasions could connect, and this had outsize benefits for the community as a whole. That place is becoming untenable, and with the loss of goodwill and stability, the community’s started to dissipate. But there isn’t one obvious transition candidate, so the diaspora is finding itself sp…

At least this will wake up more people to the existing alternative efforts such as Forgejo and its progress with federation protocol. https://forgejo.org/2023-01-10-answering-forgejo-federation-...

Do you have any more insight into its progress?

Seems like it's far from production-ready and there hasn't been much activity this year?

Re: Incident with Github.com [resolved]

#398
post #99

This is starting to feel like Twitter. For a variety of reasons, we had a centralized place where everyone of a particular set of persuasions could connect, and this had outsize benefits for the community as a whole. That place is becoming untenable, and with the loss of goodwill and stability, the community’s started to dissipate. But there isn’t one obvious transition candidate, so the diaspora is finding itself sp…

Good. That's a gain, not a loss. We're better off without social media psyop manipulations and subscription services bolted onto our code repositories. We're better off without a giant evilcorp getting wholesale access to a large majority of the world's source code.

I just use GitHub for the basics: repo hosting, reviews and actions - what are the "social media psyop manipulations"? I've never really paid much attention to the other features.

Re: Incident with Github.com [resolved]

#399
post #322

Earlier quoted context omitted.

I’m willing to give them a break as I’m assuming they have a lot of scaling problems due to the influx of LLM assisted coding. But maybe I’m wrong?

How many weeks/months can they use this excuse? They are literally are at the forefront of this emerging industry and are capturing untold value. To let their product suffer and potentially lose market share because of it is extremely foolish

Scaling can take time but they've had at least a year to prepare. It's not like any demand increase they've seen was overnight...

At most normal companies you monitor your systems and address potential bottlenecks before they reach a tipping point. And generally you want enough headroom that a sudden 2-3x increase in demand wouldn't take out the service.

Either Github's technical leadership/talent is completely out of their depth (seems unlikely), or they're not hiring enough engineers to work on these capacity issues.

Re: Incident with Github.com [resolved]

#400

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

Self hosted Gitlab at work with local runners
Post reply on HN