Live data from Hacker News

Incident with Github.com [resolved]

githubstatus.com

81–90 of 1001 posts

Re: Incident with Github.com [resolved]

#81

If you are required to use a Microsoft tool is Azure DevOps now a better choice? Our Azure hosted instance rarely (if ever) has had an outage that effects our ability to work.

The worry I would have is, is anybody at MS still working on it? It is barely updated anymore:

https://azure.microsoft.com/en-us/updates?filters=%5B%22Azur...

That's not necessarily a bad thing for stability, but long-term I would not be surprised if it gets EOL-ed, or radically shrunk to just a basic Git mirroring service.

Re: Incident with Github.com [resolved]

#82
post #22

Earlier quoted context omitted.

Self hosted Gitea

When you self host, what happens with your compute costs? CI can be super expensive depending upon your test pipeline. Do you end up spending more than you would with GitHub actions for example?

Just get some x86 and ARM runners off netcup, manage Woodpecker CI + Woodpecker Agent. Generally pretty hands-off once you get it set up. That plus a cheap VPS running Forgejo with postgres off-site backups, you are looking at a small fraction of the cost compared to per-seat GitHub pricing for orgs + Action Minutes.

Re: Incident with Github.com [resolved]

#84

> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available brutal

20%, do we just make numbers up?

Depends if this is coming from engineering or management

Re: Incident with Github.com [resolved]

#86
post #30

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

> We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" Did you have the conversation "do we need to move from $whatever CI/CD to GitHub Actions?" beforehand? What were the arguments in favour?

It's a ~year old product that started on GH. So this is the first time this team has had to think through "okay, so this isn't as reliable as it was at $oldjob."

Re: Incident with Github.com [resolved]

#87

Can anyone shed some light why GitHub is having so many problems lately?

Their infrastructure is buckling under increased demand from AI generated code.

Imagine the software development lifecycle is a river blocked by several dams- necessary processes that also slow the water’s volume and speed. Writing the code was a very big dam upstream, and successive dams didn’t need to handle a lot of water. With that dam busted wide open, the torrent of code is inundating downstream processes. GitHub happens to host many of them.

Earlier this year the GitHub COO shared a post on Twitter with explosive numbers they were facing. I cannot seem to find it at the moment. (Edit: shared in other replies to this thread.)

Re: Incident with Github.com [resolved]

#90

We're starting to have the conversation of "do we need to replace GitHub as our CI/CD pipeline?" No idea if/when we decide to pull that bandaid but I think once we can't merge a hot fix into main because of GH being down, it'll be a big org push to replace it. What's everyone else using? Self hosted Gitlab? Gitea?

Self-hosted GitLab at work, self-hosted Gitea at home.
Post reply on HN