Live data from Hacker News

Incident with GitHub Actions

githubstatus.com

31–40 of 50 posts

Re: Incident with GitHub Actions

#31

I would maybe start migrating repos away from GitHub, since they have proven to be quite unreliable. Nonetheless, I must say I do appreciate GitHub's UI and even their CLI is quite nice. Is there a service out there that basically provides the same good user experience, CI/CD and has no costs for public repos? Apart from the unreliability I can't complain about GitHub.

[deleted]

Re: Incident with GitHub Actions

#32
post #25

Earlier quoted context omitted.

scaling is a hard problem to solve, so props to them for doing it without too much issues, overall.. but damn, since the microsoft acquisition, they are having more issues more frequently

Actions runs on Azure, actions were only possible with Microsoft.

Ah yes. No one else on the internet will rent you servers.

Re: Incident with GitHub Actions

#33
post #7

Again? Last time this happened was 4 days ago as I also said: >>> Hopefully GitHub won't have another outage in a week's (or even a months) time. [0] Yet another incident less than a week / 4 days later with GitHub Actions if one was to go 'all in' on GitHub. Would have to reset the counter once again. You can see in the whole comment chain [0] and in [1] why I was totally right in the 'long term' of not 'centralizin…

Is there a reason you’re continuously posting a link to a previous comment like anyone wants to read it? If this is a passion of yours, may I suggest posting about multi cloud strategies and not being entirely dependent on any of the providers. You’ll have more opportunities to point out you’re right each time any of them go down.

Re: Incident with GitHub Actions

#34

As the person who gets pinged when our CI isn't working, I'm hoping my next employer doesn't have github in their stack. We've reached the stage where architectural decisions are being questioned. Nobody should need to ask me "well why are we using github?" but here we are. They needed to fix this two weeks ago. The next time I'm in charge of stack decisions I will be evaluating competitors. This is exactly how Slack…

I'm curious, what's stopping you from hosting on-premise and/or using your own runners?

We aren't a big enough team. The resources to do such a thing would mean less time focused on customers.

Maybe after another doubling we will have these options available. We should be a perfect fit at this stage for a hosted solution. Github is making themselves unviable for a company at this stage (low $XX million ARR)

Re: Incident with GitHub Actions

#35

As the person who gets pinged when our CI isn't working, I'm hoping my next employer doesn't have github in their stack. We've reached the stage where architectural decisions are being questioned. Nobody should need to ask me "well why are we using github?" but here we are. They needed to fix this two weeks ago. The next time I'm in charge of stack decisions I will be evaluating competitors. This is exactly how Slack…

I'm curious, what's stopping you from hosting on-premise and/or using your own runners?

Using your own runners GitHub doesn’t completely solve the problem, because GitHub won’t queue them to start the job during these outages.

Re: Incident with GitHub Actions

#36

Earlier quoted context omitted.

It's a bit of a weird thing as the whole word tries (or tried) to move away from on-prem stuff. Like Jira stopping support for non-cloud versions etc.

Probably Atlassian was sick of people never upgrading their old installations and getting hacked for it, and people did not upgrade because it is quite a hassle in the first place, not to mention plugins breaking allll the time. Oh and because cloud forces continuous payment whereas prior many customers simply bought a one year license and went on without renewing support.

We're running a selfhosted Gitlab Premium since 2019. The only two times in the last 3 years we had issues with artifacts not being deleted (causing nightly backups to become 500gb, will be fixed in the next version) and some out of date apt certs to run "apt update". Otherwise, I update Gitlab every month without problems.

Re: Incident with GitHub Actions

#37
This is getting out of hand. I had a call with github enterprise sales yesterday about this. Best they could offer is a blog post link and taking more of our money.

Hoping to migrate away our little chunk of github to a private island by end of next week.

Hackernews talks a lot of shit about how big boys can run their computers better than us lowly startup peasants. This is a fortunate situation where that stale argument starts to fall apart really quickly. I have many other systems with far higher availability than what github offers in their public cloud or even as part of the contractual 99.5% enterprise SLA.

Sure, github is really complicated and hard. Yes, it's an incredible tool. No, it's not OK to rest on your laurels and let COTS database technology kill your product when you are a billion dollar technology company with the ability to write your own systems from scratch several times per year using parallel teams and other investor money furnaces.

Re: Incident with GitHub Actions

#39

Earlier quoted context omitted.

I'm curious, what's stopping you from hosting on-premise and/or using your own runners?

We aren't a big enough team. The resources to do such a thing would mean less time focused on customers. Maybe after another doubling we will have these options available. We should be a perfect fit at this stage for a hosted solution. Github is making themselves unviable for a company at this stage (low $XX million ARR)

"We aren't big enough to self-host" is an example of the effectiveness of SaaS sales pitches.

Don't get me wrong, there's a lot of good reasons to go cloud.

There's not a lot of good reasons for a build system to be completely down with no backup plans when your SaaS provider has an outage.

What does even just Github Actions' availability look like for March? 90%? That's pretty brutal for what should be a five nines service.

Re: Incident with GitHub Actions

#40
post #27

Earlier quoted context omitted.

scaling is a hard problem to solve, so props to them for doing it without too much issues, overall.. but damn, since the microsoft acquisition, they are having more issues more frequently

Product development has also accelerated since the acquisition. Lots of interesting stuff has been released since it

Flops have been released too (Copilot I'm looking at you)
Post reply on HN