Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub Actions Down

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 54 posts

Re: GitHub Actions Down

#11

Post by david_draco »

rvz wrote: post #4

So now I am expecting a major GitHub Actions incident (worst case, the whole of GitHub) to go down every single month at least once. Last time this went down was last month. [0] I now doubt if they can consistently manage more than a month without a major incident like this one. [0] https://news.ycombinator.com/item?id=26666843

It could be interesting to track the uptime of such cloud services. Two decades ago, companies prided themselves with 4 or 5 nines (99.99% uptime). Not anymore. Worse is better won yet again ;-)

Companies and programmers should be aware what they get into when they build such dependencies. Distributed git is a thing, but distributed CI/CD that you could also run locally isn't (yet?).

Re: GitHub Actions Down

#12

Post by limoce »

My workflows have been queued for at least four hours. It's a hotfix commit. GitHub Actions helped me a lot in synchronizing updates with other repositories automatically, but this time I have to manually apply it. Because I'm the first time expecting GitHub Actions incident, I was wondering that I ran out of my free quota for this month...

By the way, should GitHub send an email to the owner if any workflow has been delayed for an unreasonable time?

Re: GitHub Actions Down

#13

Post by ddlsmurf »

limoce wrote: post #12

My workflows have been queued for at least four hours. It's a hotfix commit. GitHub Actions helped me a lot in synchronizing updates with other repositories automatically, but this time I have to manually apply it. Because I'm the first time expecting GitHub Actions incident, I was wondering that I ran out of my free quota for this month... By the way, should GitHub send an email to the owner if any workflow has been…

Personally I'd rather they focus on the issue rather than negotiating with PR and legal to formulate an e-mail. If you rely on an external service and don't monitor it that's on you.

Re: GitHub Actions Down

#14

Post by cjohansson »

Hamuko wrote: post #8
rurban wrote: post #5

Earlier quoted context omitted.

Easy to explain. They switched to the MS Azure cloud for actions. You won't get high availability from Microsoft you are used to as from proper cloud services. Plus privacy issues. But it's cheap, in this case for free.

I was amused by reports when Microsoft was in talks with Discord that one of the reasons why Microsoft wanted to buy Discord was because they wanted Discord on Azure. Like, was the grand customer acquisition strategy for Azure just acquiring the companies and then migrating over?

Sounds like MS, that is the only way they can get organic customers. All their popular products are originally built by someone else except Windows of course

Re: GitHub Actions Down

#15

Post by pilif »

sebmellen wrote: post #10

We build an Electron app for MacOS regularly. We run these builds on Github’s MacOS VMs. The internet fails to connect when running yarn install about 3 out of every 5 times. We’ve gotten multiple refunds for this issue, but it’s still a complete mess. Unfortunately we’ve built much of our process around GH Actions... if not I’m sure we would be on CircleCI or TravisCI already. We’ve also considered switching to self…

We’re running everything on self-hosted runners and it works like a charm. They are also way, way faster than what GitHub can provide.

We get the advantages of the huge GitHub Actions ecosystem while having impressive (and fully controllable) performance and very easy access to out infrastructure for deployments.

Re: GitHub Actions Down

#16

Post by _odey »

isbvhodnvemrwvn wrote: post #9
Hamuko wrote: post #8

Earlier quoted context omitted.

I was amused by reports when Microsoft was in talks with Discord that one of the reasons why Microsoft wanted to buy Discord was because they wanted Discord on Azure. Like, was the grand customer acquisition strategy for Azure just acquiring the companies and then migrating over?

My wife works as an AWS/Azure consultant, and she mentions that in our area it's much more common for the non-technical management to push Azure than it is for technology to choose it. Sounds quite IBM-ish/Oracle-ish.

When you open a Microsoft account for a new company, they do a lookup to figure out your area of activity and if you're a good match you get contacted by a sales representative asking you if you want to become an Azure re-seller. Basically for services you sell to third parties you get Azure credits meaning your Azure usage is "free", and your clients pay the premium. I know this because I used to work for a company that did this, and from personal experience when starting a company.

Edit: Here it a tip; if you see a "Microsoft partner Cloud Platform" badge on a outsourcers website stay away.

Re: GitHub Actions Down

#17

Post by NicoJuicy »

cjohansson wrote: post #14
Hamuko wrote: post #8

Earlier quoted context omitted.

I was amused by reports when Microsoft was in talks with Discord that one of the reasons why Microsoft wanted to buy Discord was because they wanted Discord on Azure. Like, was the grand customer acquisition strategy for Azure just acquiring the companies and then migrating over?

Sounds like MS, that is the only way they can get organic customers. All their popular products are originally built by someone else except Windows of course

Azure, .net, office, office 365, teams, ml.net, ...

Not a single one i could manage building ( eg. Teams has bots, quick to create apps and pretty advanced cam features)

Re: GitHub Actions Down

#18

Post by Aeolun »

david_draco wrote: post #11
rvz wrote: post #4

So now I am expecting a major GitHub Actions incident (worst case, the whole of GitHub) to go down every single month at least once. Last time this went down was last month. [0] I now doubt if they can consistently manage more than a month without a major incident like this one. [0] https://news.ycombinator.com/item?id=26666843

It could be interesting to track the uptime of such cloud services. Two decades ago, companies prided themselves with 4 or 5 nines (99.99% uptime). Not anymore. Worse is better won yet again ;-) Companies and programmers should be aware what they get into when they build such dependencies. Distributed git is a thing, but distributed CI/CD that you could also run locally isn't (yet?).

The fact that it’s down is not the problem, but it’s painful that there’s nothing I can do to fix it.

That’s the part I like about self hosting. It may ultimately be down more often, but I never have to tell someone “Nothing to be done, we wait.”.

Re: GitHub Actions Down

#19

Post by danielheath »

purerandomness wrote: post #7
rvz wrote: post #4

So now I am expecting a major GitHub Actions incident (worst case, the whole of GitHub) to go down every single month at least once. Last time this went down was last month. [0] I now doubt if they can consistently manage more than a month without a major incident like this one. [0] https://news.ycombinator.com/item?id=26666843

Isn't this normal with hosted solutions? IIRC, GitHub, Bitbucket and GitLab.com are all unusable for a few hours at least once a month as far back as I can remember. Isn't this just commonly accepted as a tradeoff for not having to manage the servers yourself?

I would say it's normal for businesses which are engaged in competitive feature development.

Stability & reliability are relatively easy to achieve if you aren't changing the software frequently.

Re: GitHub Actions Down

#20

Post by Sebb767 »

danielheath wrote: post #19
purerandomness wrote: post #7

Earlier quoted context omitted.

Isn't this normal with hosted solutions? IIRC, GitHub, Bitbucket and GitLab.com are all unusable for a few hours at least once a month as far back as I can remember. Isn't this just commonly accepted as a tradeoff for not having to manage the servers yourself?

I would say it's normal for businesses which are engaged in competitive feature development. Stability & reliability are relatively easy to achieve if you aren't changing the software frequently.

I'm pretty sure they're constantly working on their software (at least GitLab is), so "aren't changing anything" does probably not apply.
Post reply on HN