GitHub was down • Hacker News Board

Re: GitHub was down

#91

Post by brobinson » Mon, Jul 13, 2020, 7:55 AM UTC

mundanevoice wrote: post #83

brobinson wrote: post #79

Earlier quoted context omitted.

Rails is (apparently still) a popular target for haters, but for projects at Github's scale it's rarely a code logic/framework-level blunder that's taking the service down. It's generally a cascade of failures in things like multiple database systems, auto-scaling, dns/caching, etc.

There are projects bigger than GitHub that do all of these things and still doesn't fail as much as GitHub. Honestly, both of us speculating. You can't say for sure if it's not Rails and I can't say if it's not anything else. > for projects at Github's scale it's rarely a code logic/framework-level blunder that's taking the service down. It's generally a cascade of failures in things like multiple database systems, a…

Not guessing, postmortems are regularly posted on this site and elsewhere. These kinds of failures are generally not someone using some "magic" (or whatever other pejorative term) feature of a framework.

Re: GitHub was down

#92

Post by NiceWayToDoIT » Mon, Jul 13, 2020, 7:56 AM UTC

On June 29th they also had outage (lasting 2 hrs), does anyone know what was the cause back then?

Re: GitHub was down

#93

Post by dmpetrov » Mon, Jul 13, 2020, 7:57 AM UTC

Why this is usually happen on weekends? The only time that I have for coding :)

Re: GitHub was down

#94

Post by steveniscom » Mon, Jul 13, 2020, 7:58 AM UTC

Illniyar wrote: post #15

Seems like things started to go down the drain somewhere around February 2020. https://www.githubstatus.com/uptime/kr09ddfgbfsf?page=2 Wonder what was the trigger for the reliability hit - actions went GA on nov 2019, so it's something else (or possibly a combination of things)

> Wonder what was the trigger for the reliability hit

Microsoft acquisition, what can I say

Re: GitHub was down

#95

Post by mundanevoice » Mon, Jul 13, 2020, 7:58 AM UTC

brobinson wrote: post #91

mundanevoice wrote: post #83

Earlier quoted context omitted.

There are projects bigger than GitHub that do all of these things and still doesn't fail as much as GitHub. Honestly, both of us speculating. You can't say for sure if it's not Rails and I can't say if it's not anything else. > for projects at Github's scale it's rarely a code logic/framework-level blunder that's taking the service down. It's generally a cascade of failures in things like multiple database systems, a…

Not guessing, postmortems are regularly posted on this site and elsewhere. These kinds of failures are generally not someone using some "magic" (or whatever other pejorative term) feature of a framework.

Agreed. Do we have postmortems for the number of spectacular crashes Github had this year?

Re: GitHub was down

#96

Post by steveniscom » Mon, Jul 13, 2020, 8:00 AM UTC

jaekash wrote: post #71

quyleanh wrote: post #60

I still don't understand people who always mentions to Microsoft's acquisition. Until the official statement, it isn't Microsoft failure. Don't blame them.

You know if you use Microsoft products every day, and every day they let you down, and every day you experience the worst most unintuitive design in the world, and every day you have to deal with their reliability issues, and then MS acquires github, and github starts to behave like everything else MS touches ... Clearly something about how MS runs is responsible for their past outcomes, why is it a stretch to assume…

Yep, Bluescreen

Re: GitHub was down

#97

Post by throwaway20148 » Mon, Jul 13, 2020, 8:02 AM UTC

mundanevoice wrote: post #54

Move off Rails now GitHub, is clearly not working for you. It took you this far, now go move to something that doesn't sh*ts the bed twice every month.

Really, were still doing this?

Re: GitHub was down

#98

Post by moooo99 » Mon, Jul 13, 2020, 8:03 AM UTC

jaekash wrote: post #89

rplnt wrote: post #26

Earlier quoted context omitted.

You can self-host GitHub too. Or BitBucket.

> Or BitBucket But why would anybody ever do that?

I actually enjoyed BitBucket for smaller projects. They offered free private-repos earlier than GitHub did, so that was the main sellingpoint for me.

The self hosted version of BitBucket comes in at a pretty reasonable price. I don't know that for sure (and I can't check thanks to GH being down) but I'd imagine the self-hosted versions for GitHub Enterprise are quite a bit more expensive than Bitbucket or Gitlab.

Re: GitHub was down

#99

Post by echelon » Mon, Jul 13, 2020, 8:03 AM UTC

jaekash wrote: post #86

echelon wrote: post #78

Well, there goes my night. I was waiting on a build triggered by Github actions and was wondering what was up. I guess this is my sign to get some sleep. Microsoft needs to slow things down and focus on stability. This really isn't good. I need these weekend and late night hours for my side hustle. I already have enough trouble as is, I don't need an injection of additional difficulty. (That's just my frustration; I…

Why are you not using an alternative?

> Why are you not using an alternative?

I don't need two CI/CD pipelines. Nor do I have have time to build something like that. I need to spend my time working on the core product.

The risk here is that my deploy SLA is tied to Github. I accepted this risk as the cadence of my deploys tends to be once to twice a day on average and Github is usually available (many nines).

I made a choice, and now that choice is biting me. Now I'll evaluate if I should spend the time and effort to migrate off it. If this is the last of it and Microsoft makes a commitment to not break things, I'll likely stay as the effort to move is nonzero. If this begins to happen every month, on the other hand...

Re: GitHub was down

#100

Post by moooo99 » Mon, Jul 13, 2020, 8:06 AM UTC

dt3ft wrote: post #33

rvz wrote: post #13

14 Days ago, they went down [0]. And today it's happening again. Twice in less than a month. Another reminder to self host via solutions like GitLab or Gitea. [1] [0] https://news.ycombinator.com/item?id=23675864 [1] https://news.ycombinator.com/item?id=23676072

I would choose self-hosting for small to medium size teams any day. I can't fathom why people choose not to self-host at this scale. Your data. Your control. Your network. Your infrastructure. Your responsibility. Are people becoming more afraid of responsibility these days?

> Are people becoming more afraid of responsibility these days?

I wound't say people become more afraid, they just don't see the reason to bother. I'd choose GitHub or Gitlab for small teams at any time. I'd probably even be fine with the free versions.

I see little to no reason for self-hosting in a small team. I cannot imagine a performant server, bandwith and the employee it takes to maintain it to be cheaper than the 10€/month/user for a hosted solution.