Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub having issues today

githubstatus.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: GitHub having issues today

#41

Post by markphip »

mrweasel wrote: post #12
AlchemistCamp wrote: post #10

Earlier quoted context omitted.

More likely, it's efforts to migrate away from the previously solid Rails stack to MS's preferred stack. They've had a long history of this kind of stability issue when migrating or trying to migrate acquisitions from their previous stack to an MS one. This happened with Hotmail (Unix server -> Windows server), LinkedIn (custom cloud -> MS cloud) and others since.

Is Github moving to .Net and/or SQLServer, or is it "just" moving everything to Azure?

Most of GitHub runs in its own data centers. Services like actions runners and Codespaces use Azure.

Re: GitHub having issues today

#42

Post by nijave »

dcchambers wrote: post #28
acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

Every time there's a GitHub outage of any severity one of the top comments on HN is inevitably suggesting that it's probably due to Rails. It's getting pretty tiresome. Calling it a "legacy rails" stack is incredibly disingenuous as well. It's not like they're running a 5 year old unsupported version of Rails/MySQL. GitHub runs from the Rails main branch - the latest stable version they possibly can - and they update…

It's easier to blame than to admit running a service at Github's scale is highly complex and takes deep expertise.

Re: GitHub having issues today

#44

Post by acedTrex »

dcchambers wrote: post #28
acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

Every time there's a GitHub outage of any severity one of the top comments on HN is inevitably suggesting that it's probably due to Rails. It's getting pretty tiresome. Calling it a "legacy rails" stack is incredibly disingenuous as well. It's not like they're running a 5 year old unsupported version of Rails/MySQL. GitHub runs from the Rails main branch - the latest stable version they possibly can - and they update…

I wasn't really blaming rails per say, if anything their main database mysql1 seems to pop up in their post mortems more then anything else

Re: GitHub having issues today

#45

Post by darkwater »

mrweasel wrote: post #11
robinhoodexe wrote: post #6

And just as we're about to migrate 4 kubernetes clusters with a total of ~4k pods. Terraform in github actions on selfhosted runners and argoCD is failing.

Oh that sucks, there's always going to be those who will say that it's the price you pay for using Github, but locally hosted VCS and CI/CD systems have issues as well. External dependencies are always problem, but do you have the capacity and resources required to manage those dependencies internally? Most don't and will still get a better product/service by using an external service.

> Most don't

Define "most". There is a surprisingly high number of small/mid-sized companies which have dedicated people for this kind of things.

Re: GitHub having issues today

#46

Post by 8organicbits »

djbusby wrote: post #35
amelius wrote: post #21

In the old days we had mirrors for many online repositories.

Yep. Now I'm glad I vendor-mirror all my dependencies so I can keep doing tests and all that.

I wonder if projects should proactively mirror to another git site (github as main and gitlab as mirror, for example). Collaboration on the project may stop when the main is down, but consumers could proceed using the mirror. I'm not sure how well various tooling supports fallback origins. This would reduce (but not eliminate) the need for users to vendor their dependencies.

Re: GitHub having issues today

#47

Post by berkes »

izietto wrote: post #39
dtech wrote: post #27

Earlier quoted context omitted.

One difference is that Rails and MySQL on a Github scale is rare, even when taking into account Github scale is rare.

you mean that most of the other popular huge Rails companies (GitLab, Shopify) use PostgreSQL? Basecamp uses MySQL tho

Gitlab is, AFAIK, nowhere near the scale of Github. Shopify IDK, but I'm fairly certain their type of usage is very different. Basecamp is also of another scale, and certainly a very different usage/performance type.

Re: GitHub having issues today

#48

Post by sonicanatidae »

acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

Sometimes it's not Ops. Sometimes its crap code. ;)

Source: <-- OPs

Re: GitHub having issues today

#50

Post by chaxor »

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution.

This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab

Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weights or databases that are too large for git and cost too much on other services, as they would be free on ipf/torrent.

Then no one is phased by the inevitable failure of various companies.

Post reply on HN