Live data from Hacker News

GitHub having issues today

githubstatus.com

51–60 of 89 posts

Re: GitHub having issues today

#51
post #42

Earlier quoted context omitted.

Every time there's a GitHub outage of any severity one of the top comments on HN is inevitably suggesting that it's probably due to Rails. It's getting pretty tiresome. Calling it a "legacy rails" stack is incredibly disingenuous as well. It's not like they're running a 5 year old unsupported version of Rails/MySQL. GitHub runs from the Rails main branch - the latest stable version they possibly can - and they update…

It's easier to blame than to admit running a service at Github's scale is highly complex and takes deep expertise.

It's also that many Rails shops have performance problems: which isn't the same as saying "Rails is slow"!!. "Getting performance problems at some point" is almost a rite of passage in Rails; I'm certain every rails developer has pored over N+1 queries, caching, async jobs, race-conditions, gems and whatnot to keep the system running.

The only Rails projects that I worked on that never had performance problems are the ones that never reached any scale. All Rails projects that gained traction that I worked on, needed serious refactorings, partial rewrites, tuning and tweaking to keep 'em running. If only to tame the server-bills, but most of the times to just keep the servers up. Good news is that it's very doable to tune, tweak and optimize a Rails stack. But the bad news is that every "premature optimization is the root of all evil" project made a lot of choices back in the days that make this nessecary optimization today hard or impossible even.

What I'm trying to say is: Performance issues with Rails will sound very familiar to anyone who worked seriously with Rails. So it's not so strange that people reach for this conclusion if almost everyone in the community has some first-hand experience with this conclusion.

Re: GitHub having issues today

#53
post #33
post #21

In the old days we had mirrors for many online repositories.

We've lost the technology for decentralised Internet services in the early 2000s. We just hope SMTP keeps ticking along somehow or we're fcuked.

Teams, Slack, Discord, whatsapp, imessage, etc are trying hard, to replace that too, though,

Re: GitHub having issues today

#54
post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

I don't think you understand ops :), there's no 100% availability anywhere, so issues and degradation will always happen no matter what. https://sre.google/sre-book/service-level-objectives/

It's not rails nor MySQL, both proven good for years.

Re: GitHub having issues today

#56
post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Git is already decentralized....

Re: GitHub having issues today

#57
post #14
post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

Why people think it can be related to Rails when there are tons of companies out there using Rails not affected by this degradation?

>Why people think it can be related to Rails

Probably because there have been more high-profile stories of companies migrating off of Ruby on Rails to something else (e.g. Java, Go, etc) rather than vice-versa of migrating into it.

E.g. the high-profile story of Twitter's previous "whale fail" scaling problems supposedly being partially solved by switching from Ruby to Java/Scala/JVM : https://www.google.com/search?q=twitter+whale+fail+ruby+rail...

Ruby may be unfairly blamed but nevertheless, the narrative is already out there even though other big sites like Shopify, etc still use it.

Re: GitHub having issues today

#59
post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Ah yes, because version control is only about file storage.

Re: GitHub having issues today

#60
post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Git is already decentralized....

So is cryptocurrency, but people still lose their money when exchanges get hacked. :D It's a cry for help. "Make decentralization as convenient as centralization!"
Post reply on HN