Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub having issues today

githubstatus.com

Post reply on HN
Print view View original

61–70 of 89 posts

Re: GitHub having issues today

#61

Post by dissident_coder »

berkes wrote: post #51
nijave wrote: post #42

Earlier quoted context omitted.

It's easier to blame than to admit running a service at Github's scale is highly complex and takes deep expertise.

It's also that many Rails shops have performance problems: which isn't the same as saying "Rails is slow"!!. "Getting performance problems at some point" is almost a rite of passage in Rails; I'm certain every rails developer has pored over N+1 queries, caching, async jobs, race-conditions, gems and whatnot to keep the system running. The only Rails projects that I worked on that never had performance problems are th…

> The only Rails projects that I worked on that never had performance problems are the ones that never reached any scale. All Rails projects that gained traction that I worked on, needed serious refactorings, partial rewrites, tuning and tweaking to keep 'em running.

You'll be hard pressed to find any stack that doesn't require this.

Re: GitHub having issues today

#62

Post by belter »

richardwhiuk wrote: post #56
chaxor wrote: post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Git is already decentralized....

But not GitHub... :-)

Re: GitHub having issues today

#63

Post by blauditore »

chaxor wrote: post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Can't tell if this comment is sarcastic, but that's exactly what git is: Every clone of the repo is independent, and acts as a full backup. Likewise, a local repo can be pushed to various remotes, there is no inherent strong server-client coupling (even though it's often used in such a way).

Re: GitHub having issues today

#64

Post by mrweasel »

darkwater wrote: post #45
mrweasel wrote: post #11

Earlier quoted context omitted.

Oh that sucks, there's always going to be those who will say that it's the price you pay for using Github, but locally hosted VCS and CI/CD systems have issues as well. External dependencies are always problem, but do you have the capacity and resources required to manage those dependencies internally? Most don't and will still get a better product/service by using an external service.

> Most don't Define "most". There is a surprisingly high number of small/mid-sized companies which have dedicated people for this kind of things.

Perhaps "many" would have been better. If you only count companies that view themselves as IT companies then number of companies with self hosted/managed solution grows, but if you include everything then I'd guess that more than 50% don't run these services internally. If you're every small company with one or two developers that doubles as the IT staff then the numbers add up pretty quickly.

Re: GitHub having issues today

#66

Post by gjsman-1000 »

AlchemistCamp wrote: post #10
acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

More likely, it's efforts to migrate away from the previously solid Rails stack to MS's preferred stack. They've had a long history of this kind of stability issue when migrating or trying to migrate acquisitions from their previous stack to an MS one. This happened with Hotmail (Unix server -> Windows server), LinkedIn (custom cloud -> MS cloud) and others since.

The LinkedIn to Azure migration was indefinitely postponed.

Re: GitHub having issues today

#67

Post by mrweasel »

sonicanatidae wrote: post #48
acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

Sometimes it's not Ops. Sometimes its crap code. ;) Source: <-- OPs

Clients I worked with: Our service crashed, why?

Because you designed and implemented it poorly, that's why. Alternatively: How should I know, you wrote it.

If you're ever bored as a developer, switch to operations, you get to be the person developers turn to when they can't code, debug, do logging or security.

Re: GitHub having issues today

#68

Post by chuckadams »

richardwhiuk wrote: post #56
chaxor wrote: post #50

Wouldn't it be wonderful if the most popular version control system was is decentralized? This is achievable, and is the correct solution. This way your git repo could be located on: - GitHub - Your Closet (...) - UCLA's supercomputer - JBOD in Max Planck Institute (...) - GitLab Doing this with a simple file with "[ipfs, github, gitlab]" on it would be revolutionary, especially for data version control, like nn weig…

Git is already decentralized....

I think what GP was getting at is that the topology is still point-to-point, which tends to lead to a hub-and-spoke system. Hell, it’s even in the name: GitHUB. And fact is, this works out for most people, but it leads to some undesirable failure modes. Maybe we need, say, a “git-discover-remote” command…

Re: GitHub having issues today

#69

Post by dijit »

ahmgeek wrote: post #54
acedTrex wrote: post #7

The fact that Github has been so unstable for so long is absolutely insane to me. I know ops is hard, but this level of consistent outage points to an endemic problem. Is it the legacy rails/mysql stack that is the largest culprit or is there systemic rot in the engineering org?

I don't think you understand ops :), there's no 100% availability anywhere, so issues and degradation will always happen no matter what. https://sre.google/sre-book/service-level-objectives/ It's not rails nor MySQL, both proven good for years.

please permit me to indulge the most extreme example of what you just said.

"What do you mean the database is down after I loaded 500 TiB and indexed all columns? It's MySQL, Facebook uses MySQL a high scale for years without incident!"

Post reply on HN