GitHub Outage • Hacker News Board

Re: GitHub Outage

#221

Post by niutech » Thu, Jan 28, 2016, 2:09 AM UTC

Perceptes wrote: post #170

Every time this happens people make clever remarks about how Git is distributed but we're all depending on GitHub for so much that we defeat the purpose. But once GitHub comes back up, everyone just gets back to work, trusting and relying on it as much as ever. Eventually it goes down again, and we come back to complain. Convenience is the only thing that we seem to value. (I'm no different, which makes my comment co…

Companies/people don't learn on their mistakes. Almost everything is on GitHub nowadays. This makes it a SPOF even if Git itself is distributed. More companies should host their projects on premises. There are good open source alternatives to GitHub: Apache Allura, Fossil, GitBucket, GitLab, Phabricator, and Redmine.

Re: GitHub Outage

#222

Post by edem » Thu, Jan 28, 2016, 2:09 AM UTC

Analemma_ wrote: post #92

sigh Many moons ago when I was a starry-eyed lad just learning to use Git, I remember all the cheerful sentences in the Git book like "Unlike SVN, with Git you can work even when the server is down!" The more things change, the more they stay the same.

You can continue to work. For example I skipped pushing to Github but I can push to Heroku. No problem. I'll push my code when it is up again.

Re: GitHub Outage

#223

Post by asumanth » Thu, Jan 28, 2016, 2:11 AM UTC

If you were stuck like me looking up an OSS github project, http://archive.org/web/ has github pages.

Re: GitHub Outage

#224

Post by [deleted] » Thu, Jan 28, 2016, 2:11 AM UTC

rdoherty wrote: post #100

ikawe wrote: post #27

Sorry, I think I caused this. =[ bower jquery#1.11.3 not-cached git://github.com/jquery/jquery-dist.git#1.11.3 bower jquery#1.11.3 resolve git://github.com/jquery/jquery-dist.git#1.11.3 bower foundation#~5.5.2 cached git://github.com/zurb/bower-foundation.git#5.5.3 bower foundation#~5.5.2 validate 5.5.3 against git://github.com/zurb/bower-foundation.git#~5.5.2 bower ember#^2.3.0 ECMDERR Failed to execute "git ls-remo…

Can someone explain this to people who don't use bower?

[deleted]

Re: GitHub Outage

#225

Post by postila » Thu, Jan 28, 2016, 2:12 AM UTC

zymhan wrote: post #164

chuhnk wrote: post #94

Earlier quoted context omitted.

I once ran rm -rf in the production mysql data directory. Shit happens.

I did that too. Destroyed out Zabbix database. Neither that Zabbix server, nor the other one monitoring the server I destroyed, could alert us that anything had gone wrong for over an hour. I finally realized it when I couldn't login... I was able to painstakingly rebuild the server after 9 hours without anyone noticing. To this day one of my biggest fuck ups and prouder accomplishments.

Are you saying that once you break something, you should break your monitoring as well, but do it very quickly since it may be too late? :)

Re: GitHub Outage

#226

Post by Ellahn » Thu, Jan 28, 2016, 2:12 AM UTC

kelseydh wrote: post #196

Perceptes wrote: post #170

Every time this happens people make clever remarks about how Git is distributed but we're all depending on GitHub for so much that we defeat the purpose. But once GitHub comes back up, everyone just gets back to work, trusting and relying on it as much as ever. Eventually it goes down again, and we come back to complain. Convenience is the only thing that we seem to value. (I'm no different, which makes my comment co…

The bigger problem here is the number of build packs only relying on Github as a source of truth. We need to find an abstraction to distribute storage so that there is no single point of failure.

Exactly.

So I had to install Gems from RubyGems, not that big of a deal, and I had to look them up on RubyGems since Google gives me GitHub first, but that's OK too, but then all the documentation seems to be on.... Github. Except not, it's also on RDoc. (Though RDoc kinda sucks, compared to GitHub...)

Pretty big win imho, Rails got a few more points with that. :D

Now I'm wondering how NodeJS is faring at this...

Re: GitHub Outage

#227

Post by imsofuture » Thu, Jan 28, 2016, 2:13 AM UTC

robertrobert wrote: post #184

Two little birdies have tweeted that there was an electrical fire at a key datacenter.

We had networking issues on Rackspace earlier today...

Re: GitHub Outage

#228

Post by code4craft » Thu, Jan 28, 2016, 2:13 AM UTC

beat wrote: post #59

debacle wrote: post #19

I'm kind of tired of paying for a service that has so many outages.

Go to bitbucket, or self-host, or find some other shiny new competitor. The market speaks with its feet.

http://git.oschina.net/ is a good choice and it's private repo is also free.

Re: GitHub Outage

#229

Post by cs2818 » Thu, Jan 28, 2016, 2:14 AM UTC

ntaylor wrote: post #33

Thank God we're on Bitbucket! Right, guys? Anyone?

Yeah, our time to shine has finally come!

We use Bitbucket in our ~30 member academic robotics research lab, it's wonderful because they are nice enough to supply us with as much as we need (private repos, teams, etc.) for free!

Re: GitHub Outage

#230

Post by philip1209 » Thu, Jan 28, 2016, 2:15 AM UTC

This is reminding me of last year's major Facebook outage. If I recall correctly, that outage was a bug in service discovery that took down all data centers (a CLI accepted a negative value when the Zookeeper variant treated it as an unsigned int , then all service discovery went down). I feel like service discovery is the biggest point of failure at large companies, and it would explain why services across so many different domains and systems went down.