Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub was down

status.github.com

Post reply on HN
Print view View original

141–150 of 253 posts

Re: GitHub was down

#141

Post by groby_b »

brodock wrote: post #133
ericfrederich wrote: post #113

Earlier quoted context omitted.

I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch. GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.

DISCLAIMER: I work for GitLab. We have a feature to replicate GitLab to different locations (EEP). We call it "Geo", which stands for "Geographical Replication". Part of the "Geo" effort is Disaster Recovery which is under heavily development. We want with Disaster Recovery to be able to reliably promote any Secondary node to a Primary, so if your US datacenter melts down under a nuclear war, you can start working wi…

If your US datacenter melts down under a nuclear war, I can guarantee that none of your devs will give a damn about contributing to the code base.

You might want to modify your scenarios.

Re: GitHub was down

#142

Post by optimuspaul »

brodock wrote: post #133
ericfrederich wrote: post #113

Earlier quoted context omitted.

I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch. GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.

DISCLAIMER: I work for GitLab. We have a feature to replicate GitLab to different locations (EEP). We call it "Geo", which stands for "Geographical Replication". Part of the "Geo" effort is Disaster Recovery which is under heavily development. We want with Disaster Recovery to be able to reliably promote any Secondary node to a Primary, so if your US datacenter melts down under a nuclear war, you can start working wi…

(only partially serious here) I'm not sure recovery from a nuclear event that wipes out US data centers is economically feasible. If an event were to effectively wipe out 20% of the worlds economy I think having my github issues online is the least of my worries.

Re: GitHub was down

#143

Post by s73ver »

tambourine_man wrote: post #82

Insert remark on why we use a centralized service for a distributed source control system, etc. No one seems to care, unfortunately

Because most things are easier when you can have one canonical source of truth.

Re: GitHub was down

#144

Post by cwingrav »

pmoriarty wrote: post #127

Where is github hosted? Do they use AWS or another commercial cloud provider, or do they have their own servers in data centers (hopefully scattered around the globe)? If AWS, are their services spread among multiple availability groups? I'm just wondering how this could happen.

Here's this from 2009. (https://github.com/blog/530-how-we-made-github-fast)

I says Rackspace, but 2009 is a long time ago so it could have changed.

Re: GitHub was down

#146

Post by DaiPlusPlus »

jasode wrote: post #112
tambourine_man wrote: post #82

Insert remark on why we use a centralized service for a distributed source control system, etc. No one seems to care, unfortunately

>Insert remark on why we use a centralized service for a distributed source control system, Because Linus didn't put features into Git that Github solves. You must break apart the different features of Github: #1) communication (issues tracking, bug reports, pull requests, README.MD landing page, etc) #2) hosting disk storage & bandwidth #3) distributed source code merges based on content hashes (SHA1) instead of usi…

#1 can be done by creating a separate submodule repo that only stores docs+issues files. It's up to the repo's users to agree on a system by which the files should be organized, but it's doable.

I'd propose directories "issues/open", "issues/closed", with each issue filename being "{created:yyyy-MM-dd} - {subject}.md". Symlinks could be used to track ownership/responsibility if each repo contributor has their own directory in the repo too.

Re: GitHub was down

#147

Post by paublyrne »

0xbear wrote: post #116
cjbprime wrote: post #81

Earlier quoted context omitted.

You're still missing the point, by assuming that the outages are code related when you call them avoidable. Distributed systems at scale are terrifyingly hard to control. When I said "probably a fairly rare one", I didn't mean that the outages are rare. I meant that new code is probably a rare cause of the outages that happen. They have other causes unrelated to new code. (I'm also skeptical that GitHub major outages…

I'm not "missing" anything. I worked at Google for 7 years much of which was spent working on, you guessed it, distributed systems infrastructure. You guard against this by carefully canarying things and putting robust testing, monitoring, and deployment procedures in place. A release might take a few days, but you can be reasonably certain your users won't be your guinea pigs, and if shit does hit the fan, rollback…

So no Google service has ever experienced an outage? I distinctly remember Gmail being down on several occasions.

Re: GitHub was down

#148

Post by kornish »

bradfitz wrote: post #93
tinalumfoil wrote: post #54

Earlier quoted context omitted.

> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

Go has a mirror of our GitHub project via this thing "Maintner" I wrote (running at http://maintner.golang.org/ ) that syncs GitHub in realtime to a log of mutations. (As well as syncing Gerrit and all its comments etc). So then we can slurp all of our GitHub & Gerrit history into RAM (takes about 5 seconds and 500 MB) via https://godoc.org/golang.org/x/build/maintner/godata#Get and walk it in-memory and do stuff wit…

> alternate web UIs on planes

Could you expand on this? Sounds interesting. You mean like an offline UI powered by the in-memory mutation list?

Re: GitHub was down

#149

Post by [deleted] »

tinalumfoil wrote: post #54
kaikai wrote: post #27

I would love to see a chart of traffic to other sites when GitHub goes down. My bet is that HackerNews and Twitter both get significant spikes from all those bored developers.

> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

[deleted]

Re: GitHub was down

#150

Post by adventist »

brodock wrote: post #133
ericfrederich wrote: post #113

Earlier quoted context omitted.

I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch. GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.

DISCLAIMER: I work for GitLab. We have a feature to replicate GitLab to different locations (EEP). We call it "Geo", which stands for "Geographical Replication". Part of the "Geo" effort is Disaster Recovery which is under heavily development. We want with Disaster Recovery to be able to reliably promote any Secondary node to a Primary, so if your US datacenter melts down under a nuclear war, you can start working wi…

Yeah I know I'll be on vacation for a long while!
Post reply on HN