Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GitHub was down

status.github.com

Post reply on HN
Print view View original

111–120 of 253 posts

Re: GitHub was down

#111

Post by LeifCarrotson »

bradfitz wrote: post #93
tinalumfoil wrote: post #54

Earlier quoted context omitted.

> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

Go has a mirror of our GitHub project via this thing "Maintner" I wrote (running at http://maintner.golang.org/ ) that syncs GitHub in realtime to a log of mutations. (As well as syncing Gerrit and all its comments etc). So then we can slurp all of our GitHub & Gerrit history into RAM (takes about 5 seconds and 500 MB) via https://godoc.org/golang.org/x/build/maintner/godata#Get and walk it in-memory and do stuff wit…

> Maintner is short for "Maintainer"...the name of the daemon that serves the maintner data to other tools is "maintnerd".

Nice work! But as is custom on HN, I'll bikeshed on the name instead of delving into the contents of the tool. Why shorten the word by just 2 letters? Is there something special about the tooling that makes 8-letter projects more desirable than 10-letter projects, or is it linked to the removal of Artificial Intelligence from the process? I'd personally mis-type that name all the time.

Re: GitHub was down

#112

Post by jasode »

tambourine_man wrote: post #82

Insert remark on why we use a centralized service for a distributed source control system, etc. No one seems to care, unfortunately

>Insert remark on why we use a centralized service for a distributed source control system,

Because Linus didn't put features into Git that Github solves.

You must break apart the different features of Github:

#1) communication (issues tracking, bug reports, pull requests, README.MD landing page, etc)

#2) hosting disk storage & bandwidth

#3) distributed source code merges based on content hashes (SHA1) instead of using centralized locks/unlocks (check in / check out) model of CVS/SVN.

Git itself only takes care of #3.

Github handles #1 and #2 (and also gets #3 by being built on top of Git).

You can't go back in time and wonder if Linus should have addressed #1 and #2 because he wasn't interested in starting a hosting company. Instead, he focused on the data format (Merkle trees, BLOBs, SHA1) and a sync protocol (git pull, etc) for Git.

If people wonder why we can't just use email for #1 (communications), you have to see that Github has become a "Schelling Point"[1]. Attempting to use email groups & mailing lists will not prevent the emergence of a Schelling point. Email can be a workflow for existing contributors (e.g. contributors of the Linux kernel source) but it's not convenient for discovery of new repositories (e.g. the web's "landing page" of a repo).

As for #2 (hosting), not everybody who wants to share a repository wants to pay $9.99/month VPS or other hosting plan from a web hosting provider. It would also be inconvenient to host it from the home laptop and punch a hole through the ISP router to make it work. Github solves hosting+bandwidth for free for modest non-commercial projects.

To restate, Linus' Git is a distributed _protocol_ but Github is a _service_ acting as a platform for the distributed protocol.

[1] https://en.wikipedia.org/wiki/Focal_point_(game_theory)

Re: GitHub was down

#113

Post by ericfrederich »

tinalumfoil wrote: post #54
kaikai wrote: post #27

I would love to see a chart of traffic to other sites when GitHub goes down. My bet is that HackerNews and Twitter both get significant spikes from all those bored developers.

> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.

I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch.

GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.

Re: GitHub was down

#115

Post by macintux »

LeifCarrotson wrote: post #111
bradfitz wrote: post #93

Earlier quoted context omitted.

Go has a mirror of our GitHub project via this thing "Maintner" I wrote (running at http://maintner.golang.org/ ) that syncs GitHub in realtime to a log of mutations. (As well as syncing Gerrit and all its comments etc). So then we can slurp all of our GitHub & Gerrit history into RAM (takes about 5 seconds and 500 MB) via https://godoc.org/golang.org/x/build/maintner/godata#Get and walk it in-memory and do stuff wit…

> Maintner is short for "Maintainer"...the name of the daemon that serves the maintner data to other tools is "maintnerd". Nice work! But as is custom on HN, I'll bikeshed on the name instead of delving into the contents of the tool. Why shorten the word by just 2 letters? Is there something special about the tooling that makes 8-letter projects more desirable than 10-letter projects, or is it linked to the removal o…

I'd prefer it just for searchability. Good luck finding anything online called "maintainer".

Re: GitHub was down

#116

Post by 0xbear »

cjbprime wrote: post #81
0xbear wrote: post #64

Earlier quoted context omitted.

It's a nearly weekly occurrence. Avoidable one, I might add. When was the last time a major Google service had a major outage?

You're still missing the point, by assuming that the outages are code related when you call them avoidable. Distributed systems at scale are terrifyingly hard to control. When I said "probably a fairly rare one", I didn't mean that the outages are rare. I meant that new code is probably a rare cause of the outages that happen. They have other causes unrelated to new code. (I'm also skeptical that GitHub major outages…

I'm not "missing" anything. I worked at Google for 7 years much of which was spent working on, you guessed it, distributed systems infrastructure. You guard against this by carefully canarying things and putting robust testing, monitoring, and deployment procedures in place. A release might take a few days, but you can be reasonably certain your users won't be your guinea pigs, and if shit does hit the fan, rollback is easy, and you can reroute traffic elsewhere while you roll back. Most of the time no rollback is needed: you just flip a flag and do a rolling restart on the job in Borg. For some types of outages (most of which users never even see) Google has bots that calculate monetary loss. And the figures can be quite staggering and motivating, so people do postmortems and try their best to make sure the outages don't happen again.

Re: GitHub was down

#117

Post by Goladus »

xutopia wrote: post #100
Goladus wrote: post #98

Earlier quoted context omitted.

The reason is github did a fantastic job of implementing useful features. The visual design is unmatched and they have done a great job implementing developer oriented integrations and social features. A more federated approach to this sort of thing might have been nice, but so far nothing I have seen comes close to the value-add offered by github.

You're sidestepping the main reason I believe it worked so well. It benefits from network effect. It is a collaborative tool and people like to have their work on there so others can collaborate with them.

That's important but I think people over-estimate it. It's both. I predict that if you analyzed the github network, you'd find many hubs are based around companies that chose to move their workflows to github based on features other than network effects. Or at least, the existing network was only one of many reasons.

Re: GitHub was down

#118

Post by alexchamberlain »

When I break our GitHub webhooks, I joke it's time for people to practice our Disaster Recovery (DR) procedures. In all seriousness, this is a good opportunity to practice work without GitHub. Any service can go down; can you deploy a critical bug fix without it? If not, why not and what can you do to fix it?
Post reply on HN