131–140 of 253 posts
Where is github hosted? Do they use AWS or another commercial cloud provider, or do they have their own servers in data centers (hopefully scattered around the globe)? If AWS, are their services spread among multiple availability groups? I'm just wondering how this could happen.
Earlier quoted context omitted.
You're still missing the point, by assuming that the outages are code related when you call them avoidable. Distributed systems at scale are terrifyingly hard to control. When I said "probably a fairly rare one", I didn't mean that the outages are rare. I meant that new code is probably a rare cause of the outages that happen. They have other causes unrelated to new code. (I'm also skeptical that GitHub major outages…
I'm not "missing" anything. I worked at Google for 7 years much of which was spent working on, you guessed it, distributed systems infrastructure. You guard against this by carefully canarying things and putting robust testing, monitoring, and deployment procedures in place. A release might take a few days, but you can be reasonably certain your users won't be your guinea pigs, and if shit does hit the fan, rollback…
Earlier quoted context omitted.
> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.
I agree... I wish GitLab would do this too. No reason that everything can't be modeled as a git branch. GitLab I know models their Wiki as a git repo. I think issues and PRs should be their own repo as well.... a branch for each issue or pr? Could tie it all up using submodules (or not, whatever)... just please someone take the jump first and do this.
Earlier quoted context omitted.
You're still missing the point, by assuming that the outages are code related when you call them avoidable. Distributed systems at scale are terrifyingly hard to control. When I said "probably a fairly rare one", I didn't mean that the outages are rare. I meant that new code is probably a rare cause of the outages that happen. They have other causes unrelated to new code. (I'm also skeptical that GitHub major outages…
I'm not "missing" anything. I worked at Google for 7 years much of which was spent working on, you guessed it, distributed systems infrastructure. You guard against this by carefully canarying things and putting robust testing, monitoring, and deployment procedures in place. A release might take a few days, but you can be reasonably certain your users won't be your guinea pigs, and if shit does hit the fan, rollback…
It's a startup site with half the people not having a test environment.
Earlier quoted context omitted.
I keep meaning to dig into Fossil (SQLite's VCS/Project Management system), but I have no faith I could convince a team to use it. Another model to look at is Trac, which had pretty extensive integration with SVN and integrated (ie, cross-linked) issue tracking and wiki, and stored all the data and change history in a svn repository.
> I keep meaning to dig into Fossil, but I have no faith I could convince a team to use it. Developer or Fossil and SQLite here: I agree. In my experience, you'd have better luck convincing the team to switch from vi to emacs. For all its many and well-documented faults, the Git/GitHub paradigm is what people want to use because it is what they are familiar with. All the same, I intend to keep right on using Fossil,…
Earlier quoted context omitted.
Go has a mirror of our GitHub project via this thing "Maintner" I wrote (running at http://maintner.golang.org/ ) that syncs GitHub in realtime to a log of mutations. (As well as syncing Gerrit and all its comments etc). So then we can slurp all of our GitHub & Gerrit history into RAM (takes about 5 seconds and 500 MB) via https://godoc.org/golang.org/x/build/maintner/godata#Get and walk it in-memory and do stuff wit…
> Maintner is short for "Maintainer"...the name of the daemon that serves the maintner data to other tools is "maintnerd". Nice work! But as is custom on HN, I'll bikeshed on the name instead of delving into the contents of the tool. Why shorten the word by just 2 letters? Is there something special about the tooling that makes 8-letter projects more desirable than 10-letter projects, or is it linked to the removal o…
I would love to see a chart of traffic to other sites when GitHub goes down. My bet is that HackerNews and Twitter both get significant spikes from all those bored developers.
> bored developers Bored? Git's a distributed version control system, so no excuses. Get back to work! But in all seriousness I kind of wish GitHub provided a way to mirror things like issues and PRs so you never have to be fully reliant on one service. Not being able to read these really does make it impossible to get work done offline.
Earlier quoted context omitted.
They seemed to be suffering some external attacks / DDoS but I never saw a post-mortem from them on it, hopefully one is forthcoming (or maybe is out but I missed it)
They don't do post mortems.
Where is github hosted? Do they use AWS or another commercial cloud provider, or do they have their own servers in data centers (hopefully scattered around the globe)? If AWS, are their services spread among multiple availability groups? I'm just wondering how this could happen.
I am now feeling less informed than I was after the first reply.
How many more can we expect before they develop appreciation for testing _before_ they push to prod?