Live data from Hacker News

January 28th Incident Report

github.com

41–50 of 198 posts

Re: January 28th Incident Report

#41
post #34

Earlier quoted context omitted.

If you look back through the years and find a few other stories of "GitHub is down", you'll see that this conversation happens every time. Some people tread into the HackerNews thread and say "More people should be using self-hosted GitLab instances" or "if GitHub would just open source their code, we wouldn't need to be so dependent." But then the conversation stops within days because, the fact is, hosting your own…

hosting your own git servers and getting people to actually use them is a huge pain in the ass. More simply put: people just like using GitHub. Given that this is the case, I fell like GitHub is entrenched enough that they could open source their codebase and not lose any customers. People are paying them for the convenience of someone else hosting their git repository.

Technically you dont need a server. You just need access to remotes. That could be your team's repos. Then PRs just mean pulling from a coworker to your repo.

But yes, most of us are most comfortable with the central repository model.

Re: January 28th Incident Report

#42

Every time I read about a massive systems failure, I think of Jurassic Park and am mildly grateful that the velociraptor padock wasn't depending on the systems operation.

Well as long as you're not Samuel L. Jackson in that scenario you should be fine. Ish.

Re: January 28th Incident Report

#46
post #35
post #24

Earlier quoted context omitted.

Maybe. BitBucket was also an acquisition so for some time I believe there was a lack of resource provided to them and there was a huge technical debt/integration effort required. At this very time, I don't know if Atlassian actually care much about BitBucket. They are probably more concerned about delivering Stash than BitBucket, my wild guess. I was an active BB user a couple years ago, and the project I worked on w…

FYI, Stash is now Bitbucket Server, and the plan as I've heard it is to work towards feature parity between the two.

We use Stash and it is surprisingly not bad at all. Github is much more polished but for code browsing and review, it does that which it is supposed to do.

Re: January 28th Incident Report

#49
post #32

Does Github run anything like Netflix Simbian Army against it's services? As a company by engineers for engineers with the scale that github has reached, I'm a bit surprised they are lacking a bit more redundancy. Though they may not need the uptime of netflix, an outage of more than a few minutes on github could affect businesses that rely on the service.

[deleted]

[deleted]

Re: January 28th Incident Report

#50
post #3

This just shows how difficult it is to avoid hidden dependencies without a complete, cleanly isolated, testing environment of sufficient scale to replicate production operations and do strange system fault scenarios somewhere that won't kill production.

Or use the Netflix model: Chaos testing in production.

No system is perfect; as you continue to add 9s, the cost increases steeply.

Usually its just cheaper to be down for an hour or two, versus architect for the end of times.

Post reply on HN