At Scale, Rare Events Aren’t Rare
perspectives.mvdirona.com
At Scale, Rare Events Aren’t Rare
1–10 of 59 posts
Re: At Scale, Rare Events Aren’t Rare
#2[deleted]
Re: At Scale, Rare Events Aren’t Rare
#3As a pretty good rule of thumb, a system that fails 1/nth of the time and has n opportunities to fail has ~.63 probability of failure, where n is more than ~10.
Graph: http://www.meta-calculator.com/online/?panel-102-graph&data-...
Re: At Scale, Rare Events Aren’t Rare
#4Related: Wired profile of the author
Re: At Scale, Rare Events Aren’t Rare
#5It reminds me of an old story about Microsoft Windows. Back in the early 2000's, compiling and building Windows from source code took many hours on very specialized build hardware. Meanwhile there were thousands of developers who contributed to the full Windows stack. If any developer checked in a build failure, it would cause the build to be delayed. Well, at that scale (of thousands of developers), you can't compile Windows even if devs only commit 1 build break per year. Bad times...
Re: At Scale, Rare Events Aren’t Rare
#6Coincidentally, a couple months ago one of my professors told me about James Hamilton, apparently, they met when he was studying at Waterloo. I started reading his blog.
This guy is brilliant. And his comments are often gems. The articles are good too but think of them as conversation openers. The real deal is in the comments imho. I recommend it. A somewhat funny article is about Zynga, the "prodigal kid" who left AWS for on-premises, only to come back later: http://perspectives.mvdirona.com/2015/05/the-return-to-the-c...
In the comments, he debunks (or take a shot at it) the idea that cloud providers like AWS aren't a good fit for organizations which have massive, but stable workloads.
This guy is so cool. I only wish he had more time to write.
Re: At Scale, Rare Events Aren’t Rare
#7Unfortunately most people don't realize this until they are at scale.
Re: At Scale, Rare Events Aren’t Rare
#8"One in a million is next Tuesday" from Larry Osterman is another great reflection on this from the software perspective. [1]
[1] https://blogs.msdn.microsoft.com/larryosterman/2004/03/30/on...
Re: At Scale, Rare Events Aren’t Rare
#9Unfortunately most people don't realize this until they are at scale.
It's ok, since most people never get "at scale". When you start getting these problems, it means you reached the stars you were shooting for. Having those issues are a good thing.
Re: At Scale, Rare Events Aren’t Rare
#10Unfortunately most people don't realize this until they are at scale.
It's ok, since most people never get "at scale". When you start getting these problems, it means you reached the stars you were shooting for. Having those issues are a good thing.
Assuming you have the resources to cover these events, sometimes you hit scale, but not profitability..