If data is drawn from a Laplace distribution of the form p(x) = exp(-|x|), the mean absolute deviation is more informative than the standard deviation, but if its form is close to the normal, p(x) = exp(-x^2), the standard deviation is more important. So whether to use the mean absolute or standard deviation depends on the distribution of the data. There is a field called robust statistics that looks at this question.
Nassim Taleb: We should retire the notion of standard deviation
171–180 of 254 posts
Re: Nassim Taleb: We should retire the notion of standard deviation
#172Climate scientists--among others--have made similar recommendations to use the absolute mean error in place of the standard deviation, depending on the application. Taleb might have cited the extensive methodological literature--for example:
Cort J. Willmott, Kenji Matsuuraa, Scott M. Robeson. Ambiguities inherent in sums-of-squares-based error statistics. Atmospheric Environment 43 (2009) 749–752.
URL: http://climate.geog.udel.edu/~climate/publication_html/Pdf/W...
Re: Nassim Taleb: We should retire the notion of standard deviation
#173Earlier quoted context omitted.
>> The notion of standard deviation has confused hordes of scientists > What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? He is exaggerating, for sure. But the point is valid: the mean average deviation (MAD) is often very different than the standard deviation (STD), and the MAD is more intuitive, it has a natural geometrical interpretati…
> and the MAD is more intuitive, it has a natural geometrical interpretation It's less intuitive, not more intuitive, at least for me. And the standard deviation definitely has a more geometric interpretation than MAD. If you measure a hundred samples, and you want to figure out how much they differ from the expected values, what could be more intuitive than Euclidean distance? But most people never bother to try and…
Maybe I misunderstood what you meant, but I would say that the L_2 norm amounts to MAD. L2 means
sqrt(x²+y²+...)
sqrt(x²) = |x|Re: Nassim Taleb: We should retire the notion of standard deviation
#174at least he's leaving us physicists alone with it...
Re: Nassim Taleb: We should retire the notion of standard deviation
#175Earlier quoted context omitted.
Yeah a polyglot who writes classical Greek, Arabic, French, English and can do advanced statistical modeling is clearly an idiot right?
Well, it surely looks like it seeing level of argument he presents and lack of logical consistency in his writing. Even in this article he claims SD doesn't model real world as well as MAD and then as arguments ask you what would you do if you were asked to calculate... MAD, you would clearly see it's not SD :-) His latest book is worse than the article and my conclusion about him being and idiot is based mainly on i…
I don't see a problem with the sentence "Much of aging comes from a misunderstanding of the effect of comfort - a disease of civilization". I read it like this: people think comfort is good and healthy and prolongs life, so they seek comfort and get fragile/sick - they misunderstand the effect of comfort.
Re: Nassim Taleb: We should retire the notion of standard deviation
#176Earlier quoted context omitted.
Maybe I'm the idiot, but I found the book's point to be pretty trivial. "Sometimes unexpected things happen, and people are bad at expecting the unexpected" seems pretty damn trivial to me. Maybe it's because I think like a programmer rather than a finance guy, but a large portion of what I do on a daily basis is about mitigating risk from the unknown. A programmer who only guards against known risks is going to get…
I agree with you except that you left out the only really important (in my opinion) part of his pretty trivial message: "Sometimes unexpected things happen, and people are bad at expecting the unexpected, and then they FAIL TO NOTICE that they are bad at expecting the unexpected" It can be difficult for folks (myself included!) to separate their message from their irascible personalities.. ;-)
But if we were good at noticing that we're bad at expecting the unexpected, then we would expect the unexpected, making it no longer really all that unexpected, no? His whole argument just felt tautological to me.
Re: Nassim Taleb: We should retire the notion of standard deviation
#177Earlier quoted context omitted.
I don't understand calling him a "vile idiot" is so terrible. Vile is a subjective assessment in the eye of the beholder. Bluecalm began with a few paragraphs making the case that Taleb has pretensions of being an expert where he doesn't know what he is talking about, and that his statements are unsupported. Calling him a crank is a more objective statement, and to me seems to impute an unknowable motive to what Tale…
> I don't understand calling him a "vile idiot" is so terrible. Then you are a vile idiot.
Let's say it was worded instead: "... these are some reasons that he was wrong, and his style of argumentation leads me to feel actual antipathy toward him."
Maybe it is because "vile" doesn't sound like a very extreme word to me—almost like Daffy Duck saying "despicable". I suppose I would have had the same reaction, if instead bluecalm had denigrated him with a word that sounds more serious to me like "worthless".
Re: Nassim Taleb: We should retire the notion of standard deviation
#178It is sad that Taleb does not see the value in the standard deviation; standard deviation is far more natural, and more useful, than MAD.
For example, if X has a standard deviation of s, and Y has a standard deviation of t, then the standard deviation of X + Y is sqrt(s^2 + t^2). There is a geometry of statistics, and the standard deviation is the fundamental measure of length.
To retire the standard deviation is to ignore the wonderful geometry inherent in statistics. Covariance is one of the most important concepts in statistics, and it is a shame to hide it from those who use statistics.
Additionally, I will mention that we do not need normal distributions to make special the idea of standard deviations. In fact, it is the geometry of probability - the fact that independent random variables have standard deviations which "point" in orthogonal directions - which causes the normal distribution to be the resulting distribution of the central limit theorem.
Re: Nassim Taleb: We should retire the notion of standard deviation
#179Nassim Taleb somehow likes to beat up on normals...
We Bayesians have similar notions, but we usually try not to overly bully frequentist methods, the poor things. Also, being familiar with Bayesian methods, a lot of what Taleb is saying sounds vaguely familiar...
Re: Nassim Taleb: We should retire the notion of standard deviation
#180So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…
This guy sounds like a complete joke. Standard deviation is an immensely useful tool in stats. It's not a historical accident we use it everywhere, for example it naturally appears in the very useful Chebyshev inequality, and a million other contexts.