Nassim Taleb: We should retire the notion of standard deviation

It took me ten seconds to figure out what 'Goldstein and Taleb' refers to: http://www-stat.wharton.upenn.edu/~steele/Courses/434/434Con...

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

He specifies that SD is reasonable as a mathematical tool, but not for inference about society, finance, etc.

Your comment appears to have an agenda or ideology underlying it.

Further you don't seem to have read the article fully.

>Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations. THis information is useful for many things: risk estimation or calculating sample size needed for required confidence (if you need more experiments, how careful should you be with conclusions and predictions etc). He didn't mention how are we going to achieve those with his proposal. Meanwhile he managed to throw insults towards various groups without giving one single example of misuse he describes.

That is pretty much what his entire article is about: taking squares may not be the best idea universally.

Taleb is definitely mad but his use of the MAD acronym (mean absolute deviation) is actually correct. However the STD acronym (all caps) refers to "sexually transmitted disease" and not generally used for "standard deviation". Most people use SD, Stdev, StDev or sigma.

Once again his ability to coin new terminology outstrips his ability to form coherent ideas that are anything more than trivial (eg. we have known about fat tails in stock returns for 50+ years). Like George Soros[1], Taleb's success says more about the state of the world of finance than their contributions to our knowledge.

[1]-See his book "The Alchemy of Finance"

> What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ?

He is exaggerating, for sure. But the point is valid: the mean average deviation (MAD) is often very different than the standard deviation (STD), and the MAD is more intuitive, it has a natural geometrical interpretation - STD's usage of squaring the distance makes it more complex.

And yes, this confuses people in some cases, including scientists. Many scientists are not statistical experts, they use tools as they were taught, and they often assume MAD is approximately the STD, because it usually is, except in rare cases when it is not. I've seen examples of those people in grad school, he is not making this up.

The STD is far more easy to analyze in a mathematical way. That is the huge value it brings - squaring is an operation you can take the derivative of, but absolute value you cannot. STD gives us nice properties like easily provable sum of variances is the variance of the sum, for independent variables.

MAD, however is nicer for reporting data since it is more intuitive. I think he makes a valid point that STD is used more frequently than it should be.

> Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations.

His point is that many people are not aware of that property and do not want it.

I am saying that pointing out that some people misuse some tools is not interesting without specific examples and problems it causes.

>It is however important if large groups of disciplines are not able to apply it properly.

Of course it is, let's try to identify such situations... o just let's try to find one problematic example.

>It might not be interesting to you and I am sure you know how to apply them. It is however important if large groups of disciplines are not able to apply it properly. Instead of calling him a vile idiot and refusing to even look at what he is trying to say you could see this statement as a cry for help, a provocation to start a debate and so on.

I've read his 3 books and listened to many talks. He just doesn't have anything specific to say. If he is crying for help as you said why not describe some situations where using standard deviation instead of MAD causes problems ? I am not refusing to listen to him. The fact that he writes about those problem is the reason I invested a lot of time reading his books despite terrible writing and lack of editing. I was willing to go through just to see what he has to say. It turns out there is nothing, not one example of systematic error people make which could be corrected. It's just hate for anything and everything often paired with failure to understand the most basic statistical concept.

>The deeper issues here, I think is that he is probably not death wrong nor death right and so it's really impossible to prove anything objectively but rather different experiences yield different opinions.

He is dead wrong about standard deviation not being useful for "real life". He is probably right about "some people misuses it some of the time thing" but it's really not interesting unless you give something specific which he never does.

>I personally believe that sometimes you have to simplify in order to get a fundamental discussion started.

I agree. The problem is not simplifying he just doesn't have anything to say. It's not one article without examples or arguments. His whole book is like that and then his talks.

> In fact i find the aftermath of those provocations to be the most enlightening.

And I see danger in it. The guy has some serious following. It can't be good if people read his stuff and start believing that those risk managers are just morons, that scientists misuse standard deviation because of misunderstanding in 1893 and all other ridiculous things he claims. You can really gain an impression that everybody is a moron, academia is a waste and math doesn't apply if you don't know better and take his writing seriously.