Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Taleb: We should retire the notion of standard deviation

edge.org

Post reply on HN
Print view View original

131–140 of 254 posts

Re: Nassim Taleb: We should retire the notion of standard deviation

#131

Post by msellout »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

>> There is no scientific reason to use it in statistical investigations in the age of the computer

> As someone who uses it daily I am eagerly awaiting his argument.

The use of root mean square error|deviation comes from the ease of calculation in the days before computers. Now that we have computers we can just use mean absolute error|deviation. Before computers, we needed calculus to find the "best fit line" -- parameters that create the least distance between predicted and actual values. Calculus plays well with parabolas but not so great with absolute value functions. Now that computers are around, we don't need calculus (as much).

Re: Nassim Taleb: We should retire the notion of standard deviation

#132

Post by pessimizer »

antiterra wrote: post #56
bluecalm wrote: post #29

Earlier quoted context omitted.

I just can't pass it. I gave my best at reading his book. The guy is unable to write one paragraph which doesn't contain some nonsense or serious misuse of established terms. He is insulting whole groups of people and institutions without giving any examples or research. I am pretty sure that if his stuff were posted without his name the conclusion would be obvious: it's just nonsense and very offensive at that. I re…

The guy is clearly not an idiot in the general sense; I bet he is of, at least, average intelligence. I also have my doubts that he is 'vile' for the majority of connotations and denotations of that word. Because of this, just shouting 'he's an idiot!' is vulnerable to being dismissed as hyperbole, and your baby gets thrown out with the bathwater. You seem exasperated at his condescending attitude, but make no effort…

>That said, even dismissing him as a 'crank' would add more substance to your argument than calling him a 'vile idiot.'

Neither "crank" or "vile idiot" would add substance to bluecalm's argument. "Vile idiot" isn't even part of the argument, it's part of the conclusion.

I disagree with bluecalm on the "vile idiot" conclusion - I think that he's either vile or an idiot. If he's really communicating how he sees the world, he's an idiot. If he's just cynically trying to come up with another Gladwell/Taleb style thesis that will catch fire with the NYT and TED crowd and bring in buckets of money, then he's vile.

Re: Nassim Taleb: We should retire the notion of standard deviation

#133

Post by tsax »

antiterra wrote: post #56
bluecalm wrote: post #29

Earlier quoted context omitted.

I just can't pass it. I gave my best at reading his book. The guy is unable to write one paragraph which doesn't contain some nonsense or serious misuse of established terms. He is insulting whole groups of people and institutions without giving any examples or research. I am pretty sure that if his stuff were posted without his name the conclusion would be obvious: it's just nonsense and very offensive at that. I re…

The guy is clearly not an idiot in the general sense; I bet he is of, at least, average intelligence. I also have my doubts that he is 'vile' for the majority of connotations and denotations of that word. Because of this, just shouting 'he's an idiot!' is vulnerable to being dismissed as hyperbole, and your baby gets thrown out with the bathwater. You seem exasperated at his condescending attitude, but make no effort…

Yeah a polyglot who writes classical Greek, Arabic, French, English and can do advanced statistical modeling is clearly an idiot right?

Re: Nassim Taleb: We should retire the notion of standard deviation

#135

Post by a8da6b0c91d »

brohee wrote: post #129
a8da6b0c91d wrote: post #44

Earlier quoted context omitted.

Eric Falkenstein's various Taleb takedowns are fun. http://falkenblog.blogspot.com/2009/03/fama-on-taleb.html http://falkenblog.blogspot.com/2012/11/taleb-mishandles-frag... http://falkenblog.blogspot.com/2010/12/nassim-taleb-imitates... http://falkenblog.blogspot.com/2009/03/review-of-talebs-blac...

Is it telling that in his review of Black Swann he's seriously citing Malcolm Gadwell of all people?

Your reading comprehension is questionable if you consider that a citation.

Re: Nassim Taleb: We should retire the notion of standard deviation

#136

Post by rz2k »

antiterra wrote: post #56
bluecalm wrote: post #29

Earlier quoted context omitted.

I just can't pass it. I gave my best at reading his book. The guy is unable to write one paragraph which doesn't contain some nonsense or serious misuse of established terms. He is insulting whole groups of people and institutions without giving any examples or research. I am pretty sure that if his stuff were posted without his name the conclusion would be obvious: it's just nonsense and very offensive at that. I re…

The guy is clearly not an idiot in the general sense; I bet he is of, at least, average intelligence. I also have my doubts that he is 'vile' for the majority of connotations and denotations of that word. Because of this, just shouting 'he's an idiot!' is vulnerable to being dismissed as hyperbole, and your baby gets thrown out with the bathwater. You seem exasperated at his condescending attitude, but make no effort…

I don't understand calling him a "vile idiot" is so terrible. Vile is a subjective assessment in the eye of the beholder. Bluecalm began with a few paragraphs making the case that Taleb has pretensions of being an expert where he doesn't know what he is talking about, and that his statements are unsupported.

Calling him a crank is a more objective statement, and to me seems to impute an unknowable motive to what Taleb is doing in these types of articles. Is it an egotistical attempt to feel important, a strategy to sell books, or does he believe all the stuff he says?

What I personally find so unpleasant and objectionable about Taleb is probably unfair, but he seems to fit the archetype of an entire group of people who make loud statements that are wrong or unsupported, but take a long and nuanced discussion to explain why.

For example he seems to be terrible at economic analysis, but portrays himself as an excellent trader so he asserts that scientific study of the economy is bunk. Let's say that he really is an outlier in finance, operating with localized phenomena is different than having a comprehensive view of the economy and refining models.

I think his idea of Black Swans is incredibly useful, and it is interesting to explore the psychology of underestimating the likelihood of rare events. His contributions are less useful when he sounds like the people who say "economics is not a science", before making a bold assertion about the economy that has no rigorous models to support it. Such situations are analogous to saying meteorology is not a science when it comes to forecasting whether or not it will rain in ten days. "A science" is a field where everything is known? "A science" is not a field where models are refined through research and experimentation? The science part of meteorology is the process of understanding the mechanisms behind how air masses interact and improving models, not predicting weather at a future date when a good deal of what will determine whether or not it rains has not yet occurred.

Re: Nassim Taleb: We should retire the notion of standard deviation

#137

Post by tsax »

bluecalm wrote: post #106
bnegreve wrote: post #88

Earlier quoted context omitted.

> The guy is a vile idiot of the worst kind: ignorant and aggressive. Your comment is a bit irritating. I don't know the author of this blog post but I think he has a couple of valid points. You seem to dislike him for some other reasons. Feel free to share them. The blog post is still perfectly reasonable. Anyway, I agree some of your comments as well, namely this: > I can give hundreds when std dev is useful and me…

>Your comment is a bit irritating. I don't know the author of this blog post but I think he has a couple of valid points. You seem to dislike him for some other reasons. Feel free to share them. The blog post is still perfectly reasonable. Really ? You think call for substituting one of the most common use measures in science, finance, gambling etc. with some other measure which doesn't have the same crucial properti…

Why this attitude? Simply because it's 'insulting'? Is it true? Should you not be far more concerned if it's true? That lots of scientists are misusing statistical tools? What does it say about you that you are posting angry vitriolic comments online because someone called out scientists misusing statistical tools but that you are not as alarmed that such a thing could be happening?

Re: Nassim Taleb: We should retire the notion of standard deviation

#138

Post by justin66 »

canvia wrote: post #130
bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

Take this with a grain a salt. Taleb has made a lot of enemies with his writings about the bank bailouts and gov inefficiency. There are a lot of reasons for people to attack him, especially anonymously on the internet. I am not saying that the arguments are without merit, just to be cautious in forming an opinion on the man.

> There are a lot of reasons for people to attack him, especially anonymously on the internet.

On the positive side, if you're going to judge the guy by the insights of his critics, his supporters have included the likes of Benoit Mandelbrot and Jack Bogle.

Re: Nassim Taleb: We should retire the notion of standard deviation

#139

Post by bluecalm »

shas3 wrote: post #128
bluecalm wrote: post #116

Earlier quoted context omitted.

>He specifies that SD is reasonable as a mathematical tool, but not for inference about society, finance, etc. Yes, and as SD is widely used in those so this is quite a bold statement to make an Taleb doesn't give one example of things going wrong with it and his proposal working better. This observation is also wrong and I pointed out some areas where standard deviation is useful. >Your comment appears to have an ag…

> Taleb doesn't point one situation when MAD works better than SD (I pointed out some where SD works better than MAD) he also doesn't address obvious problems which such substitutions create. Ummm, are we both discussing the same article? > Yes it's ideology of calling people out when they assert incorrect things in condescending tone without any arguments while insulting whole groups of scientists and statisticians.…

>Ummm, are we both discussing the same article?

Yes. He doesn't mention the fact that MAD loses information about volatility. He ignores the fact because that would make his whole article look silly.

>But please argue to the merit of his arguments.

It's kinda difficult when he is not saying anything specific. When he says: "look, those scientists don't grok basic math but fail for catchy names instead" without any facts, examples etc. all I can do is call him out on this nonsense. When he argues for abandoning SD I can give situations where it's not gonna work and I did.

>1) MAD is more accurate in sample measurements, and less volatile than STD since it is a natural weight whereas standard deviation uses the observation itself as its own weight, imparting large weights to large observations, thus overweighing tail events.

But this is nonsense. It's like saying measuring temperature is better than measuring mass. Those are just different things to measure and saying one is less volatile isn't really meaningful. SD contains information about volatility, MAD doesn't. That the reason SD is used for many things. When you want to substitute one with the other you gotta address how you handle that lost information.

>Also he alludes to his paper with Goldstein. It is clear form Goldstein and Taleb's manuscript that that Taleb is not just throwing these arguments to talk trash about practitioners of statistics. They

This paper wouldn't pass peer review. What was the methodology ? How much time they have ? Did they have access to a computer ? Even if it was really serious experiment it's just toy example of people getting question asked in tricky way wrong. What about actual mistakes they make in real world because of using SD instead of MAD ? This is what he claims is a problem. He claims SD should be abandoned in favor of MAD, what is one situation which people would get better if they do it ? I am mocking him because is a master of using many words without making a point and somehow is good at seducing readers to like him.

Re: Nassim Taleb: We should retire the notion of standard deviation

#140

Post by azakai »

Daniel_Newby wrote: post #126
azakai wrote: post #78

Earlier quoted context omitted.

>> The notion of standard deviation has confused hordes of scientists > What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? He is exaggerating, for sure. But the point is valid: the mean average deviation (MAD) is often very different than the standard deviation (STD), and the MAD is more intuitive, it has a natural geometrical interpretati…

> MAD, however is nicer for reporting data since it is more intuitive. I have a hard time fathoming how anyone could think such a thing. This is math: we should be using things because they map to reality in some way, not because they are aesthetically pleasing. The standard deviation maps to processes where the "importance" of changes is proportional to their square. For example, electrical power is proportional to…

> The standard deviation maps to processes where the "importance" of changes is proportional to their square.

Sure, but the point of the article is that STD is used very commonly, in places where that does not make sense. For example, it is common to see things like "the weight of the test subjects was 170cm (STD 5cm)".

Post reply on HN