Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Taleb: We should retire the notion of standard deviation

edge.org

Post reply on HN
Print view View original

61–70 of 254 posts

Re: Nassim Taleb: We should retire the notion of standard deviation

#61

Post by cwyers »

cwyers wrote: post #35
lstamour wrote: post #27

Earlier quoted context omitted.

> which Goldstein? From ClementM above, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

Thank you. From that paper: "We first posed this question to 97 portfolio managers, assistant portfolio managers, and analysts employed by investment management companies who were taking part in a professional seminar. The second group of participants comprised 13 Ivy League graduate students preparing for a career in financial engineering. The third group consisted of 16 investment professionals working for a major…

And all his examples come from the financial sector, it would seem, but in the article he refers to "hordes of scientists," "people in social science," and even "problems with social and biological science." So even the hard sciences get pulled into his indictment of standard deviation, even though he never once gives an example from them.

Re: Nassim Taleb: We should retire the notion of standard deviation

#63

Post by ThomPete »

bluecalm wrote: post #53
ThomPete wrote: post #23

Earlier quoted context omitted.

Context I think http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480 And I think you should pay more attention to this: "...as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems..." if you want to understand where he is coming from.

>>"...as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems..." Well, for sure there are problem with people applying tools without fully understanding them. It's not very interesting statement. As obviously many things in nature form normal distribution and standard deviation describes those distributions well…

It might not be interesting to you and I am sure you know how to apply them.

It is however important if large groups of disciplines are not able to apply it properly.

Instead of calling him a vile idiot and refusing to even look at what he is trying to say you could see this statement as a cry for help, a provocation to start a debate and so on.

The deeper issues here, I think is that he is probably not death wrong nor death right and so it's really impossible to prove anything objectively but rather different experiences yield different opinions.

I personally believe that sometimes you have to simplify in order to get a fundamental discussion started. In fact i find the aftermath of those provocations to be the most enlightening.

Re: Nassim Taleb: We should retire the notion of standard deviation

#64

Post by raverbashing »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

I understand your point, and I agree mostly with it

But I think the headline was sensationalised.

Taleb doesn't seem to be against "standard deviation" mostly, but calling it "standard deviation" (which should be called something else), and providing a more "user friendly" number.

The positive square root of the second central moment of the density distribution function will keep being that.

Re: Nassim Taleb: We should retire the notion of standard deviation

#65

Post by dferlemann »

ThomPete wrote: post #50
bluecalm wrote: post #29

Earlier quoted context omitted.

I just can't pass it. I gave my best at reading his book. The guy is unable to write one paragraph which doesn't contain some nonsense or serious misuse of established terms. He is insulting whole groups of people and institutions without giving any examples or research. I am pretty sure that if his stuff were posted without his name the conclusion would be obvious: it's just nonsense and very offensive at that. I re…

Funny I find your comments to be doing exactly what they are critiquing. And I think you are letting your hate to him cloud your ability to want to understand what he is trying to say. I personally find his books mostly valid critiques of established "truths", probably because I agree with much of it. If that makes me a vile idiot I am fine with that. Each to their own.

Too bad no upvote button. I agree with you. I find the comment also a little too worked up with a little useful counter argument. I agree STD has its uses in situations. Nassim is not saying to get rid of the concept, but more like saying a lot of people are not using it correctly. I don't know why so worked up about that... Quite a lot of pointless insults.

Re: Nassim Taleb: We should retire the notion of standard deviation

#66

Post by marvin »

corresation wrote: post #51
bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing." While I am not a Taleb fan (he had his own black swan and became exactly the reviled expert his own books warn us about), your hostility in this case seems out of place. The root of his seemingly casual contemplation is that standard deviation is simply misnamed. That t…

What is Taleb's own black swan?

Re: Nassim Taleb: We should retire the notion of standard deviation

#67

Post by bluecalm »

corresation wrote: post #51
bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing." While I am not a Taleb fan (he had his own black swan and became exactly the reviled expert his own books warn us about), your hostility in this case seems out of place. The root of his seemingly casual contemplation is that standard deviation is simply misnamed. That t…

Well, I read his article as "let's abandon standard deviation and use average mean deviation instead". I think that's what he claims. It's not: "some people are sometimes confused by the concept and in some places mean deviation would serve better, for example here and here". It has way more aggressive tone and way wider assertions about people misusing standard deviation. Remarks about "real life" doesn't help either, especially if you don't provide even one example where standard deviation is commonly used but mean absolute deviation would serve better.

>> Which -- as someone in the financial industry -- I can absolutely confirm.

This is what he is doing: he takes common sentiment often and makes into war. "Abandon standard deviation", "the confusion started", "PhD's are confused" It's easy to like him and nod in agreement while omitting that he took it way too far, proposed things way too radical and again: didn't give one single example of stuff not working because of it.

Re: Nassim Taleb: We should retire the notion of standard deviation

#68

Post by calroc »

cheald wrote: post #36

I really tried to get through "The Black Swan" and Taleb's writing struck me as so pretentious and self-involved that it made it impossible for me to finish. He strikes me as someone who is so desperate to be important and recognized that an assertion like this doesn't really surprise me.

I started reading that book and I thought he was a genius, but then, right abut the point where he starts bagging on the Uncertainty Principle, I realized that he's actually kind of an idiot.

The book does make a good point though.

Re: Nassim Taleb: We should retire the notion of standard deviation

#69

Post by skybrian »

klodolph wrote: post #54
programminggeek wrote: post #28

I think because it's called "standard deviation" that it sounds like the thing to use or look for. It sounds more correct because of the word standard. I feel like it is the same kind of failing due to human perception of language that programmers have with the idea of exceptions and errors, especially the phrase "exceptions should only be used for exceptional behaviors". That's a cool phrase, but people latch on to…

> I think because it's called "standard deviation" that it sounds like the thing to use or look for. Yes! Because it's an awesome trick and lets you do good estimates on napkins. The other day I was buying lunch at a food cart and thought about how much change the food carts had to carry, as a function of how many customers they have, under the assumption that they want to be able to provide correct change to 99% of…

It's a neat math trick, but it seems more accurate to say this lets you calculate bad estimates on the back of a napkin. Unless you really think food carts carry $5000 in change.

The quantitative work I do has to do with measuring latency, where the minimum, median, 90%, and 99% values are more meaningful than the mean or standard deviation. Programs typically have a best-case scenario (everything cached) and a long one-sided tail.

Re: Nassim Taleb: We should retire the notion of standard deviation

#70

Post by perlgeek »

RivieraKid wrote: post #17

I was just wondering about a very related problem. I do 5 measurenments of some random variable (let's say execution time) and average them. How should I report the variability of that average? State the sample size and standard deviation?

If it were 10 or more measurements, I'd use average, standard deviation and sample size if we can expect that variable to be normally distributed.

I don't know how to make meaningful statistics with fewer data points.

Post reply on HN