Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Taleb: We should retire the notion of standard deviation

edge.org

Post reply on HN
Print view View original

21–30 of 254 posts

Re: Nassim Taleb: We should retire the notion of standard deviation

#21

Post by tptacek »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

This comment was great fun to read, and I suspect that I share some of your opinions about Taleb (although mine are decaying with each day that passes since I last finished reading any of his books), but it would have been stronger as an argument if you had left off the part about him being a "vile idiot".

Re: Nassim Taleb: We should retire the notion of standard deviation

#22

Post by RivieraKid »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

> Standard deviation tells you how volatile measurements are not what mean deviation is.

Why would you use sd to convey the amount of volatility? Isn't the mean deviation much more easily understood?

Re: Nassim Taleb: We should retire the notion of standard deviation

#23

Post by ThomPete »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

Context I think

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

And I think you should pay more attention to this:

"...as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems..."

if you want to understand where he is coming from.

Re: Nassim Taleb: We should retire the notion of standard deviation

#24

Post by drblast »

ClementM wrote: post #13

This article is based on paper Taleb published in 2007. If you want to test yourself, submit yourself to experiment in page 3: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

Thanks! Always nice to read the full research.

Re: Nassim Taleb: We should retire the notion of standard deviation

#25

Post by tlarkworthy »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

Yeah. Eliminating stdev so social scientists don't get confused is the tail wagging the dog.

Re: Nassim Taleb: We should retire the notion of standard deviation

#26

Post by lambdasquirrel »

I think we'd be better off if we recognized that there are statistical distributions in the world besides the plain old Gaussian. For example, wealth does not follow a Gaussian, so why the heck do we throw around ideas like "above average wealth"?

Is MAD any better? Definitely. But I'd like to see a visual demonstration of how well it models exponential-based distributions. How well does it describe their "shape", the skew of the tail?

Re: Nassim Taleb: We should retire the notion of standard deviation

#27

Post by lstamour »

cwyers wrote: post #10
icebraining wrote: post #9

Earlier quoted context omitted.

The next two paragraphs have examples with some context.

Like this? "But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of commerce and the Federal Reserve partaking of the conflation, even regulators in statements on market volatility. What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life." It doesn't tell us what happened, it just asserts that…

> which Goldstein?

From ClementM above, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480

Re: Nassim Taleb: We should retire the notion of standard deviation

#28

Post by programminggeek »

I think because it's called "standard deviation" that it sounds like the thing to use or look for. It sounds more correct because of the word standard.

I feel like it is the same kind of failing due to human perception of language that programmers have with the idea of exceptions and errors, especially the phrase "exceptions should only be used for exceptional behaviors". That's a cool phrase, but people latch on to it because of the word exception sounding like something extremely rare and out of the ordinary whereas we see errors as common, but they are in fact the same thing. Broke is broke, it doesn't matter what you call it, but thousands of programmers think differently because of the name we gave it.

We are human and language absolutely plays a role in our perception of things.

Re: Nassim Taleb: We should retire the notion of standard deviation

#29

Post by bluecalm »

tptacek wrote: post #21
bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

This comment was great fun to read, and I suspect that I share some of your opinions about Taleb (although mine are decaying with each day that passes since I last finished reading any of his books), but it would have been stronger as an argument if you had left off the part about him being a "vile idiot".

I just can't pass it. I gave my best at reading his book. The guy is unable to write one paragraph which doesn't contain some nonsense or serious misuse of established terms. He is insulting whole groups of people and institutions without giving any examples or research. I am pretty sure that if his stuff were posted without his name the conclusion would be obvious: it's just nonsense and very offensive at that.

I realize calling people idiots are frown upon here but I am ready to defend it. There isn't one other author which infuriates me with his ignorance and inability to make specific point without throwing condescending remarks. If there is anybody who deserve the term it's him. I mean "we should retire standard deviation because everybody misues it and it's not useful for real life" without any examples ? He just called out thousands of scientist, risk managers and statisticians without giving one single argument. His suggestion to just use mean deviation is laughable and obviously doesn't work for many purposes which he didn't address while being condescending in the process.

It's worse than saying: "we should retire Java and just use Python because it has dynamic typing and seemingly developers at even serious companies fail to see this!". Remark of this kind would gain me "idiot" badge pretty quickly. This article is on this level (well, actually lower as remark about Python actually contains an argument) and his book is worse. If that doesn't qualify him as an idiot I don't know what else could.

Re: Nassim Taleb: We should retire the notion of standard deviation

#30

Post by regularfry »

n00b101 wrote: post #15

Taleb has a good point about people mistakenly interpreting standard deviation (sigma) as Mean Absolute Deviation (MAD). I like that he gives some conversions (sigma ~= 1.25 * MAD, for Normal distribution). I think it's rather silly to talk about "retiring" standard deviation, but we can't blame Taleb - the publication itself posed the question "2014: What Scientific Idea is Ready for Retirement?" to various scientif…

I think his point is that Black-Scholes et al are holed beneath the waterline precisely because they involve standard deviations. In his world, you're better off being unable to price an option than you would be with Black-Scholes. Your example of "That put option is currently selling at 30% vol" is actually an example of why the system is so completely broken: if volatility as standard deviation was valid, all options against the same underlying instrument would have the same implied volatility. The volatility smile shouldn't exist.

This wouldn't matter if the down-side wasn't so crippling.

I don't think Taleb has to be the one to propose a replacement for portfolio theory, and I think criticism of him for not doing so is pointless. You don't need to have a spare tire handy to point out that your neighbour's car has a flat, and you don't have to run an airline to tell people not to get on a plane with the engines visibly on fire.

Post reply on HN