Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nassim Taleb: We should retire the notion of standard deviation

edge.org

Post reply on HN
Print view View original

151–160 of 254 posts

Re: Nassim Taleb: We should retire the notion of standard deviation

#151

Post by Helianthus »

jerf wrote: post #150
Helianthus wrote: post #143

Earlier quoted context omitted.

>"The realization that the normal distribution is not the only way to model data". Realization by who? If you understand the normal distribution you had damn well better know that there are other probability distributions. >The problem of course is that the Cauchy distribution is pathological in that it doesn't have a mean or variance, you can calculate similar properties for it (location and scale), but it doesn't o…

"Realization by who? If you understand the normal distribution you had damn well better know that there are other probability distributions." See Glyptodon's post: https://news.ycombinator.com/item?id=7065067 If you hang out with mathematicians, yeah, sure, everybody knows there's a ton of distributions. Try hanging out with, say, biologists. The undergrad statistics education is basically "mumble mumble guassian mum…

"A general science education needs far firmer statistical grounding" doesn't equate to "The standard deviation should be retired."

Re: Nassim Taleb: We should retire the notion of standard deviation

#152

Post by washedup »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

His point has always been to cut off the tails. The MAD actually allows you to cut off the tails sooner, and that moves outside of your chosen confidence interval will be seen as tail risk much sooner than if you use STD. For example, think about the following example of normal returns versus S&P returns:

http://managed-futures-blog.attaincapital.com/wp-content/upl...

A distribution built from MAD will more closely resemble the reality of the S&P returns, and will create more robust models. There are many examples of this.

Re: Nassim Taleb: We should retire the notion of standard deviation

#153

Post by jerf »

Helianthus wrote: post #151
jerf wrote: post #150

Earlier quoted context omitted.

"Realization by who? If you understand the normal distribution you had damn well better know that there are other probability distributions." See Glyptodon's post: https://news.ycombinator.com/item?id=7065067 If you hang out with mathematicians, yeah, sure, everybody knows there's a ton of distributions. Try hanging out with, say, biologists. The undergrad statistics education is basically "mumble mumble guassian mum…

"A general science education needs far firmer statistical grounding" doesn't equate to "The standard deviation should be retired."

I was establishing that there are plenty of people who don't really realize there are other distributions.

You seem to be having some trouble reading what I'm writing, rather than what you think I should be writing.

Re: Nassim Taleb: We should retire the notion of standard deviation

#154

Post by skybrian »

klodolph wrote: post #99
skybrian wrote: post #69

Earlier quoted context omitted.

It's a neat math trick, but it seems more accurate to say this lets you calculate bad estimates on the back of a napkin. Unless you really think food carts carry $5000 in change. The quantitative work I do has to do with measuring latency, where the minimum, median, 90%, and 99% values are more meaningful than the mean or standard deviation. Programs typically have a best-case scenario (everything cached) and a long…

Saying that it's silly to think that food carts have $5,000 in change is unproductive because I was illustrating how the calculation works, not how the economics of food carts works, and the numbers I used for illustrative purposes were not intended to reflect reality. (1,000 customers in a day? Not likely, my guess is 200 for the busiest food carts.) But it's good to have bad estimates, at least, it's better to have…

Sure, two points is better than one but what's special about two? I'd rather have a graph. We have computers so there's rarely a reason to compress the data so much.

Re: Nassim Taleb: We should retire the notion of standard deviation

#155

Post by snake_plissken »

I've always thought his writings were more allegorical than scientific; you can't rely on the standard deviation to never go against you at the worst possible time. But like anything else, it can and it (probably) will.

Also, yes, his writing style is grating and he takes opportunistic character swipes at pretty much everyone.

Re: Nassim Taleb: We should retire the notion of standard deviation

#156

Post by coldtea »

rz2k wrote: post #136
antiterra wrote: post #56

Earlier quoted context omitted.

The guy is clearly not an idiot in the general sense; I bet he is of, at least, average intelligence. I also have my doubts that he is 'vile' for the majority of connotations and denotations of that word. Because of this, just shouting 'he's an idiot!' is vulnerable to being dismissed as hyperbole, and your baby gets thrown out with the bathwater. You seem exasperated at his condescending attitude, but make no effort…

I don't understand calling him a "vile idiot" is so terrible. Vile is a subjective assessment in the eye of the beholder. Bluecalm began with a few paragraphs making the case that Taleb has pretensions of being an expert where he doesn't know what he is talking about, and that his statements are unsupported. Calling him a crank is a more objective statement, and to me seems to impute an unknowable motive to what Tale…

>I don't understand calling him a "vile idiot" is so terrible.

Then you are a vile idiot.

Re: Nassim Taleb: We should retire the notion of standard deviation

#157

Post by altrego99 »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

I am so glad that this comment is at the top!

Some other points - std.dev shows up in nature and is easier to understand, and analyze, simply because variances from independent sources add up. That's why something called ANOVA exists.

In OLS regression, we minimize the std. dev. of the error, not the MAD. (It is not because statisticians "thought" std. dev is actually MAD, it's because minimizing std. dev or square error has very nice properties that we use it even though the solution to MAD minimization is also known.)

Lastly, I don't follow who he proposes to be using MAD - if he decrees statisticians and physicists should still use Std. Dev?

Often it is that I find these articles, and I let them pass by. But sometimes, they have a potential to come back - and I stand a chance to be quoted this article, by someone else, who I then have to drain my energy to make the point that it's not really worth reporting MAD from now on.

Re: Nassim Taleb: We should retire the notion of standard deviation

#158

Post by Helianthus »

jerf wrote: post #153
Helianthus wrote: post #151

Earlier quoted context omitted.

"A general science education needs far firmer statistical grounding" doesn't equate to "The standard deviation should be retired."

I was establishing that there are plenty of people who don't really realize there are other distributions. You seem to be having some trouble reading what I'm writing, rather than what you think I should be writing.

>I was establishing that there are plenty of people who don't really realize there are other distributions.

Something I never contested.

From Glyptodon:

>>"This is std dev. This is how you compute it. Make sure you put it your tables and report."

>>it wasn't always a sensible thing to be asked to calculate but was instead just an instinctive requirement.

I hate to break it to you, but this is how rote mathematics is taught. You can't communicate concepts purely, and hammering instinctive math is better than no math at all.

To reiterate, there's nothing wrong with the normal distribution. We're not about to retire addition or subtraction just because there are "plenty of people who don't really realize" there's more to math.

>You seem to be having some trouble reading what I'm writing, rather than what you think I should be writing.

Sure whatever, same to you.

Re: Nassim Taleb: We should retire the notion of standard deviation

#159

Post by dnc »

bluecalm wrote: post #16

So first about the article: >>The notion of standard deviation has confused hordes of scientists What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ? >>There is no scientific reason to use it in statistical investigations in the age of the computer As someone who uses it daily I am eagerly awaiting his argument. >>Say someone just asked you…

You will find example here: http://www-stat.wharton.upenn.edu/~steele/Courses/434/434Con...

There is relatively simple, yet illustrative problem described on the first page. The solution is on the second page. Try to solve it w/o looking the solution.

Re: Nassim Taleb: We should retire the notion of standard deviation

#160

Post by tptacek »

coldtea wrote: post #156
rz2k wrote: post #136

Earlier quoted context omitted.

I don't understand calling him a "vile idiot" is so terrible. Vile is a subjective assessment in the eye of the beholder. Bluecalm began with a few paragraphs making the case that Taleb has pretensions of being an expert where he doesn't know what he is talking about, and that his statements are unsupported. Calling him a crank is a more objective statement, and to me seems to impute an unknowable motive to what Tale…

> I don't understand calling him a "vile idiot" is so terrible. Then you are a vile idiot.

Super helpful comment. Thanks. Definitely, the transition from calling public figures "vile idiots" to calling individual commenters on HN threads "vile idiots" is sure to produce a productive discussion.

I started this subthread by pointing out that 'bluecalm had weakened his argument by using the words "vile idiot". But 'bluecalm also made a bunch of testable and non-obvious observations in his comment; his writing had value. Whereas you just jumped in to call someone you disagreed with a name, and nothing else; your writing had no value.

Could you stop writing comments like that?

Post reply on HN