Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dunning-Kruger Effect Is Autocorrelation

economicsfromthetopdown.com

Post reply on HN
Print view View original

171–180 of 203 posts

Re: The Dunning-Kruger Effect Is Autocorrelation

#171

Post by SomewhatLikely »

Dave_Rosenthal wrote: post #110

This was interesting to me so I spent a while this AM playing with a Python simulation of this effect. I used a simple process model of a normally-distributed underlying 'true skill' for participants, a test with questions of varying difficulty, some random noise in assessing whether the person would get the question right, noise in people's assessments of their own ability, etc. I fiddled with number of test questio…

Wouldn't variance be influenced by a similar bounding effect but this time from the upper side? That is, if your true skill is 98% you aren't going to ever overestimate by more than 2%, but if your true skill is 50% you could be off by up to 50% in either direction.

Re: The Dunning-Kruger Effect Is Autocorrelation

#172

Post by cortesoft »

jakear wrote: post #162
cortesoft wrote: post #141

Earlier quoted context omitted.

> In short, “there are more natural numbers above smaller natural numbers than bigger ones” I get what you are trying to say, but this isn’t true… every natural number has the same amount of numbers greater and smaller… an infinite number.

Natural numbers are not the same as integers. They’re strictly positive. (Now rate your confidence in making that assertion.)

Oh right, not sure what I was thinking.

I don’t understand what you are saying about my confidence?

Re: The Dunning-Kruger Effect Is Autocorrelation

#173

Post by jakear »

cortesoft wrote: post #172
jakear wrote: post #162

Earlier quoted context omitted.

Natural numbers are not the same as integers. They’re strictly positive. (Now rate your confidence in making that assertion.)

Oh right, not sure what I was thinking. I don’t understand what you are saying about my confidence?

Just a meta-joke attempt regarding Dunning Kruger.

Re: The Dunning-Kruger Effect Is Autocorrelation

#175

Post by mattwilsonn888 »

I would have been more interested in seeing the raw data from the original Dunning-Kruger study reformatted to avoid auto-correlation. Maybe I've skipped over an important detail in my head, but I don't see why plotting perceived test score vs. actual test score would cause any problems; neither variable is in terms of the other.

The final study discussed is convincing as far as I thought. By using academic rank (Freshman, Sophomore, ...) they can plot the difference between difference in score and predicted score against rank without auto-correlation. Its just that using academic rank seems a possibly unreliable metric and an unnecessary complication - why not just use data about test scores and predictions of scores which already exists in a proper statistical interpretation?

Re: The Dunning-Kruger Effect Is Autocorrelation

#176

Post by [deleted] »

hungrygs wrote: post #123
TimPC wrote: post #99

The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed. If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment…

Just anecdotal, but my life observation of DK is often highly intelligent and competent people in a particular field who then generalize that to pontificate and proclaim, directly or indirectly, superior understanding to certified domain experts (e.g., have directly related advanced degree(s), work in the field for decades.) It thus seems more or as much a psychological effect - in short, people with a personality ty…

[deleted]

Re: The Dunning-Kruger Effect Is Autocorrelation

#177

Post by derbOac »

hungrygs wrote: post #123
TimPC wrote: post #99

The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed. If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment…

Just anecdotal, but my life observation of DK is often highly intelligent and competent people in a particular field who then generalize that to pontificate and proclaim, directly or indirectly, superior understanding to certified domain experts (e.g., have directly related advanced degree(s), work in the field for decades.) It thus seems more or as much a psychological effect - in short, people with a personality ty…

To be fair, I think the counterargument is that ostensible experts can overstate their ability/skill/knowledge to detrimental effect just as easily, and by virtue of the label ignore the reality of the argument/scenario at hand. That is, experts can overlook mistakes they're making, or conflicts of interest, etc because of their status, and because they overestimate their own ability. There's been studies of this in group decision making in crisis situations, where hierarchies can cause failures because the "leader" becomes overconfident and fails to heed warnings by others in the group.

This all gets really murky quickly in practice because of what "low" and "high" competence means, and what constitutes the actual scope of expertise with reference to a particular scenario.

Re: The Dunning-Kruger Effect Is Autocorrelation

#178

Post by bouncycastle »

andersource wrote: post #40
bouncycastle wrote: post #36

Earlier quoted context omitted.

The problem is that they calculated each person’s ‘self-assessment error’ with the actual test score. This error is the difference between a person’s self assessment and their test score, This is like comparing x - y to x, and if you do this, you will get a correlation no matter what.

Not necessarily. If for example y =~ x, comparing x - y to x would yield zero correlation, which to me is what DK is all about - people _aren't_ as good as we would expect at estimating their own skill.

They are not saying that the hypothesis is wrong, just that the proof of the hypothesis is wrong. You can frame it however you like, but it won't work if you have x on both sides of the equation.

Re: The Dunning-Kruger Effect Is Autocorrelation

#180

Post by jl2718 »

So, they observe a bias toward the average, and the dependence goes exactly as one would naively expect. If scientists exist to explain things we find interesting, statisticians exist to make those things boring. Seriously, work as a data scientist and you end up busting hopes and dreams as a regular part of your job. Almost everything turns out to be mostly randomness. The famous introduction to a statistical mechanics textbook had me pondering this. If life really is just randomness, it’s hard to find motivation. From a different viewpoint, however, I’ve found that the people that embrace this concept by not trying to control things too much, actually end up with the most enviable results, although I may be guilty of selection bias in that sample.
Post reply on HN