Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dunning-Kruger Effect Is Autocorrelation

economicsfromthetopdown.com

Post reply on HN
Print view View original

141–150 of 203 posts

Re: The Dunning-Kruger Effect Is Autocorrelation

#141

Post by cortesoft »

jakear wrote: post #37

Article seems to be saying “DK doesn’t exist because it always exists”. Which is… absurd? The point of DK is that when you don’t know shit, any non-degenerate self assessment will result in overestimating your ability. In short, “there are more natural numbers above smaller natural numbers than bigger ones”. This doesn’t have to do with psychology, and it’s expected that it appears when evaluating random data. That’s…

> In short, “there are more natural numbers above smaller natural numbers than bigger ones”

I get what you are trying to say, but this isn’t true… every natural number has the same amount of numbers greater and smaller… an infinite number.

Re: The Dunning-Kruger Effect Is Autocorrelation

#142

Post by msrenee »

andersource wrote: post #7

Very interesting article and statistical analysis, but I really don't see how it concludes that the DK effect is wrong based on the analysis. The fact that the DK effect emerges with _completely random data_ is not surprising at all - in this case the intuitive null hypothesis would be that people are good at estimating their skill, therefore there would be strong a correlation between their performance and self-eval…

Please feel free to tell me why my interpretation is wrong. I understand stats just well enough to get myself in trouble.

The line for actual ability is basically x=y. If you scored 10%, you're in the bottom quartile. If you scored 100%, you're in the top quartile. That line isn't really data, just something for comparison. The perceived ability line is the one that utilizes the data. It seems to show that once you average out what everyone rated themselves, it ends up kind of in the middle between ~55-70%. So the people who scored 10% assumed, on average, they would score around 55%. The people who scored 100% assumed, on average, that they would score about 75%. That makes the average expected score much higher than the actual score on the low end and somewhat lower than the actual score on the high end.

I'd interpret this as the bottom quartile thinks they're average and the top quartile thinks they're a bit above average. So basically everyone thinks they're average-ish, but the people who did worst on the test were the most wrong about that. But then again, I can't remember what the questions on the test were even about and just the single graph isn't terribly useful to argue over because it's missing all of the context of the paper.

Now that I've sat and interpreted the graph using my own set of notions about what the numbers mean and what the graph actually shows, I feel like this ought to be used as one of those life lessons about how quotes and diagrams outside of their context within a paper are the epitome of the phrase "lies, damn lies, and statistics." Statistics aren't always lies, but they're incredibly easy to bend to your own biases and assumptions.

Re: The Dunning-Kruger Effect Is Autocorrelation

#143

Post by knorker »

No it isn't.

If everyone responded that they are 50% skilled (or per this article, that it's randomly distributed), then we

1. See the same graph, and

2. Bad people overestimate, and good people underestimate

This article merely describes Dunning Kruger. Accidentally proves it mathematically, but thinks that it debunks it.

Re: The Dunning-Kruger Effect Is Autocorrelation

#144

Post by knorker »

TimPC wrote: post #99

The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed. If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment…

> The effect is statistical not psychological.

It is, though. This article says that if people are bad at estimating their skill (towards randomness / the midpoint), then bad people will overestimate, and good people will underestimate.

The psychological part is that people indeed will assume they're closer to the mean than they actually are. DK effect would not be seen if people correctly estimated their skill, nor if experts overestimated, nor if the incompetent underestimated.

Re: The Dunning-Kruger Effect Is Autocorrelation

#145

Post by pdonis »

bumby wrote: post #138
pdonis wrote: post #129

Earlier quoted context omitted.

> DK doesn't mean no correlation, it means inverse correlation. No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores. Which is not what we would expect: we would expect that their predictions of their test scores would be strongly (or at least more st…

> their prediction of what their test scores will be, is uncorrelated I couldn't access the original 1999 full article because it's behind a paywall, but is this accurate? The abstract states that people "grossly overestimated their test performance and ability". If it were truly uncorrelated, wouldn't we expect there to be roughly equal number of people who "grossly underestimate" their performance? In any event, I…

> The abstract states that people "grossly overestimated their test performance and ability".

I understand that the words D-K used were along those lines; but the actual data shown in their graphs says what I said. Just look at the lines on the graphs. The "actual" lines are 45 degree lines up and to the right--as they must be. But the "perceived" lines are horizontal, or roughly so. That means the two lines are uncorrelated, or nearly so.

Saying that underperformers grossly overestimated their performance is only describing part of that--the part where the "perceived" lines on the left sides of the graphs are above the "actual" lines. It does not describe the other part--the part where the "perceived" lines on the right sides of the graphs are below the "actual" lines.

D-K don't talk about that at all in their paper, which means their paper itself misdescribes the actual data they found. And if this article had said that, it would have been a valid criticism. But this article is not talking about what D-K said in words; it's talking about D-K's actual graph. And the article's claimed criticism of D-K's graph is not valid: the article is in fact just describing the same graph in different words. It's not offering a different "explanation" of why the graph looks that way; in fact, as I said, it offers no explanation of that at all; it doesn't even recognize that as a question.

Re: The Dunning-Kruger Effect Is Autocorrelation

#146

Post by pdonis »

bumby wrote: post #138
pdonis wrote: post #129

Earlier quoted context omitted.

> DK doesn't mean no correlation, it means inverse correlation. No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores. Which is not what we would expect: we would expect that their predictions of their test scores would be strongly (or at least more st…

> their prediction of what their test scores will be, is uncorrelated I couldn't access the original 1999 full article because it's behind a paywall, but is this accurate? The abstract states that people "grossly overestimated their test performance and ability". If it were truly uncorrelated, wouldn't we expect there to be roughly equal number of people who "grossly underestimate" their performance? In any event, I…

> I couldn't access the original 1999 full article because it's behind a paywall

The article this thread is about links to a PDF of it, I was able to download it just fine:

https://www.avaresearch.com/files/UnskilledAndUnawareOfIt.pd...

Re: The Dunning-Kruger Effect Is Autocorrelation

#148

Post by NaturalPhallacy »

hungrygs wrote: post #123
TimPC wrote: post #99

The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed. If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment…

Just anecdotal, but my life observation of DK is often highly intelligent and competent people in a particular field who then generalize that to pontificate and proclaim, directly or indirectly, superior understanding to certified domain experts (e.g., have directly related advanced degree(s), work in the field for decades.) It thus seems more or as much a psychological effect - in short, people with a personality ty…

Classic example:

>The V-tail design gained a reputation as the "forked-tail doctor killer",[16] due to crashes by overconfident wealthy amateur pilots,[17] fatal accidents, and inflight breakups.[18] "Doctor killer" has sometimes been used to describe the conventional-tailed version, as well.

https://en.wikipedia.org/wiki/Beechcraft_Bonanza

Re: The Dunning-Kruger Effect Is Autocorrelation

#149

Post by bumby »

pdonis wrote: post #145
bumby wrote: post #138

Earlier quoted context omitted.

> their prediction of what their test scores will be, is uncorrelated I couldn't access the original 1999 full article because it's behind a paywall, but is this accurate? The abstract states that people "grossly overestimated their test performance and ability". If it were truly uncorrelated, wouldn't we expect there to be roughly equal number of people who "grossly underestimate" their performance? In any event, I…

> The abstract states that people "grossly overestimated their test performance and ability". I understand that the words D-K used were along those lines; but the actual data shown in their graphs says what I said. Just look at the lines on the graphs. The "actual" lines are 45 degree lines up and to the right--as they must be. But the "perceived" lines are horizontal, or roughly so. That means the two lines are unco…

Thanks for taking the time to help me understand the data.

Looking at the graphs, isn't the implication that the distance between the two lines is the metric of interest? In other words, that the distance between real/perceived scores in Q1 & Q2 is "grossly" wider than the distance between Q3 & Q4, with the noted exception that only Q4 underestimates?

Re: The Dunning-Kruger Effect Is Autocorrelation

#150

Post by ianferrel »

bumby wrote: post #138
pdonis wrote: post #129

Earlier quoted context omitted.

> DK doesn't mean no correlation, it means inverse correlation. No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores. Which is not what we would expect: we would expect that their predictions of their test scores would be strongly (or at least more st…

> their prediction of what their test scores will be, is uncorrelated I couldn't access the original 1999 full article because it's behind a paywall, but is this accurate? The abstract states that people "grossly overestimated their test performance and ability". If it were truly uncorrelated, wouldn't we expect there to be roughly equal number of people who "grossly underestimate" their performance? In any event, I…

Overestimating vs underestimating their ability has less to do with whether their estimates are correlated and more to do with the average estimation.

If estimates of ability were a truly random sample with mean of 75th percentile ability, then the net effect will be vast overestimation. If they're a truly random sample with mean at 50th percentile, it will be equal. At 25th, underestimation.

It's like how 90% of people estimate that they're in the top 50% of drivers by ability.

Post reply on HN