The Dunning-Kruger Effect Is Autocorrelation
economicsfromthetopdown.com
The Dunning-Kruger Effect Is Autocorrelation
1–10 of 203 posts
Re: The Dunning-Kruger Effect Is Autocorrelation
#2So dunning and Kruger were victims of the dunning-kruger effect ?
Re: The Dunning-Kruger Effect Is Autocorrelation
#3Great article, there should be much more common knowledge of statistics and it's problems, it is surely the most abused of all sciences.
Re: The Dunning-Kruger Effect Is Autocorrelation
#4Great article, there should be much more common knowledge of statistics and it's problems, it is surely the most abused of all sciences.
Hence the quote:
"There are white lies, damned lies and statistics"
Funny that all the major ML marvels are also built on statistical foundations - a tool used as much as abused.
Re: The Dunning-Kruger Effect Is Autocorrelation
#5I don't find the "autocorrelation" explanation intuitive (although it may be equivalent to what I'm about to suggest). The way I think about it, is that it comes about because the y-axis is a percentile rank. How does it actually work for people to give unbiased estimates of their performance as percentiles? For the people at the 50th percentile in truth, they could give a symmetric range of 45-55 as their estimates, and it would be unbiased. But what about the people at the 99th percentile? They can't give a range of 94-104, the scale only goes as high as 100. So even if they are unbiased (whatever that means in this context), their range of estimates in percentile terms has to be asymmetrical, by construction. So, even if people are unbiased, if you were to plot true percentile vs subjective estimated percentile, the estimated scores would "pull toward" the centre. Then the only thing you need to replicate the Dunning-Kruger graph is to suppose that people have a uniform tendency to be overconfident, i.e. that people over-rate their abilities, but to an extent unrelated to their true level of skill. The estimated score at the left side of the graph goes higher, but it can't go as high on the right side of the graph because it butts up against the 100 percentile ceiling. Then you end up with a graph that looks like lower skilled people are more overconfident than higher skilled people are underconfident.
Re: The Dunning-Kruger Effect Is Autocorrelation
#6I think the gist of the article is this:
Suppose you make 1000 people take a test. Suppose all 1000 of these people are utterly incapable of evaluating themselves, so they just estimate their grade as a uniform random variable between 0-100, with an average of 50.
You plot the grades of each of the 4 quartiles and it shows a linear increase as expected. Let's say the bottom quartile had an average of 20, and the top had 80. But the average of estimated grades for each quartile is 50. Therefore, people who didn't do well ended up overestimating their score, while people who did well underestimated it.
In reality, nobody had any clue how to estimate their own success. Yet we see the Dunning-Kruger effect in the plot.
Re: The Dunning-Kruger Effect Is Autocorrelation
#7Very interesting article and statistical analysis, but I really don't see how it concludes that the DK effect is wrong based on the analysis. The fact that the DK effect emerges with _completely random data_ is not surprising at all - in this case the intuitive null hypothesis would be that people are good at estimating their skill, therefore there would be strong a correlation between their performance and self-evaluation of said performance. If the data weren't related, then this hypothesis isn't likely, which is exactly what DK means. And indeed if you look at the plots in the article (of the completely random data), they depict a world in which people are very bad at estimating their own skill, therefore, statistically, people with lower skills tend to overestimate their skills, and experts tend to underestimate it.
Also wanted to point out that in general there is no issue with looking at y - x ~ x, this is called the residual plot, and is specifically used to compare an estimate of some value vs. the value itself.
That being said, the author seems very confident in their conclusion, and from the comments seems to have read a lot of related analyses, so I might be missing something. ¯\_(ツ)_/¯
Re: The Dunning-Kruger Effect Is Autocorrelation
#8I don't find the "autocorrelation" explanation intuitive (although it may be equivalent to what I'm about to suggest). The way I think about it, is that it comes about because the y-axis is a percentile rank. How does it actually work for people to give unbiased estimates of their performance as percentiles? For the people at the 50th percentile in truth, they could give a symmetric range of 45-55 as their estimates,…
It's an interesting article but the author is using terms a little incorrectly or strangely I think, and making untrue statements. The basic points are important and interesting to think about, but could've been explained more clearly.
Re: The Dunning-Kruger Effect Is Autocorrelation
#9I think the gist of the article is this: Suppose you make 1000 people take a test. Suppose all 1000 of these people are utterly incapable of evaluating themselves, so they just estimate their grade as a uniform random variable between 0-100, with an average of 50. You plot the grades of each of the 4 quartiles and it shows a linear increase as expected. Let's say the bottom quartile had an average of 20, and the top…
That's the way I understand the statistical analysis, and in my view this exactly supports (not contradicts) DK:
> In reality, nobody had any clue how to estimate their own success.
Wouldn't that mean unskilled people tend to overestimate their skill, and experts tend to underestimate it? Why is there a contradiction with DK's conclusions?
Re: The Dunning-Kruger Effect Is Autocorrelation
#10I don't find the "autocorrelation" explanation intuitive (although it may be equivalent to what I'm about to suggest). The way I think about it, is that it comes about because the y-axis is a percentile rank. How does it actually work for people to give unbiased estimates of their performance as percentiles? For the people at the 50th percentile in truth, they could give a symmetric range of 45-55 as their estimates,…
It's an interesting article but the author is using terms a little incorrectly or strangely I think, and making untrue statements. The basic points are important and interesting to think about, but could've been explained more clearly.
Yes, when I read "autocorrelation" I think of a time-series variable that is correlated with its own lagged values.