Most citations of D-K are themselves examples of D-K.
The Dunning-Kruger Effect Is Autocorrelation
121–130 of 203 posts
Re: The Dunning-Kruger Effect Is Autocorrelation
#122My takeaway, which may be flawed, is that the DK effect really hasn’t been debunked it any fundamental way. It’s just that the effect is statistical rather than psychological. High skilled individuals are still more likely to underestimate their skill level while low skilled individuals are still more likely to overestimate theirs. It’s just that everyone is bad at estimating their skill level and high skilled individuals have more room to estimate below their actual, while low skilled individuals have more room to miss above.
Is my reasoning flawed in some way?
Re: The Dunning-Kruger Effect Is Autocorrelation
#123The article is correct. The effect is statistical not psychological. It emerges even from artificial data and occurs independently of the supposed psychological justifications even for data where those justifications are clearly removed. If you adjust the experiment design to avoid introducing the auto-correlation you get data that doesn't show the DK effect at all. Some might take issue with the adjusted experiment…
Just anecdotal, but my life observation of DK is often highly intelligent and competent people in a particular field who then generalize that to pontificate and proclaim, directly or indirectly, superior understanding to certified domain experts (e.g., have directly related advanced degree(s), work in the field for decades.)
It thus seems more or as much a psychological effect - in short, people with a personality type of superiority and know-it-all, yet have never done the deep and hard work to gain or demonstrate any competency in said areas.
A common side observation is of course unfounded conspiracy theories, that the derided experts have sinister intentions.
Re: The Dunning-Kruger Effect Is Autocorrelation
#124Earlier quoted context omitted.
My guess is that the spread in self-evaluation is largest at low skill level and decreases as skill level increases. This would produce results more similar to what D-K actually observed, and is much more plausible than postulating that highly skilled people have no more idea of their skill level than low skilled people.
What strikes me about that graph is that the entire group is likely to be more skilled than the general population. The selection only of people who have the interest and means to attend higher education seems like a narrow window at the furthest edge of the true graph. So I wonder what it would look like if we included people of all education levels and social strata? My gut, based on this article, is that it would…
I dunno, if we account for the whole mosaic I think there is a higher probability of younger Cornell undergraduates performing poorly and self-assessing highly. They could very well be thinking, and especially against the backdrop of their high school peers, that they know it all. I think the explanation of "You don't know what you don't know." with a splash of "I'm the top 10% of my public high school class!" moreover I'd operate with the presumption they don't read outside of the curricula and so they're exceedingly unaware of the boundaries of their knowledge within the bounds of all knowledge, and the scope of the surface they cover. It's a wonder that they're as accurate as they are. But I think that has to do with the nature of the test, which as I recall was something like the ACT/SAT where they're effectively only testing to 8th grade standards. What happens if they test people against the whole of human knowledge?
Re: The Dunning-Kruger Effect Is Autocorrelation
#125I've felt inadequate throughout most of my early career. That's how I know that the confidence I have today is well deserved.
I've never had impostor syndrome though. To have impostor syndrome, you have to be given opportunities which are significantly above what you deserve.
I did get a few opportunities in my early career which were slightly above my capabilities but not enough to make me feel like an impostor. In the past few years, all opportunities I've been given have been below my capabilities. I know based on feedback from colleagues and others.
For example, when I apply for jobs, employers often ask me "You've worked on all these amazing, challenging projects, why do you want to work on our boring project?" It's difficult to explain to them that I just need the money... They must think that with a resume like mine I should be in very high demand or a millionaire who doesn't need to work.
I've worked for a successful e-learning startup, launched successful open source projects, worked for a YC-backed company, worked on a successful blockchain project. My resume looks excellent but it doesn't translate to opportunities for some reason.
Re: The Dunning-Kruger Effect Is Autocorrelation
#126Earlier quoted context omitted.
> But when you're bad at the skill and can't underestimate, they look the same. The (definitional) difference between bias and variance isn't related to do with whether you're bad at the skill or not. It's just mean vs variance of a probability distribution. If there's a good faith acknowledgement on your part that there's something here you're not getting then I'm very happy to try and help you understand it, and in…
Here's my main point of confusion - what does the random data experiment have to do with the DK results? As stated elsewhere DK has 2 claims: 1. Low-skilled people overestimate their performance and skilled people underestimate their performance 2. Skill correlates with self-assessment accuracy My first issue with the article is that it implies that since we get effect #1 with random data, that invalidates the respec…
For 1. Their evidence does not support the argument, since they impose a boundary condition on how low you can estimate and how low you can perform.
To show that high performers don't overestimate their skills you have to give them the opportunity to
Re: The Dunning-Kruger Effect Is Autocorrelation
#127Very interesting article and statistical analysis, but I really don't see how it concludes that the DK effect is wrong based on the analysis. The fact that the DK effect emerges with _completely random data_ is not surprising at all - in this case the intuitive null hypothesis would be that people are good at estimating their skill, therefore there would be strong a correlation between their performance and self-eval…
> I really don't see how it concludes that the DK effect is wrong based on the analysis.
Neither do I. Basically what the article actually shows is that these two statements are equivalent:
(1) People with low test scores tend to overpredict their test scores, while people with high test scores tend to underpredict their test scores.
(2) People's predictions of their test scores are uncorrelated (or more precisely very weakly correlated [1]) with their actual test scores.
This is not a statement that the D-K effect is wrong. It's just restating what the D-K effect is in different words. All the talk about "autocorrelation" is just another way of saying that, if people's predictions of their test scores are only weakly correlated with their test scores, then people with low test scores will have to overpredict their test scores (because there's virtually no room to underpredict them--there's a minimum possible test score and their actual score is already close to it), and people with high test scores will have to underpredict them (because there's virtually no room to overpredict them--there's a maximum possible test score and their actual score is already close to it). But the real question is: why are x and y so weakly correlated? Why are people's predictions of their test scores so weakly correlated with their actual test scores? That is not what one would intuitively expect. That is the question the D-K effect raises, and the author not only doesn't answer it, he doesn't even see it.
Also, this statement in the description of the Nuhfer research doesn't make sense:
"What’s important here is that people’s ‘skill’ is measured independently from their test performance and self assessment."
Um, the test performance is the people's "skill". And in the original D-K research, it was "measured independently" from the people's self-assessment (their prediction of their test performance).
[1] Notice that in the "uncorrelated data" graph, Figure 10, the red line is basically horizontal. That's what you get when x and y are uncorrelated. But in the original D-K graph, Figure 2, the thick black line is not horizontal--it slopes upward. That's what you get when x and y are weakly correlated. If the author had put in a weak correlation between x and y in his own experiment, he would have gotten a graph that looked like Figure 2. But of course that still would do nothing to explain why x and y are so weakly correlated, which is the actual question.
Re: The Dunning-Kruger Effect Is Autocorrelation
#128My intuition for this is: given a fixed and known scoring range (say 0..100), when scoring very low there is simply a lot of room for overestimating yourself and when scoring very high there is simply a lot of room for underestimating yourself. So all noise ends up adding to the inverse correlation naturally.
Re: The Dunning-Kruger Effect Is Autocorrelation
#129Very interesting article and statistical analysis, but I really don't see how it concludes that the DK effect is wrong based on the analysis. The fact that the DK effect emerges with _completely random data_ is not surprising at all - in this case the intuitive null hypothesis would be that people are good at estimating their skill, therefore there would be strong a correlation between their performance and self-eval…
> therefore there would be strong a correlation between their performance and self-evaluation of said performance. If the data weren't related, then this hypothesis isn't likely, which is exactly what DK means. DK doesn't mean no correlation, it means inverse correlation. It's the correct analysis at the bottom that shows what no correlation actually looks like (at least no correlation in tend, there is heteroskedast…
> DK doesn't mean no correlation, it means inverse correlation.
No, it means that people's self-assessment, their prediction of what their test scores will be, is uncorrelated (or more precisely weakly correlated--that's what the original D-K data showed) with their actual test scores. Which is not what we would expect: we would expect that their predictions of their test scores would be strongly (or at least more strongly) correlated with their actual test scores. The question the D-K effect raises is why that is not the case, and it's a valid question--one which this article does not even attempt to answer.
Re: The Dunning-Kruger Effect Is Autocorrelation
#130Seems like a half-baked analysis. You would plot x=x to show where y is above and where it is below. It is useful for exposition. The author questions this as if it is an analytical oversight.