The Dunning-Kruger effect may just be a data artefact (2020)

A 101 course should not teach frontiers of research. It should have fundamental established theories upon which the rest of the field is based. My experience with introductory psychology textbooks is that they spend a lot of time discussing ideas discarded decades ago, and then they spend some time discussing ideas that were discarded after the textbook was written. There is typically one poorly explained chapter on neuroscience that the authors summarized from another textbook.

> The strict academic definition hasn’t followed the colloquial usage for a long time. Maybe ever:

This is my understanding as well, and as I recall the results of the research were also more nuanced than most people seem to indicate. So not only were the novices with a wider range of self-estimates, the average result of those perceived scores was still below the experts. So it was never that low performers thought they were experts (although this may show up in the raw results with some portion of the population tests), just that on average they perceived their performance to be better than it actually was. And the high competence group while overestimating their results, still thought they'd do better than the low performers.

Unfortunately it also is present in medicine, chemistry and increasingly physics.

No, the argument is that the best person cannot overestimate his own rank, and the worst person cannot underestimate it. The better you are the less room there is for you to overestimate your skill, second place can at most be off by one etc.

This effect would disappear almost completely if they instead of estimating their rank they estimated their score, since then unless the test is so easy the best scores perfectly there will be a lot of room for everyone to overestimate and underestimate themselves.

But as is when the top 10% all estimate themselves to be in the top 10%, you will say they are underestimate themselves since on average the top 10% are in the top 5%. At the same time if the bottom 10% say they are in the bottom 10%, you will say they overestimate themselves since actually on average they are bottom 5%. But both these groups were making the same mistake, and its impossible for that not to happen unless everyone is perfect.

Regardless of whether it's real or not, there's a significant public perception that it doesn't matter. There's a real "fake it till you make it" sentiment coupled with anti-intellectualism that will make people overestimate their ability.

On that note, I don't think the "random simulation" is described well enough. If I randomly assign a self-assessment and an actual score on a test, all of the quartiles will be the same because they should be uniformly distributed. I read through the papers mentioned and in these simulations, they hard-coded the correlation, "As in the Kruger and Dunning (1999) comparison, these random variables were correlated r=0.19"[1] so of course the graph will look similar.

On the other hand, the original Kruger and Dunning comparison could have been explained through differences in expected test scoring. It looks like people who thought they would get a D (60 percentile) or an F (50 percentile) objectively scored within the ranges of a 10 to 40 percent. While this is an overestimation, perhaps further studies can instead bucket the test scores according to how we expect them on a grading scale instead.

My takeaway is that the conclusions and discussions from the Dunning-Kruger effect study are valid. At the same time, the methodology and statistical significance is different from how popular science presents it. Also, there might be better ways to measure this phenomenon, and I would be interested in understanding how those who believe college is worth it and those who don't compare on objective testing.

[1] https://www.sciencedirect.com/science/article/abs/pii/S01918...