A similar argument was made a couple of years ago, here's the rebuttal of the argument from back then [2022]: https://andersource.dev/2022/04/19/dk-autocorrelation.html
The Dunning-Kruger effect may just be a data artefact (2020)
71–80 of 192 posts
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#72Very hard to understand the meat behind all the fluff of the article, especially as the simulation code is not available, and as the presented simulated and original graphs are effectively the same (I don't see a disagreement). It's clear that the perceived curve will be differently sloped, as no one will evaluate themselves as the topmost or the bottommost percentiles, so the edges will be biased. And if in both cas…
Huh? The point is that the two graphs come out looking the same, making the original no more meaningful than random. Source code is here: https://github.com/pem725/Dunning-Kruger (found here: https://pem725.github.io )
Except "two graphs look the same" in no way means "therefore the results are equivalent"
The article is baseless and fluffy
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#73The key claim: "Random data actually mimics the effect really well." This makes some sense. If people are asked to guess a number between 1 and 6 and then roll a die, the people who roll low are more likely to overestimate and the people who roll high are more likely to underestimate. But the key is precisely how well random data mimics the effect.
They're simulating randomness incorrectly: relationship between true and perceived will average 0.5, not 0; and bias will average 50%, not 0%. That's why their "random data" is sloped.
Add negative relationship and negative bias, and the random data will act as intended - hovering randomly around 50%.
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#74Earlier quoted context omitted.
I've seen at least 3 different ideas on this topic: - people not skilled in a thing are bad at estimating their skills - people are generally bad at estimating their skills - people skilled in a particular areas often feel they are intellectually fit in other areas Out of these three I feel like there's some truth in it, at least anecdotally.
> people skilled in a particular areas often feel they are intellectually fit in other areas This I think is a separate phenomenon, maybe Nobel Disease but there might be a more general term, for example that includes celebrities.
It's called "engineer's disease" which is pretty appropriate for this community:
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#75It’s obviously real, at least as used in conversation, whether it meets some rigorous definition I’m sure there’s an out, but we’ve all known these people. With vibe coding they’re everywhere. Is this going to be a modern “begging the question” where everyone knows what you mean but someone pipes up that actually the technical meaning is different?
[dead]
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#76The key claim: "Random data actually mimics the effect really well." This makes some sense. If people are asked to guess a number between 1 and 6 and then roll a die, the people who roll low are more likely to overestimate and the people who roll high are more likely to underestimate. But the key is precisely how well random data mimics the effect.
They're simulating randomness incorrectly: relationship between true and perceived will average 0.5, not 0; and bias will average 50%, not 0%. That's why their "random data" is sloped. Add negative relationship and negative bias, and the random data will act as intended - hovering randomly around 50%.
They, themselves, show the Dunning-Kruger effect by overestimating how much they understand randomness
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#77Earlier quoted context omitted.
I've seen at least 3 different ideas on this topic: - people not skilled in a thing are bad at estimating their skills - people are generally bad at estimating their skills - people skilled in a particular areas often feel they are intellectually fit in other areas Out of these three I feel like there's some truth in it, at least anecdotally.
I think the third one is a separate thing. I don’t know of a name for it, but I’d call it the engineer’s (or physicist’s) blindspot or something like that. I can’t remember of DK suggests some sort of effect where the expert has undue self-doubt, though…
> I can’t remember of DK suggests some sort of effect where the expert has undue self-doubt, though…
There is a brief bit where they attribute that observation to other causes:
In making these predictions, we felt that we could account for an anomaly that appeared in all three previous studies: Despite the fact that top-quartile participants were far more calibrated than were their less skilled counterparts, they tended to underestimate their performance relative to their peers. We felt that this miscalibration had a different source then the miscalibration evidenced by bottom-quartile participants. That is, top-quartile participants did not underestimate themselves because they were wrong about their own performances, but rather because they were wrong about the performances of their peers. In essence, we believe they fell prey to the false-consensus effect (Ross, Greene, & House, 1977). In the absence of data to the contrary, they mistakenly assumed that their peers would tend provide the same (correct) answers as they themselves—an impression that could be immediately corrected by showing them the performances of their peers. By examining the extent to which competent individuals revised their ability estimates after grading the tests of their less competent peers, we could put this false-consensus interpretation to a test.
Link if you need it:
https://www.researchgate.net/publication/12688660_Unskilled_...
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#78Replication crisis. More than half of all psychology studies are not reproducible. I'm at the point honestly, where I don't even consider psychology to be a science anymore.
If you ever took psych 101, it was immediately obvious it's mostly horse shit. Made up nonsense theory, with studies that have few participants and are done by people who don't know math.
It’s pretty handy as a signal. The moment someone starts banging on about Jungian this and Freud that, I know immediately to discount anything they say, ever.
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#79Replication crisis. More than half of all psychology studies are not reproducible. I'm at the point honestly, where I don't even consider psychology to be a science anymore.
The replication crisis is not limited to psychology, that was just the first area where it was noted. I saw some meta-study comparing multiple social sciences and according to it replication failure in sociology and educational science was even worse than in psychology.
Re: The Dunning-Kruger effect may just be a data artefact (2020)
#80Earlier quoted context omitted.
If you ever took psych 101, it was immediately obvious it's mostly horse shit. Made up nonsense theory, with studies that have few participants and are done by people who don't know math.
It is both amazing and saddening to me how you can have such an arrogant view of a whole field with just taking a 101 course.
Take physics 101 (or something, anything from a hard science) and understand how they know what they claim to know, and compare it to psych. It's night and day.
One doesn't need to continue learning a topic to know it's bs when it's bs right from the start.