Live data from Hacker News

The Dunning-Kruger effect may just be a data artefact (2020)

mcgill.ca

151–160 of 191 posts

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#151
post #133

I first ran across the Dunning-Kruger effect in my own data: a survey of my classmates which asked them 34 questions about their study habits, and a question in the middle asked them what quintile of the class they believed they stood in. Even in this completely self-reported sample, the data strongly pointed to a Lake Wobegon bias in self-assessment - everyone self-assessed as above average, despite having more than…

One problem here is that you know about the outcome. If the outcome is going to be public, this creates a different type of competing pressure, where classmates are invited to compete with each other, while the one they should be competing with, is themselves.

It is like you have a teacher saying: 'my previous year's group, they were like so amazing [exhibit 1, 2, 3, ...]' and you know your group is never going to reach those limits -> #very_demotivational since in many ways, the way people are isn't going to be malleable. So a teacher is going to have to work with the material they got, and take that as a starting point. So instead, the teacher needs to have the intelligence acting like an adult, and refraining from expressing said enthusiasm.

I believe, if the results are discussed 1[-3] on 1 ie. in private chat, there is more room for openness, honesty, humbleness, and -yes- humility. On a macro level, this also exists on schools. Like, when the parents come to school to discuss their child's performance, it is a private setting without the other children and other parents, but with teacher, child, and parents. 1[-3] is still a private setting, but there are still things the child would only discuss with their parents, or only with their teacher. You could also achieve this in small groups on school, but as soon as you got a too big group there's this huge competing factor going out. Which is going to trample individuals. The shy kid isn't going to have their say, the ADHD kid gets annoying, ... and it can be tackled, with smaller classes.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#152

publishing auto-correlation is not a 'data artefact', but a mistake. and a rather ironic one at that. The #Dunning-Kruger Effect is #Autocorrelation https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...

another modelling problem with it: ceiling effect https://news.ycombinator.com/item?id=38416412 How much can your top performers overestimate their performance? The opposite problem happens for the worst performers.

A Statistical Explanation of the Dunning–Kruger Effect https://www.frontiersin.org/journals/psychology/articles/10....

The DK effect says roughly, "low performers tend to overestimate their abilities." Yet when researchers analyzed the data, they found that high and low performers overestimate and underestimate with the same frequency. [0] It's just that high performers are more accurate than low performers (note how this statement differs from the DK effect). Since you can completely explain the "X graph" by the random noise combined with the ceiling effect, and since beginners' self evaluations are noisier than experts', you don't even need regression to the mean to explain why you get the "X graph."

0. Nuhfer, Edward, Steven Fleisher, Christopher Cogan, Karl Wirth, and Eric Gaze. "How Random Noise and a Graphical Convention Subverted Behavioral Scientists' Explanations of Self-Assessment Data: Numeracy Underlies Better Alternatives." Numeracy 10, Iss. 1 (2017): Article 4. DOI: http://dx.doi.org/10.5038/ 1936-4660.10.1.4

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#153
The chart they show from the original paper is approximately just:

    perceived_ability = 0.5 + (actual_ability / 4)
(With the caveat that "ability" is a percentile rather than some absolute measure.)

That's not the negative correlation that it's often presented as in discussions. And it could as easily relate to mis-measuring everyone else's skill as to mis-measuring your own skill.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#154

The strict academic definition hasn’t followed the colloquial usage for a long time. Maybe ever: If a specific novice is over-confident and out of their depth, we say “Dunning-Kruger” If a specific is under-confident and performing better than their self-estimate, that’s not commonly considered Dunning Kruger, in the colloquial use. It’s called imposter syndrome, or not labeled at all. The researchers aren’t really d…

I think people overlook the natural experience of Dunning-Kruger we all have. You think you are good at chess because you win against your neighbour, then eventually a class mate absolutely embarrasses you multiple times and you realize there is a whole world you don't know, and before now you didn't know that world existed. Then you eventually beat your classmate before getting whomped by the beginner in the chess c…

That one time I was able to beat a 'good' chess player at school :-) I still remember that one. Usually I lost, but... I also came to the chess club rather to play Magic: the Gathering, so I didn't mind.

With that in mind, it has been a while (more than twenty five years) but what you said sums up the one time I went to my first Magic: The Gathering tournament, by having to build a deck with a limited amount of packs. You don't have the advantage of owning any power cards there (unless if you are lucky). What happened? Well, you could see who was doing well based on where they sat. And, I got absolutely wrecked! But, I also had a very bad (useless) rare [Shauku, Endbringer]. Which I shouldn't even have put in my deck, but I still did, cause.. it was a rare! [1] Still, there's much more luck to be had with decent commons and uncommons. In later tournaments (different types), I did better, and my fav experience (apart from some of my cards being stolen on a later tournament at that location) was the one where you'd buy a bunch of packs, open a pack up, and pass the packs on. Some people would pick only meta cards, some people would only pick expensive ones, others would always pick rare ones. If you don't intend to win, that could be a decent strategy. But I went for the strategy where I would pick the best cards for my deck, and that worked out well.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#156

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

Related: The Gell-Mann Amnesia Effect. In Michael Crichton's own words:

"Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know."

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#157

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

Your anecdote is not describing dunning-kruger, though, it's describing "some people exist in society with too much confidence in their ability" which can be true on its own, and is not exclusive to beginners.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#158

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

>If the research can't demonstrate the DK effect, then the research is bad, like poorly designed.

Why do the research if you are so convinced from your own personal anecdotes? Quite a weird thing to say...

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#159

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

> If the research can't demonstrate the DK effect, then the research is bad, like poorly designed.

What? This is anecdotal and subjective. If the research doesn’t back up those anecdotes, then the research doesn’t back up those anecdotes. You want the research to back up your lived experience because it’s validating. That doesn’t mean that’s the actual reality.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#160
post #7

Even if it isn’t true, it’s got the feeling of truthiness (1). I don’t expect it to ever go out of the public consciousness. Like other things that were never real like Stockholm Syndrome I suspect it’s just stuck in the zeitgeist now. 1. https://en.wikipedia.org/wiki/Truthiness

Power poses, imposter syndrome, Stanford prison experiment, marshmallow test... It seems like the overlap between "real psychological effect" and "subtle enough that it requires research to discover" is vanishingly small. I guess that's not really surprising.

Freudian theories as well - relevant SMBC.

https://www.smbc-comics.com/comic/dongworld

Post reply on HN