Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dunning-Kruger effect may just be a data artefact (2020)

mcgill.ca

Post reply on HN
Print view View original

181–190 of 191 posts

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#181

Post by dolni »

pibaker wrote: post #165
belviewreview wrote: post #136

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

We want theories to be backed by more rigorous data precisely because "having run into this many times in their own lives" is not reliable. People's own perception is incredibly biased. We can see a bias at play in your very own closing sentence — "your study is bad if it doesn’t not confirm my existing belief" is plainly not science.

It's not science, but also a lot of stuff that purports to be science is not. "Science" is subject to many of the same forces that politics is.

Using your personal experiences and the personal experiences of people you trust as an anchor is a good thing. It keeps you from falling for propaganda. Note that this DOES NOT mean studies conflicting with your personal experience are all wrong.

The tricky part here is understanding the limit of what your own experience can tell you.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#182

Post by program_whiz »

klustregrif wrote: post #180
program_whiz wrote: post #175

Interesting, maybe the fact that it is replicated by random noise is actually confirming the theory though. The article claims "completely random guessing replicates the DK graph". But that is compatible with "the ability to assess one's skill is not correlated with measured skill". If random guesses reproduce DK, it implies that people might be randomly guessing at their level of skill. As a result people with littl…

So your claim is that if there is no correlation between A and B then it proves that there is in fact negative correlation between A and B because there is no correlation? And so the Dunning Kruger effect becomes true of all things that are uncorrelated?

not really, my claim is that if random data reproduces the trend (e.g. experts undepredict, layment overpredict skill), then it might actually make the argument stronger than the traditional explanation. The traditional explanation is that there is some secret bias where idiots think they are smart and geniuses think they are dumb. A stronger explanation supported by the random data example is "people can't assess their skill at all". The expert / laymen tails appear only because they are at the high/low end of the distribution and there isn't any higher/lower to go, so the errors go towards the middle. So there isn't a bias its just that the ability to understand skill is a skill itself, so the random errors look like a bias.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#183

Post by program_whiz »

klustregrif wrote: post #180
program_whiz wrote: post #175

Interesting, maybe the fact that it is replicated by random noise is actually confirming the theory though. The article claims "completely random guessing replicates the DK graph". But that is compatible with "the ability to assess one's skill is not correlated with measured skill". If random guesses reproduce DK, it implies that people might be randomly guessing at their level of skill. As a result people with littl…

So your claim is that if there is no correlation between A and B then it proves that there is in fact negative correlation between A and B because there is no correlation? And so the Dunning Kruger effect becomes true of all things that are uncorrelated?

And it would only be true of things that are uncorrelated with an ascending sequence. As an example, I hand 4 people envelopes filled with a random amount of cash. Each looks at the cash and I ask what rank they think they are (1 to 4, 1 being least cash received). Since its a random guess, it will look like "people with the most cash underestimate their rank" and people with the least cash "overestimate their rank". The truth is, they have no information, they are just guessing. The rank and guess are uncorrelated, but it shows the DK effect (but its not because "people with more money tend to estimate their rank lower", its due to randomness). You'd probably see something like this:

Cash Quartile: 1 2 3 4 Average guess: 2.2 2.4 2.3 2.5

And of course it would vary based on cash. If you hand someone $1M maybe they guess correctly they are the highest, but if everyone has $50-$60 dollars, everyone will probably guess 2 or 3. If you're in a class with a bunch of people in your major taking a random test, how well you think you did relative to others is a similar question (or how many questions you got right, if you don't already know).

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#184

Post by belviewreview »

pibaker wrote: post #165
belviewreview wrote: post #136

The reason people find the DK effect persuasive is that everyone has run into this many times in their own lives. Like with an idiot boss who thinks he knows your job better than you do, or an article by someone that gets things completely wrong about a topic that you yourself know very well, through personal experience or extensive study. And we have all been dismayed about the impact such people can have, like thro…

We want theories to be backed by more rigorous data precisely because "having run into this many times in their own lives" is not reliable. People's own perception is incredibly biased. We can see a bias at play in your very own closing sentence — "your study is bad if it doesn’t not confirm my existing belief" is plainly not science.

I was explaining why people believe it. I think they are often correct in their belief. So let me ask you, have you experienced this yourself, like for a subject that you have a good knowledge of, observed someone who claims they do but is wildly off track?

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#186

Post by qsera »

dolni wrote: post #181
pibaker wrote: post #165

Earlier quoted context omitted.

We want theories to be backed by more rigorous data precisely because "having run into this many times in their own lives" is not reliable. People's own perception is incredibly biased. We can see a bias at play in your very own closing sentence — "your study is bad if it doesn’t not confirm my existing belief" is plainly not science.

It's not science, but also a lot of stuff that purports to be science is not. "Science" is subject to many of the same forces that politics is. Using your personal experiences and the personal experiences of people you trust as an anchor is a good thing. It keeps you from falling for propaganda. Note that this DOES NOT mean studies conflicting with your personal experience are all wrong. The tricky part here is under…

> It's not science, but also a lot of stuff that purports to be science is not. "Science" is subject to many of the same forces that politics is.

I am glad that people are finally waking up to this.

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#187

Post by qsera »

specialist wrote: post #176
jeffnash wrote: post #156

Earlier quoted context omitted.

Related: The Gell-Mann Amnesia Effect. In Michael Crichton's own words: "Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents th…

Michael Crichton also was an outspoken skeptic of climate crisis. Crichton's Law: unwittingly serving as the best example of one's own snarky law.

[flagged]

Re: The Dunning-Kruger effect may just be a data artefact (2020)

#188

Post by specialist »

qsera wrote: post #187
specialist wrote: post #176

Earlier quoted context omitted.

Michael Crichton also was an outspoken skeptic of climate crisis. Crichton's Law: unwittingly serving as the best example of one's own snarky law.

[flagged]

I can't wait to hear your take. Please. Continue.
Post reply on HN