Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

61–70 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#61

Post by sam0x17 »

galaxyLogic wrote: post #3

I think Dunning Krueger makes intuitive sense. When you become skilled in your field you learn from other people in your field, and your assessment of yourself is based on your relation to the skills of those other people. But if you know very little about something, you have no reference point to evaluate yourself against. When you learn something you also learn what are some of the mistakes you can make. You evalua…

Put another way, to get good at something, you have to get good at self-assessing your performance at that thing, or you have no way of advancing.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#62

Post by quanto »

The conclusion in the article:

> Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a lot of that is related to people’s distrust in science (and statistics in particular). Something about the way the author conveniently swapped “purely random” with “null hypothesis” (when it’s inappropriate!) and happily went on to call the authors “unskilled and unaware of it”, and about the ease with which people jumped on to the “lies, damned lies, statistics” wagon but were very stubborn about getting off, got to me. Deeply. I couldn’t let this go.

I am afraid I actually agree with the author's point. The anti-intellectual, anti-scientific streak in many poor analyses claiming to debunk some scientific research is deeply concerning in our society. If someone is trying to debunk some scientific research, at least he should learn some basic analytic tools. This observation is independent of whether the original DK paper could have been better.

That said, I give the benefit of doubt to the author of "The DK Effect is Autocorrelation." It is a human error to be overly zealous in some opinions without thinking it through.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#63

Post by zharknado »

Thanks for writing! Really valuable rebuttal imo.

I’m not a statistician but I do have some basic training in psychometrics. It might be interesting/helpful to point out that your priors about self-assessment seem more reasonable generally but also put a lot of faith in the test’s validity as a measure of skill.

I’m relying on intuition here, but it seems a little problematic that the actual score and the predicted score are both bound to the same measurement scheme. Given that constraint on some level we’re not really talking about an external construct of skill, just test performance and whether people estimate it well. Which is different from estimating their skill well.

Maybe someone with more actual skill can elaborate or correct haha.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#64

Post by 0x20cowboy »

MichaelBurge wrote: post #56

The plot to me always read "People estimate themselves at 60-70% percentile - above average, but not the best". And then given this broad prior, people do place themselves accurately(because the plot is increasing). So it seems people are bad at doing global rankings. If I tried to rank myself amongst all programmers worldwide, that seems really hard and I could see myself picking some "safe" above-average value just…

60% of the time, it works every time.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#65

Post by sumanthvepa »

I read about DK and I was absolutely convinced that the effect was real. Then I read the article about DK being mere autocorrelation and I came away absolutely convinced that DK was bullshit. Then I read this article and I'm absolutely convinced that the 'DK is autocorrelation' hypothesis is utter BS. Sigh. There are lies, damned lies and statistics... :-)

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#66

Post by weird-eye-issue »

sumanthvepa wrote: post #65

I read about DK and I was absolutely convinced that the effect was real. Then I read the article about DK being mere autocorrelation and I came away absolutely convinced that DK was bullshit. Then I read this article and I'm absolutely convinced that the 'DK is autocorrelation' hypothesis is utter BS. Sigh. There are lies, damned lies and statistics... :-)

Sounds like DK in effect

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#67

Post by closed »

jldugger wrote: post #25

> Again, my main point is that there’s nothing inherently flawed with the analysis and plots presented in the original paper. I find the use of quartiles suspicious, personally. It's very nearly the ecological fallacy[1]. > I’m not going to start reviewing and comparing signal-to-noise ratios in Dunning-Kruger replications DK has been under fire for a while now, nearly as long as the paper has existed[2]. At present,…

FWIW extreme groups (e.g. using upper and lower quartiles) is well understood in its inflation of effect size (there are even formulas to correct this, given an extreme groups design).

It's definitely related to ecological fallacy in the sense that both underestimate relative error and inflate effect sizes.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#68

Post by [deleted] »

0x20cowboy wrote: post #64
MichaelBurge wrote: post #56

The plot to me always read "People estimate themselves at 60-70% percentile - above average, but not the best". And then given this broad prior, people do place themselves accurately(because the plot is increasing). So it seems people are bad at doing global rankings. If I tried to rank myself amongst all programmers worldwide, that seems really hard and I could see myself picking some "safe" above-average value just…

60% of the time, it works every time.

[deleted]

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#69

Post by jameshart »

marcholagao wrote: post #48

tldr; D+K's experiment was: Assign the numbers 1 thru 10 to ten people. Have each role a 10 sided die. The person assigned a 1 will roll higher than his assigned number 90% of the time. Daniel: >It’s not a “statistical artifact” - that will be your everyday experience living in such a world. You can experience statistical effects. I think a lot of controversy comes from how Dunning and Kruger's paper leads people to…

This is such a bizarre argument.

Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance!

If you accept the 'D-K is autocorrelation' argument, you don't get to throw out the existence of the D-K effect: you are saying Dunning + Krueger failed to show that humans have any ability to estimate how skilled they are at all.

That seems like an even more radical position than the D-K thesis.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#70

Post by jasonhong »

sumanthvepa wrote: post #65

I read about DK and I was absolutely convinced that the effect was real. Then I read the article about DK being mere autocorrelation and I came away absolutely convinced that DK was bullshit. Then I read this article and I'm absolutely convinced that the 'DK is autocorrelation' hypothesis is utter BS. Sigh. There are lies, damned lies and statistics... :-)

Consider taking a more Bayesian view of the world, especially with scientific papers. I informally tell the students I work with to look for a constellation of papers that offer supporting evidence from multiple perspectives.
Post reply on HN