Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

41–50 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#41

Post by jasonhansel »

aaaronic wrote: post #7

It seems like the people who want to disprove Dunning-Kruger are falling victim to it. I honestly think people take it way too seriously and apply it too generally. Quantifying "good" is hard if you don't know much about the field you're quantifying. Getting deep into a particular field is humbling -- Tetris seems relatively simple, but there are people who could fill a book with things _I_ don't know about it, despi…

> Is there an answer to that humility gained by being an expert in one field being translated to better self-assessment in other fields?

I think that often the opposite is true: people who become experts in one domain often assume that they are automatically experts in completely unrelated fields. I suspect that this is the cause of "Nobel disease": https://en.wikipedia.org/wiki/Nobel_disease

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#42

Post by jasonhansel »

haberman wrote: post #20

On a pure human level, a large portion of DK discourse seems to be a fight over which people are the "Unskilled and Unaware." Or more bluntly, who gets to call who stupid. The author says as much in this article: > Why so angry? [...] [Frankly], for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a lot of that is related to people’s distrust in science (an…

> it functions as a highbrow and socially acceptable way of calling someone else stupid

I think it's even worse that that: it's also a socially acceptable way of enforcing credentialism and looking down on others for not having a sufficiently elite education.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#43

Post by AmericanChopper »

jasonhansel wrote: post #38
parentheses wrote: post #30

What’s more interesting to me is what all the buzz over DK tells me. We are asymmetrically skeptical. In the same way as intelligent people doubt their own performance, they rightly doubt others’ performance. Maybe too much.

I think that most people who talk a lot about DK believe that they are the experts in one field or another. It serves mostly as a way of reassuring themselves of their own superiority. The message (for them) basically amounts to "other people's claim to knowledge is just further proof that they don't know anything."

It’s a zero-effort, zero-evidence-required way for people to disparage others, in a way that they believe makes them sound smart. It’s also basically unfalsifiable in most of the cases where it’s referenced.

I feel like I’m honestly yet to see somebody make DK accusations in a way that’s not totally cringe.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#44

Post by clwk »

rafaeltorres wrote: post #8

Yeah, this makes sense to me. Imagine in the Dunning-Kruger chart the second plot (perceived ability) was a horizontal line at 70, which is not true but not far off from the real results. Now imagine I told you "did you know that, regardless of their actual score, everyone thought they got a 70?" That's a surprising fact.

"Most believe themselves 'above average' at most things."

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#45

Post by obastani »

I feel like this article is severely over-complicating the analysis. Looking at the original blog post [1], their key claim appears to be that "random data produces the same curves as the DK effect, so the DK effect is a statistical artifact".

However, by "random data", the original blog means people and their self-assessments are completely independent! In fact, this is exactly what the DK effect is saying -- people are bad at self-evaluating [2]. (More precisely, poor performers overestimate their ability and high performers underestimate their ability.) In other words, the premise of the original blog post [1] is exactly the conclusion of DK!

Looking at the HN comments cited [3] by the current blog post, it appears that the main point of contention from other commenters was whether the DK effect means uncorrelated self-assessment or inversely correlated self-assessment. The DK data only supports the former, not the latter. I haven't looked at the original paper, but according to Wikipedia [2], the only claim being made appears to be the "uncorrelated" claim. (In fact, it is even weaker, since there is a slight positive correlation between performance and self-assessment.)

So, my conclusion would be that DK holds, but it does depend on exactly what is the exact claim in the original DK paper.

[1] https://economicsfromthetopdown.com/2022/04/08/the-dunning-k...

[2] https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

[3] https://news.ycombinator.com/item?id=31036800

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#46

Post by devit »

The "The Dunning-Kruger Effect is Autocorrelation" article is an example of obvious bullshit.

Their claim that "If we have been working with random numbers, how could we possibly have replicated the Dunning-Kruger effect?" is the first blatantly false statement, and then the rest is built upon that so it can be safely disregarded.

It's easy to see this because while the effect is present if everyone evaluates themselves randomly, it's not present if everyone accurately evaluates themselves, and these are both clearly possible states of the world a priori, so it's a testable hypothesis about the real world, contrary to the bizarre claim in the paper.

Also, the knowledge that the authors published that article provides evidence for the Dunning-Kruger effect being stronger than one would otherwise believe.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#47

Post by rootusrootus »

jasonhansel wrote: post #38
parentheses wrote: post #30

What’s more interesting to me is what all the buzz over DK tells me. We are asymmetrically skeptical. In the same way as intelligent people doubt their own performance, they rightly doubt others’ performance. Maybe too much.

I think that most people who talk a lot about DK believe that they are the experts in one field or another. It serves mostly as a way of reassuring themselves of their own superiority. The message (for them) basically amounts to "other people's claim to knowledge is just further proof that they don't know anything."

> I think that most people who talk a lot about DK believe that they are the experts in one field or another.

I recall that either Dunning or Kruger once made a remark to that effect. That rather than an indictment of stupid people, it would be better to view it as a warning to those who consider themselves the smart ones.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#48

Post by marcholagao »

tldr; D+K's experiment was: Assign the numbers 1 thru 10 to ten people. Have each role a 10 sided die. The person assigned a 1 will roll higher than his assigned number 90% of the time.

Daniel: >It’s not a “statistical artifact” - that will be your everyday experience living in such a world.

You can experience statistical effects. I think a lot of controversy comes from how Dunning and Kruger's paper leads people to interpret the data as hubris on the part of low-performers, and the statistical analysis demolishes that interpretation. Not knowing how well you performed is not the same thing psychologically as "overestimating" your performance.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#50

Post by _carbyau_ »

jldugger wrote: post #25

> Again, my main point is that there’s nothing inherently flawed with the analysis and plots presented in the original paper. I find the use of quartiles suspicious, personally. It's very nearly the ecological fallacy[1]. > I’m not going to start reviewing and comparing signal-to-noise ratios in Dunning-Kruger replications DK has been under fire for a while now, nearly as long as the paper has existed[2]. At present,…

Do they have to encourage replication?

If others can't replicate it entirely on their own without "encouragement". Then it isn't useful at all and the original experiment can be safely ignored as irrelevant to humanity, along with any "prestige" associated with it.

Post reply on HN