Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

151–160 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#151

Post by sandgiant »

kybernetikos wrote: post #112

Would it be possible to understand the results differently? It looks to me that the data could be explained by the participants moderating their self assessment away from extremes or perhaps towards the population mean which is arguably not an unreasonable thing to do if your knowledge of the population mean is better than your knowledge of your own performance.

And this is why we need error bars on all plots. Looking at these plots there is no way to know whether people guessed uniformly or whether the self assessment is clustered around the mean.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#152

Post by notahacker »

slavik81 wrote: post #137
jameshart wrote: post #69

Earlier quoted context omitted.

This is such a bizarre argument. Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance! If you accept the 'D-K is autocorrelation' argument, you don't get to throw out the existence of the D-K effect: you are saying Dunning + Krueger failed to show that humans have any ability to estimate how skilled they are at all . That seems like an even more radical position…

> Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance Isn't DK about estimating your performance relative to the rest of the population ? To do that, you need to not only know your own performance but also everyone else's. To me, guessing the performance of others sounds quite difficult.

The implicit hypothesis is that if a test is full of questions you have no idea how to answer, you really ought to have strong priors that you're a below average performer. Tests are generally designed so that people familiar with the relevant material and methodologies can attempt answers; you dont need to know exactly how good other test takers are for it to be reasonable to assume you're in the bottom quartile if you can't. Same as you should have a lot less difficulty than most cyclists estimating whether your time trial was a good one relative to the rest of the field if you struggled to stay on the bike.

Of course, there are tests where the bottom quartile find the majority of it easy and have no particular reason to assume that most others found it even easier, and circumstances in which the weak undergrad who can only answer half the questions may reasonably believe that the test is being administered to a general population full of people who won't understand any of the material at all. But in general, it's reasonable to assume that if there's a lot of stuff you don't know, other people will know better.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#153

Post by coffeeblack »

pavlov wrote: post #136
coffeeblack wrote: post #128

Earlier quoted context omitted.

That happens when science is politicized, and any scientists critical of the “official” results is destroyed. From climate to Covid, so many areas where that happens.

If by “destroyed” you mean “billionaires will happily fund their contrarian research for years regardless of its peer reviews”…

No, I mean researchers and professors suddenly not getting any research grants anymore, suddenly getting fired from their tenured jobs, not being invited anymore to conferences, etc.

What billionaire funds “contrarian” research?

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#154

Post by qfwfq_ »

c1ccccc1 wrote: post #116
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

It still seems to me like "The DK Effect is Autocorrelation" is basically correct. The important thing isn't whether or not independence should be the null hypothesis, because calling something a "null hypothesis" is just an arbitrary label that doesn't affect reality. The important thing is that what we can actually conclude from the Dunning-Kruger paper is a lot less than popular presentations of the concept claim.…

No, nulls matter a great deal. If you want to test a claim in Null Hypothesis Statistical Testing, the "significance" of the claim is in direct reference to the null. Changing a null will change the significance of the alternative. My favorite statement of this is from Gelman:

> the p-value is a strongly nonlinear transformation of data that is interpretable only under the null hypothesis, yet the usual purpose of the p-value in practice is to reject the null. My criticism here is not merely semantic or a clever tongue-twister or a “howler” (as Deborah Mayo would say); it’s real. In settings where the null hypothesis is not a live option, the p-value does not map to anything relevant.

https://statmodeling.stat.columbia.edu/2017/01/07/we-fiddle-...

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#155

Post by formerly_proven »

seniortaco wrote: post #143

Beyond the validity of the statistical methods used.. can someone clarify what is the actual hypothesis we are debating about competence? And what does each article propose? My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves". DK says: True DK is Autocorrelation says: ??? "I cant let go..." says: True? HN says: also True? Is there really…

> My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves".

The DK hypothesis is "double burden of the incompetent": "Because incompetent people are incompetent, they fail to comprehend their incompetence and therefore overestimate their abilities more than expertes underestimate theirs"

Arguably the hypothesis that matches the data from the DK paper best is: "Everyone thinks they're average regardless of skill level"

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#156

Post by sandgiant »

darawk wrote: post #86
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

I agree, but I don't think this is limited to science. I think a great deal of everything is in fact trash. This is why we need education and good faith discussion.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#157

Post by gadders »

darawk wrote: post #86
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

Science itself is the best method we have for exploring and making sense of the world around us. The method is rock solid.

In between us (gen pop) and The Method are scientists, and scientists are just as fallible as any other group of people - lawyers, politicians, coders, shop assistants.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#158

Post by gadders »

sandgiant wrote: post #156
darawk wrote: post #86

Earlier quoted context omitted.

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

I agree, but I don't think this is limited to science. I think a great deal of everything is in fact trash. This is why we need education and good faith discussion.

https://en.wikipedia.org/wiki/Sturgeon%27s_law

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#159

Post by t_mann »

It can give us an indication of how the growth rate depends on size

Except that what you've plotted there isn't the growth rate, but the absolute growth. Your argument for DK isn't convincing either, they claimed sth much stronger than that we can't assess our own skills.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#160

Post by dragonwriter »

formerly_proven wrote: post #155
seniortaco wrote: post #143

Beyond the validity of the statistical methods used.. can someone clarify what is the actual hypothesis we are debating about competence? And what does each article propose? My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves". DK says: True DK is Autocorrelation says: ??? "I cant let go..." says: True? HN says: also True? Is there really…

> My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves". The DK hypothesis is "double burden of the incompetent": "Because incompetent people are incompetent, they fail to comprehend their incompetence and therefore overestimate their abilities more than expertes underestimate theirs" Arguably the hypothesis that matches the data from the…

> The DK hypothesis is "double burden of the incompetent"

The actual DK result (which is much criticized, but that's a different issue) was actually a pretty much linear relationship between actual relative performance and self-estimated relative performance, crossing over at about the 70th percentile.

(Because there is more space below 70 than above, that also means that the very bottom performers overestimated their relative performance more than top performers underestimated, not because of any “double burden” (overstimation didn't rise faster as one moved below the crossover), but just because there was more space below the crossover point.

> Arguably the hypothesis that matches the data from the DK paper best is: "Everyone thinks they're average regardless of skill level"

If there was a perceptual nudge toward average relative performance, you'd expect a crossover at the median with a slope below 1, the nudge is toward a particular point above average.

Post reply on HN