Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

281–290 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#281

Post by SamoyedFurFluff »

lelanthran wrote: post #275
SamoyedFurFluff wrote: post #251

Earlier quoted context omitted.

> I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently. Are you genuinely serious or were you completely unaware of anti-communist government sanctioned blacklisting of academics suspected of being communist for political clout? Are you unaware of churches excommunicating Galileo for daring to scholarly research into the earth rotating around the sun? Are you unaware of…

> Are you genuinely serious or were you completely unaware of anti-communist government sanctioned blacklisting of academics suspected of being communist for political clout? Are you unaware of churches excommunicating Galileo for daring to scholarly research into the earth rotating around the sun? Are you unaware of our own Alan Turing, of the Turing award, literally castrated not for his research but because he was…

If you think castrating gay men wasn’t mob Justice at the time maybe you should reconsider. Same for whether or not black scholars were considered equal to white ones.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#282

Post by fshbbdssbbgdd »

MichaelBurge wrote: post #56

The plot to me always read "People estimate themselves at 60-70% percentile - above average, but not the best". And then given this broad prior, people do place themselves accurately(because the plot is increasing). So it seems people are bad at doing global rankings. If I tried to rank myself amongst all programmers worldwide, that seems really hard and I could see myself picking some "safe" above-average value just…

Thank you! This article creates a dichotomy where our hypothesis must be either

1) self-assessment is perfectly correlated with skill, or

2) completely uncorrelated.

I think neither of these makes sense as a null hypothesis.

The model you describe matches my intuition about what we should expect: people know something about their own skill level, but not everything.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#283

Post by LaurieKoudstaal »

mewse wrote: post #123
clwk wrote: post #44

Earlier quoted context omitted.

"Most believe themselves 'above average' at most things."

Most people have an above-average number of legs. There's really no contradiction there; all it takes is for there to be a couple low scores pulling the average down.

In the context of the paper, we should be talking about the median, not the mean.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#284

Post by samhw »

mike_hearn wrote: post #277
samhw wrote: post #272

Earlier quoted context omitted.

> The paper reports on the results of a 'logic' test administered to undergrads and uses this to define competence. Right, but my point is that 'logic' is simply being used as an example of 'a task'. It's immaterial whether it's actually a good test of logic. As long as you agree that whatever it is is a good example of 'a task', then it's equally probative for the purpose of their argument.

The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. That's why DK is a well known effect, it claims to hold true for anything even though they can't test every possible task. > we presented participants with tests that assessed their ability in a domain in which knowledge, wisdom, or savvy was crucial: humor (Study 1), logical reasoning (Studies 2-and 4), and English gram…

> The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence.

I’ve seen absolutely nothing suggesting this. It’s explicitly about task competency; no particular task is specified nor needs to be specified.

> That's why DK is a well known effect, it claims to hold true for anything even though they can't test every possible task.

Yes, they claim it holds true for everything because it’s how human beings introspectively experience being poor at a task. It’s really not necessary to have some Platonic ideal of Task Competency … which is then specifically restricted to logical tasks for reasons known only to you.

> Logical reasoning was chosen because: It’s objective.

I think there’s a kernel of truth in this, albeit assuming by ‘objective’ you instead mean (as people often do) something like “people almost always agree in their evaluations of this quality”. You need that for a good experiment. I’m still not sure how it relates at all to your point here. Personally I would find it easier to just say “I was wrong, it’s not explicitly about logic, I just associated it with that because it’s commonly adduced in silly arguments about logic/intelligence on the internet” - but ah well, it’s an interesting theory so I’m happy to discuss it.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#285

Post by mike_hearn »

samhw wrote: post #284
mike_hearn wrote: post #277

Earlier quoted context omitted.

The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. That's why DK is a well known effect, it claims to hold true for anything even though they can't test every possible task. > we presented participants with tests that assessed their ability in a domain in which knowledge, wisdom, or savvy was crucial: humor (Study 1), logical reasoning (Studies 2-and 4), and English gram…

> The tasks aren't arbitrary. They're meant to be a proxy for some universal concept of competence. I’ve seen absolutely nothing suggesting this. It’s explicitly about task competency; no particular task is specified nor needs to be specified. > That's why DK is a well known effect, it claims to hold true for anything even though they can't test every possible task. Yes, they claim it holds true for everything becaus…

The reason no particular task needs to be specified to invoke DK is exactly because they argue that their initial selection of experimental tasks is so general, that the effect must apply to everything.

It feels like you and danbruc are inverting causality here. You start from the assumption that DK is a real effect and then say, because it's real and general, it doesn't matter what tasks they used to prove it. But that's backwards. We have to start from the null hypothesis of no effect existing, and then they have to present evidence that it does in fact exist. And because they claim it's both large and very general, they need to present evidence to support both these ideas.

That's why they explicitly argued that their tasks reflect general attributes like wisdom and intelligence: they wanted to be famous for discovering a general effect, not one that only applies in very specific situations.

But their tasks aren't great. The worst are ridiculous, the best are unverifiable. Thus the evidence that DK is a real and general effect must either be taken as insufficient, or you could widen the argument to include studies by other psychologists that pursue the same finding via different means.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#286

Post by specialist »

darawk wrote: post #86
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

In other words, emphasize the process more than the outcomes. If the scientific process used proves sound, then I have more confidence in the outcomes.

Alas, that's a hard sell to laypersons thru the mediums of soundbites and tweets.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#287

Post by kortilla »

HWR_14 wrote: post #245
coffeeblack wrote: post #153

Earlier quoted context omitted.

No, I mean researchers and professors suddenly not getting any research grants anymore, suddenly getting fired from their tenured jobs, not being invited anymore to conferences, etc. What billionaire funds “contrarian” research?

You could build your whole career from the 50's on discrediting the link between smoking and cancer. The Koch brothers were (and the current brother and estate of the other) are happy to write economists checks to prove laissez-faire capitalism under a libertarian government is the best system. Or that climate change isn't real. If you're willing to generate evidence climate change isn't real, Exxon etc have some nic…

> You could build your whole career from the 50's on discrediting the link between smoking and cancer.

> If you're willing to generate evidence climate change isn't real, Exxon etc have some nice checks for you.

Do you have any evidence of either of these being true today?

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#288

Post by kortilla »

Pyramus wrote: post #219
jtc331 wrote: post #210

Earlier quoted context omitted.

One does not need to have done social science research to be able to recognize obvious general philosophy of science level problems with the methods used in much social science. I’d we take your claim seriously then we have to disallow all critiques of the replicatability crisis in the social sciences that don’t come from social scientists, but that would present an obvious new problem: conflict of interest. It’s als…

You are correct - I should have been more precise: I hypothesis parent has not done science research professionally (again happy to be proven wrong). Don't get me wrong, I'm not defending social science research per se (yes, there are questionable methods). I'm critiquing parent who has high confidence in pointing out issues with the DK paper, yet misses the real issues. Which, in the context of discussing whether th…

None of your points actually address the sample size and study design issues that wouldn’t be unacceptable even in social sciences today. Generalizing results from a fistful of privileged undergrads is a well-known issue even in the community.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#289

Post by pastage »

lelanthran wrote: post #194
pastage wrote: post #173

Earlier quoted context omitted.

No this is not new. You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superior. There is still "science" being done to bolster those political views.

>> What's new is that any research that might produce results counter to the what the PC-mob deems acceptable is attacked. > No this is not new. I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently. > You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superio…

Mobs that try to force their own views on others are common. PTAs, or strong willed individuals with kids in school, are mob experts. Good things have come out of those mobs. Amsterdams good bicycle infrastructure was built on mob rule instead of listening to "rational engineers" who wanted to build motorways through the city.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#290

Post by robocat »

JetAlone wrote: post #104
javajosh wrote: post #29

Earlier quoted context omitted.

>I think Dunning Krueger makes intuitive sense. If human cultures can be characterized as default arrogant or default humble then it stands to reason that arrogant cultures will have a DK effect, and in humble cultures you won't.

Even people conditioned to be humble could have a strong motivation to believe something is true and overestimate their own knowledge/ability in order to stand on what they perceive as evidence. For example a person's depression, religious beliefs, or an over-emphasized belief in DK itself could be a possible reason they have an erroneously deflated opinion of themselves, and simultaneously employ inflated confidence…

> I un-learned things like "all or nothing thinking" and the expectation that I could accurately predict the outcome of any course of action

Do you know any words, stereotypes, or clichés for this? Or even what the related mental disorder is called if it were to become debilitating? Or a specific word for the complete clustering of related signals?

I am guessing those issues plus there related issues (¿syndromic?) are common - but I don’t know where to group it in my own mind.

Post reply on HN