Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

251–260 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#251

Post by SamoyedFurFluff »

lelanthran wrote: post #194
pastage wrote: post #173

Earlier quoted context omitted.

No this is not new. You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superior. There is still "science" being done to bolster those political views.

>> What's new is that any research that might produce results counter to the what the PC-mob deems acceptable is attacked. > No this is not new. I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently. > You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superio…

> I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently.

Are you genuinely serious or were you completely unaware of anti-communist government sanctioned blacklisting of academics suspected of being communist for political clout? Are you unaware of churches excommunicating Galileo for daring to scholarly research into the earth rotating around the sun? Are you unaware of our own Alan Turing, of the Turing award, literally castrated not for his research but because he was a known gay researcher? Are you unaware of why HBSUs exist(black scholars were segregated for being black, their research dismissed because of the race of the researcher)?

Politics in academia isn’t new, like, at all.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#253

Post by dahart »

denton-scratch wrote: post #169
dahart wrote: post #78

One of the best commentaries on DK is Tal Yarkoni’s, and he came to the (perhaps similar?) conclusion that DK is probably regression to the mean. https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-... It bugs me that DK reached popular consciousness and get misinterpreted and misused more often than not. For one, the paper shows a positive correlation between confidence and skill. The paper is very clearly l…

If DK is regression to the mean (a view I find convincing) that doesn't mean the effect isn't real; i.e. one would still observe that people of low ability overestimate their ability, simply because there is more "room" for overestimates than underestimates. And v.v. Put differently, if everyone's estimate was exactly the mean, you'd still see a "DK effect".

I’m not sure I understand. If the effect shown in the paper is regression to the mean, then that does mean the paper doesn’t actually demonstrate what it claims to, right? I mean you can argue that the idea is still plausible, but this would mean that the paper doesn’t support the claim that low skill people overestimate themselves, right?

It’s also an interpretation to focus on unskilled people as the explanation. DK’s data shows the very same effect on highly skilled people. The people in the top quartile were just as bad at self-estimating as the bottom quartile, yet the paper claims only the unskilled people were unaware!

I recommend reading the DK paper. It didn’t test any people of low ability, and it did not evaluate skill in absolute terms. The sample size was tiny. The kids who participated were all earning extra credit in a class (it’s a self-selecting population that might have excluded both A students and F students.) The students were all Ivy League undergrads who might all overestimate their abilities precisely because they’re in a prestigious school and their parents told them they’re great. The paper didn’t test any actual low IQ population. The paper has methodology problems when it comes to non-native English speakers.

It absolutely blows my mind that the paper is held up as evidence for some kind of universal human trait with such miniscule and completely questionable evidence. I have no doubt that some people overestimate their abilities in some situations. Like you, I’m sure, I’ve witnessed that. But as a commentary on all of humanity, I’m becoming convinced that the so-called DK effect does not exist, that they didn’t show what they claim to show. It doesn’t help that many replication attempts have not only failed to replicate, but have ended up showing the opposite effect: that for many kinds of skilled activities, people.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#254

Post by TimPC »

I feel like the author read the autocorrelation result, hated it and ignored the central point. There are ways to bucket data that removes the autocorrelation and in those experiments we also see the DK effect disappear. Trying to argue that we should study the effect with the autocorrelation present but ignore the autocorrelation for 'reasons' is not the way forward.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#255

Post by anonymoushn »

photochemsyn wrote: post #31

Any discussion of statistics-based reasoning should include the concept of systematic bias, and that's not mentioned in this article at all. An example of systematic bias is that of an accurate but miscalibrated thermometer, where the spread of measurements at fixed temperature is small, but all measurements are off by some large factor. Now with D-K the proposed problem is statistical autocorrelation, not systematic…

> Regardless, it's fairly obvious that D-K enthusiasts are of the opinion that a small group of expert technocrats should be trusted with all the important decisions

It seems like you're roughly the only person who thinks this.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#256

Post by Cd00d »

lelanthran wrote: post #194
pastage wrote: post #173

Earlier quoted context omitted.

No this is not new. You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superior. There is still "science" being done to bolster those political views.

>> What's new is that any research that might produce results counter to the what the PC-mob deems acceptable is attacked. > No this is not new. I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently. > You have always have a direction set by political views, even if we have decided they are wrong they are still hard to kill like: smoking is good, white people are superio…

> I don't recall a PC-mob being used to silence any and all non-supportive voices until quite recently.

Scopes would like a word with you.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#257

Post by jcranberry »

oh_my_goodness wrote: post #250
jcranberry wrote: post #244

I don't really understand the article. My understanding was that the mistake was that the error bounds differ depending on the test score from the original DK paper. A test score of 0 or 100 means a potential error of 0-100, whereas a test score of 50 means a potential error of 50. So if you take a group of people who score 0-25 points, if their self-assessment is completely random you'd still see a bias of overestim…

The charts make it clear that people's self-assessment was (roughly) independent of their skill level. It's not obvious that students' self-assessment would be mostly random / unrelated to skill level. For me that's a non-obvious result. If people wander off through the verbiage of any article, where the chatter isn't supported by data, sure, they'll tend to get speculation.

I don't really understand what you're saying. Are you the saying the charts don't actually make it clear, or that they make it clear that self assessment is independent but not necessarily uniform?

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#258

Post by [deleted] »

jameshart wrote: post #69
marcholagao wrote: post #48

tldr; D+K's experiment was: Assign the numbers 1 thru 10 to ten people. Have each role a 10 sided die. The person assigned a 1 will roll higher than his assigned number 90% of the time. Daniel: >It’s not a “statistical artifact” - that will be your everyday experience living in such a world. You can experience statistical effects. I think a lot of controversy comes from how Dunning and Kruger's paper leads people to…

This is such a bizarre argument. Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance! If you accept the 'D-K is autocorrelation' argument, you don't get to throw out the existence of the D-K effect: you are saying Dunning + Krueger failed to show that humans have any ability to estimate how skilled they are at all . That seems like an even more radical position…

[deleted]

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#259

Post by dahart »

hef19898 wrote: post #119
dahart wrote: post #76

Earlier quoted context omitted.

> I think Dunning Krueger makes intuitive sense. It’s been argued before that this is the only reason that DK gained any notoriety; because it feels right, not because it is right. It’s the “just-world” theory: we want to believe that confident people are overcompensating. Is it actually intuitive though? Consider your own example. Most people who don’t know piano or figure skating are well aware that they don’t know…

>> figure skating, or law Thinking about all the "INAL" answers and the obviously wrong "legal" advice and opinion on legal topics you can come across on HN on a daily basis, I think law is good example of people overestimating their knowledge. Replace figure skating with any other sports, so, and try having discussions about, e.g, a defeat of any soccer team. All of a sudden everyone just became a soccer coach. And…

You might be forgetting that DK demonstrates a positive correlation between confidence and skill. They gave statistical evidence for people who believe they’re right actually being right more often on average. The question the paper is actually asking is why aren’t people’s self-estimates perfect, but it does not, contrary to popular misunderstanding, demonstrate that confident people are lower skilled. Reading bad legal advice on HN is not a demonstration of the so-called DK effect.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#260

Post by dahart »

sn41 wrote: post #106
dahart wrote: post #76

Earlier quoted context omitted.

> I think Dunning Krueger makes intuitive sense. It’s been argued before that this is the only reason that DK gained any notoriety; because it feels right, not because it is right. It’s the “just-world” theory: we want to believe that confident people are overcompensating. Is it actually intuitive though? Consider your own example. Most people who don’t know piano or figure skating are well aware that they don’t know…

> It’s been argued before that this is the only reason that DK gained any notoriety I'm sorry, but I have to comment on a word in this line: I feel that increasingly, "notoriety" is used when "notability" might be better. A thief is notorious. A statistical effect is notable, imho. But I agree with your basic point: I can't skate, much less figure skate. And I am pretty accurately aware of that fact despite my lack o…

Notoriety means “the state of being famous”, which is the meaning that I intended. I actually don’t want to use “notable” in this case, because that would imply that I believe the DK effect is real. Notorious might even be a good word here, since the paper has problems with its claims and its interpretation of it’s own data.
Post reply on HN