Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

131–140 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#131

Post by sally_glance »

hef19898 wrote: post #119
dahart wrote: post #76

Earlier quoted context omitted.

> I think Dunning Krueger makes intuitive sense. It’s been argued before that this is the only reason that DK gained any notoriety; because it feels right, not because it is right. It’s the “just-world” theory: we want to believe that confident people are overcompensating. Is it actually intuitive though? Consider your own example. Most people who don’t know piano or figure skating are well aware that they don’t know…

>> figure skating, or law Thinking about all the "INAL" answers and the obviously wrong "legal" advice and opinion on legal topics you can come across on HN on a daily basis, I think law is good example of people overestimating their knowledge. Replace figure skating with any other sports, so, and try having discussions about, e.g, a defeat of any soccer team. All of a sudden everyone just became a soccer coach. And…

But is DK not explicitly about assessing yourself and not about assessing others? I feel like being able to critique the performance of others is different from being able to critique yourself. There might be a correlation between the two, but they're not the same.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#132

Post by nosefrog »

civilized wrote: post #52
nosefrog wrote: post #37

I think the main point of this post is correct -- just because you can find the effect in random noise, doesn't mean it's not real phenomenon that happens in real life. But it's missing a nuance there: if an effect can be replicated with random noise, then it's not a psychological effect (e.g. something that you would explain as a human bias), but a statistical effect. E.g. regression towards the mean is a real effec…

> if an effect can be replicated with random noise, then it's not a psychological effect This isn't true either. Statistical dependence does not determine or uniquely identify causal interpretation or system structure. See Judea Pearl's works (e.g. The Book of Why) for more on this. People lacking the ability to self-assess is interesting psychologically. People can learn from experience in many other contexts. Peopl…

Sure, it's an interesting question, but if the way you're measuring it can't be distinguished from noise, you need to find a different measure.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#133

Post by [deleted] »

coffeeblack wrote: post #128
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

That happens when science is politicized, and any scientists critical of the “official” results is destroyed. From climate to Covid, so many areas where that happens.

[deleted]

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#134

Post by gambler »

quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

> The anti-intellectual, anti-scientific streak in many poor analyses claiming to debunk some scientific research is deeply concerning in our society.

People endlessly reference the Dunning-Kruger effect as a meme, without ever having read the paper, let alone having checked its methods. You don't seem to have a problem with that.

On the other hand, after seeing an article that uses essentially statistical arguments to debate a scientific study you conclude that there is some "anti-intellectual, anti-scientific streak" in our society and that it should be of grave concern.

This doesn't make any sense except as an extreme case of virtue-signaling.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#135

Post by lelanthran »

russdill wrote: post #90
darawk wrote: post #86

Earlier quoted context omitted.

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

I think there's two extremes here. One is the issue covered well above. There is a great deal of junk science that gets published. That is a problem and it does erode trust. But in some ways it's also how the sausage gets made, there's going to be room for things to get published they later gets refuted. People rightly so have distrust for results coming out in fields they don't have good knowledge in. Without becomi…

> I think there's two extremes here. One is the issue covered well above. There is a great deal of junk science that gets published.

It's more than that, I think. Sibling-thread poster hit the nail on the head when he complained of politicised science.

The social sciences have this dominating and silencing effect on the rest of the sciences.

There's always been junk science, and when found out it gets discredited. This is still happening and is a good thing.

What's new is that any research that might produce results counter to the what the PC-mob deems acceptable is attacked. Whether or not there is consensus amongst researchers in that field is irrelevant when the mob calls for the firing of any researcher who doesn't toe the current political party-line.

Sure, we're not actually in the dark ages, but a trend of silencing voices in the name of purity of thought is particularly troubling, especially as the mob asking for this is unashamedly attempting to implement NewsSpeak[1].

[1] See the argument in yesterdays threads about what "man" and "woman" mean, and should dictionaries be changed, etc.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#136

Post by pavlov »

coffeeblack wrote: post #128
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

That happens when science is politicized, and any scientists critical of the “official” results is destroyed. From climate to Covid, so many areas where that happens.

If by “destroyed” you mean “billionaires will happily fund their contrarian research for years regardless of its peer reviews”…

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#137

Post by slavik81 »

jameshart wrote: post #69
marcholagao wrote: post #48

tldr; D+K's experiment was: Assign the numbers 1 thru 10 to ten people. Have each role a 10 sided die. The person assigned a 1 will roll higher than his assigned number 90% of the time. Daniel: >It’s not a “statistical artifact” - that will be your everyday experience living in such a world. You can experience statistical effects. I think a lot of controversy comes from how Dunning and Kruger's paper leads people to…

This is such a bizarre argument. Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance! If you accept the 'D-K is autocorrelation' argument, you don't get to throw out the existence of the D-K effect: you are saying Dunning + Krueger failed to show that humans have any ability to estimate how skilled they are at all . That seems like an even more radical position…

> Dunning Krueger is precisely about the surprising result that people are bad at estimating their performance

Isn't DK about estimating your performance relative to the rest of the population? To do that, you need to not only know your own performance but also everyone else's. To me, guessing the performance of others sounds quite difficult.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#138

Post by audiometry »

dahart wrote: post #76
galaxyLogic wrote: post #3

I think Dunning Krueger makes intuitive sense. When you become skilled in your field you learn from other people in your field, and your assessment of yourself is based on your relation to the skills of those other people. But if you know very little about something, you have no reference point to evaluate yourself against. When you learn something you also learn what are some of the mistakes you can make. You evalua…

> I think Dunning Krueger makes intuitive sense. It’s been argued before that this is the only reason that DK gained any notoriety; because it feels right, not because it is right. It’s the “just-world” theory: we want to believe that confident people are overcompensating. Is it actually intuitive though? Consider your own example. Most people who don’t know piano or figure skating are well aware that they don’t know…

"Most people who don’t know piano or figure skating are well aware that they don’t know, and do not rate themselves highly at all. "

I think this case (real_skill = 0, perceived_skill = 0) is maybe a trivial case, and that the bit of truth DK-idea catches is when someone with very little skill considers how much work it would be to get to whatever a 'fully skilled' version would be, they woefully underestimate.

Picture someone in their first summer of mountain biking watching youtube videos of the best guys. Yes, you know you can't jump like they do, or turn as skillfully, but you're getting better each month. However, you still grossly underestimate how hard it is to get to that skill level.

At least it's my personal experience as a thoroughly unskilled!

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#139

Post by molf »

c1ccccc1 wrote: post #116
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

It still seems to me like "The DK Effect is Autocorrelation" is basically correct. The important thing isn't whether or not independence should be the null hypothesis, because calling something a "null hypothesis" is just an arbitrary label that doesn't affect reality. The important thing is that what we can actually conclude from the Dunning-Kruger paper is a lot less than popular presentations of the concept claim.…

> calling something a "null hypothesis" is just an arbitrary label that doesn't affect reality

It does affect your conclusions though.

The choice of null hypothesis in "The DK Effect is Autocorrelation" determined how the random data was generated. The hypothesis is: "nobody has any clue whatsoever how competent they are". The random data was specifically crafted for that hypothesis.

The choice of null hypothesis in this article is: "everyone roughly knows how competent they are". This random data, too, is specifically crafted for the null hypothesis.

So what does this mean? If you pick the a particular null hypothesis then you can try to argue that the DK is a statistical artefact. But it's not, it is an artefact of choosing a particular null hypothesis.

There are valid criticisms of the DK study, though. See this comment for example: https://news.ycombinator.com/item?id=31119196

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#140

Post by Lutger »

russdill wrote: post #90
darawk wrote: post #86

Earlier quoted context omitted.

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

I think there's two extremes here. One is the issue covered well above. There is a great deal of junk science that gets published. That is a problem and it does erode trust. But in some ways it's also how the sausage gets made, there's going to be room for things to get published they later gets refuted. People rightly so have distrust for results coming out in fields they don't have good knowledge in. Without becomi…

Exactly this. It's even worse, distrust of broad scientific consensus is purposefully cultivated to further political and economic goals, and the methods to do so increasingly perfected. This damages our ability as societies to function in a healthy way. Our capacity to navigate hard problems is diminished by the ever decreasing influence of hard science on politics and policy.
Post reply on HN