Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

andersource.dev

Post reply on HN
Print view View original

141–150 of 308 posts

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#141

Post by lelanthran »

boppo1 wrote: post #72
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

What about the replication crisis? It's possible to use rigorously sound statistics to lie (or at least unknowingly spread falsehoods). I can't tell you how many times I've seen headlines or abstracts of studies that seem to contradict ones I've seen previously, and back and forth! Particularly in the social sciences. I recall one study that said all white people are committing environmental racism against all non-wh…

> It's possible to use rigorously sound statistics to lie (or at least unknowingly spread falsehoods)

The book "How to lie with statistics" is one of the best statistics textbooks that I have read. It basically makes you immune to misleading stats (charts, tables, everything).

IIRC, the only thing that is missing from the book (it's a really old book) which is very relevant is p-hacking.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#142

Post by kortilla »

whatshisface wrote: post #101
boppo1 wrote: post #72

Earlier quoted context omitted.

What about the replication crisis? It's possible to use rigorously sound statistics to lie (or at least unknowingly spread falsehoods). I can't tell you how many times I've seen headlines or abstracts of studies that seem to contradict ones I've seen previously, and back and forth! Particularly in the social sciences. I recall one study that said all white people are committing environmental racism against all non-wh…

> it's possible to use rigorously sound statistics to lie (or at least unknowingly spread falsehoods). I don't think this is true. It is possible to put a lot of work into unsound statistics and to make a lot of "noise and fury" about how mathematical you are while failing some basic principle, but I don't think sound statistics can mislead. The replication crisis was caused by scientists not being rigorous and journ…

> but I don't think sound statistics can mislead.

Of course they can, unless you magically exclude all statistics that made a bad assumption on independence.

I plot all the daily high temperatures and the presence of the ice cream cart and it turns out the ice cream cart causes warmer highs! Solid statistics.

Turns out the guy that has the ice cream cart has a weather app on his phone though and doesn’t come out on forecasted cold days.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#143

Post by seniortaco »

Beyond the validity of the statistical methods used.. can someone clarify what is the actual hypothesis we are debating about competence? And what does each article propose?

My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves".

DK says: True

DK is Autocorrelation says: ???

"I cant let go..." says: True?

HN says: also True?

Is there really any debate here? The "DK is Autocorrelation" article seems to be the only odd one out, and it's not clear if it even makes a proposal either way about the DK hypothesis. It talks about the Nuhfer study, but that seems Apples vs Oranges since it buckets by education level. Then it also points out that random noise would also yield the DK effect. But that also does not address the DK hypothesis, and it would indeed be very surprising if people's self evaluation was random!

So should my takeaway here just be that the DK hypothesis is True and that this is all arguing over details?

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#144

Post by avereveard »

darawk wrote: post #86
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

Let's not forget though that a great deal of "science" is in fact trash[1]. The problem isn't really people being anti-science or pro-science. The problem is science being done poorly, whether by scientists in the credentialed sense, or amateurs. There is no pat "trust science more" or "trust amateurs less" answer here. The actual answer is that if you want to understand research, you need to actually understand math…

part of the issue is that producing shit research cost very little, and debunking shit research cost a lot. just pick up one of the many "x reasons why earth is not a sphere" videos, some are pretty easy to debunk, but other require to understand i.e. potential fields (if earth is round why don't train engineer take into account curvature when laying tracks and variations thereof)

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#145

Post by vetleen »

seniortaco wrote: post #143

Beyond the validity of the statistical methods used.. can someone clarify what is the actual hypothesis we are debating about competence? And what does each article propose? My understanding is that the hypothesis is "Those who are incompetent overestimate themselves, and experts underestimate themselves". DK says: True DK is Autocorrelation says: ??? "I cant let go..." says: True? HN says: also True? Is there really…

DK says: True

DK is Autocorrelation says: The DK article is based on a false premise, we got to disregard it

"I cant let go..." says: Actually, given that we assume people are somewhat capable of self-assessment, which is reasonable, "DK is Autocorrelation" is the one based on a false premise, and we should disregard that one instead, and not DK.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#146

Post by andersource »

omnicognate wrote: post #126

Gah, I wish I had time to fully read this and get into it, but I have to spend the next few hours driving. Unfortunately the original article isn't very clearly explained, and it's only on reading the discussion in the comments under it that it becomes clear what it's actually saying. The point is about signal & noise. Say your random variable X contains a signal component and a noise component, the former determinis…

Hey omnicognate, good to see you here, appreciated our previous discussion.

What you're saying is that we need to verify the statistical reliability of the skill tests DK gave, and to some extent that we need to scrutinize the assumption that there indeed is such a thing as "skill" to be measured in the first place. I hope we can both agree that skill exists. That leaves the test reliability (technical term from statistics, not in the broad sense).

What's simulated by purely random numbers is tests with no reliability whatsoever. Of course if the tests DK gave to subjects don't actually measure anything at all, the DK study is meaningless. If that's what the original article's author is trying to say, they sure do it in a very roundabout way, not mentioning the test reliability at all. I'd be completely fine reading an article examining the reliability of the tests. Otherwise, I again fail to see how the random number analysis has anything to do with the conclusions of DK.

In fact, DK do concern themselves with the test reliability, at least to some extent. That doesn't appear in the graph under scrutiny but appears in the study.

If you assume the tests are reliable, and you also assume that DK are wrong in that people's self-assessment is highly correlated with their performance, and generate random data accordingly, you'll still get no effect even if you sample twice as you propose.

> The key point though is that if you use a separate sample of X that correlation disappears completely

Separate sample of X under the assumption of no dependence at all of the first sample, i.e., assuming there is no such a thing as skill, or assuming completely unreliable tests. So, not interesting assumptions, unless you want to call into question the test reliability, which neither you nor the author are directly doing.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#147

Post by hef19898 »

sally_glance wrote: post #131
hef19898 wrote: post #119

Earlier quoted context omitted.

>> figure skating, or law Thinking about all the "INAL" answers and the obviously wrong "legal" advice and opinion on legal topics you can come across on HN on a daily basis, I think law is good example of people overestimating their knowledge. Replace figure skating with any other sports, so, and try having discussions about, e.g, a defeat of any soccer team. All of a sudden everyone just became a soccer coach. And…

But is DK not explicitly about assessing yourself and not about assessing others? I feel like being able to critique the performance of others is different from being able to critique yourself. There might be a correlation between the two, but they're not the same.

Thinking that you are better than the actual pros is the easiest to spot indicator of lacking self assessment, so.

Re: I can't let go of “The Dunning-Kruger Effect is Autocorrelation”

#149

Post by qfwfq_ »

boppo1 wrote: post #72
quanto wrote: post #62

The conclusion in the article: > Why so angry? I know I’ve taken this far too personally. I have no illusions that everything I read online should be correct, or about people’s susceptibility to a strong rhetoric cleverly bashing conventional science, even in great communities such as HN. But frankly, for the last few years, the world seems to be accelerating the rate at which it’s going crazy, and it feels to me a l…

What about the replication crisis? It's possible to use rigorously sound statistics to lie (or at least unknowingly spread falsehoods). I can't tell you how many times I've seen headlines or abstracts of studies that seem to contradict ones I've seen previously, and back and forth! Particularly in the social sciences. I recall one study that said all white people are committing environmental racism against all non-wh…

Given the explosion in the number of journals and the impossibility of effective peer review, being published in a journal does not mean what it used to. This is part of the material drivers for the replication crisis (journals can no longer effectively gatekeep scientific validity), but it also reflects something real about the practice of science: little social cliques come up with pet theories and, over time, "fight" with these theories on epistemic common ground. The successful ones, we'd like to think, are the ones that last the most rounds in the fight, but that probably only holds in the long run. Contradiction, in itself, is normal (and was before!)
Post reply on HN