Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Knowledge overconfidence is associated with anti-consensus views

science.org

Post reply on HN
Print view View original

101–110 of 132 posts

Re: Knowledge overconfidence is associated with anti-consensus views

#101

Post by mlyle »

somenameforme wrote: post #100
mlyle wrote: post #92

Earlier quoted context omitted.

> . For instance, among their "objective scientific knowledge" questions asked whether, "The novel coronavirus was unleashed in a laboratory in Wuhan and spread from there." Another was "The numbers of people that have died from COVID-19 are artificially inflated." This is an incorrect reading of the paper. Those are statements intended to determine whether a participant has anti-consensus beliefs, but do not occur o…

No, you're the one that's suffered an incorrect reading here. They created different objective knowledge questions for each of the different studies (there were 5) within this study. I am referencing the COVID "objective knowledge" quiz. "Subjective knowledge" for each study was determined by a single question where participants were asked to rate their knowledge of the various topics on a scale of 1-7. The questions…

I now see the question you mentioned from study 5. IMO study 5 is not the interesting substudy. As you mention, you cherry-picked the extreme example.

> But, perhaps most importantly, the observed effect from this study was extremely small.

The substudies that looked at those set of 34 questions above had an extremely significant result-- both statistical significance and magnitude of effect. I also don't see anything too controversial in that set of 34 questions. Do you?

Re: Knowledge overconfidence is associated with anti-consensus views

#102

Post by pbak »

honkkong420 wrote: post #99
joshuamorton wrote: post #89

Earlier quoted context omitted.

Sure but "my personal experiences match up with the expert consensus from the past decade" (and really longer, the 2005 food pyramid was for slightly more grains than veggies, not double) isn't a particularly strong argument for disagreeing with the expert consensus. If what you're saying is "I disagree with expert consensus from the 80s", well then sure, but so does the expert consensus! That's how the scientific pr…

My argument is that my prior experiences seem to work better for guiding my meal choice than all the formal knowledge presented to me.

Which is ironic considering what the discussion is about: you conducted your own scientific experiment on yourself, disproved the consensus-accepted rule, and came to new conclusions.

To achieve that, you needed to have enough confidence in yourself and your ability to measure and feel stuff (which some people think is bad, apparently) versus some consensus which purports to know better.

Courageous new science regularly blasts old knowledge and education to bits, and that's a good things.

Re: Knowledge overconfidence is associated with anti-consensus views

#103

Post by somenameforme »

mlyle wrote: post #101
somenameforme wrote: post #100

Earlier quoted context omitted.

No, you're the one that's suffered an incorrect reading here. They created different objective knowledge questions for each of the different studies (there were 5) within this study. I am referencing the COVID "objective knowledge" quiz. "Subjective knowledge" for each study was determined by a single question where participants were asked to rate their knowledge of the various topics on a scale of 1-7. The questions…

I now see the question you mentioned from study 5. IMO study 5 is not the interesting substudy. As you mention, you cherry-picked the extreme example. > But, perhaps most importantly, the observed effect from this study was extremely small. The substudies that looked at those set of 34 questions above had an extremely significant result-- both statistical significance and magnitude of effect. I also don't see anythin…

Yes. It's a complete non sequitur to ask somebody to self rate their knowledge on e.g. climate change, and then try to objectively measure that knowledge by asking e.g. what the largest organ in the human body is. It doesn't even make any sense. It's like me asking to self estimate your knowledge of javascript, and then giving you a quiz on C# to measure that objective knowledge.

The study is also filled with various issues. For instance they also removed everybody who completely agreed with the scientific consensus on studies 1-4 (see buried in page 44 on the materials and methods supplement) who undoubtedly also rated their objective knowledge highly. They drew participants from outlets like Amazon Mechanical Turk where people are paid peanuts to carry out menial tasks ($0.85 for this study). That's obviously not going to be a representative sample of society, let alone America. The paper claimed to have preregistered their study, yet didn't provide a link to such - which completely negates the point of preregistering, and so on.

Something I find quite troubling is how increasingly common studies of this sort are. This study is mostly junk, but it's enticing click bait junk that will also undoubtedly bump up this journal's impact fact as other's unquestioningly cite it more with the goal of generating interest even if in lieu of science. What exactly were the peer reviewers/editors doing? It seems that so long as one concludes the right thing in exciting enough fashion, what editorial process there is becomes quite deferent, a la Sokal.

Re: Knowledge overconfidence is associated with anti-consensus views

#104

Post by mlyle »

somenameforme wrote: post #103
mlyle wrote: post #101

Earlier quoted context omitted.

I now see the question you mentioned from study 5. IMO study 5 is not the interesting substudy. As you mention, you cherry-picked the extreme example. > But, perhaps most importantly, the observed effect from this study was extremely small. The substudies that looked at those set of 34 questions above had an extremely significant result-- both statistical significance and magnitude of effect. I also don't see anythin…

Yes. It's a complete non sequitur to ask somebody to self rate their knowledge on e.g. climate change, and then try to objectively measure that knowledge by asking e.g. what the largest organ in the human body is. It doesn't even make any sense. It's like me asking to self estimate your knowledge of javascript, and then giving you a quiz on C# to measure that objective knowledge. The study is also filled with various…

> by asking e.g. what the largest organ in the human body is.

Which is part of why they had subscores for related fields. Obviously we can't quiz about the actual controversy, because we disgaree on the facts.

> It doesn't even make any sense.

I think it's interesting that people who disagree with these consensus have, on average, lower overall literacy in the sciences-- both adjacent sciences and other science, but believe they have above average literacy and will outperform the average on tests.

Re: Knowledge overconfidence is associated with anti-consensus views

#105

Post by joshuamorton »

honkkong420 wrote: post #99
joshuamorton wrote: post #89

Earlier quoted context omitted.

Sure but "my personal experiences match up with the expert consensus from the past decade" (and really longer, the 2005 food pyramid was for slightly more grains than veggies, not double) isn't a particularly strong argument for disagreeing with the expert consensus. If what you're saying is "I disagree with expert consensus from the 80s", well then sure, but so does the expert consensus! That's how the scientific pr…

My argument is that my prior experiences seem to work better for guiding my meal choice than all the formal knowledge presented to me.

And my point is that your failed to do even a trivial amount of research into what consensus is.

If "knowledge" was only what you were presented in primary school, the world would be a very different place.

Re: Knowledge overconfidence is associated with anti-consensus views

#106

Post by IceMetalPunk »

Insert "not that shocked" Fry meme here

There are two axes here, each with two options: Axis 1 = "I understand" or "I don't understand". Axis 2 = "I think I understand" or "I think I don't understand". That leaves 4 options. (Obviously, these can be thought of as two continuous axes as well, and I'll address that later.)

Option 1: "I understand and I think I understand" -- good, you understand, you'll probably come to the same conclusion of the experts who also understand, since you're working with the same data they are.

Option 2: "I understand, but I'm not sure how well" -- good, you understand, see option #1 with the added benefit that you'll look for more evidence to increase your confidence.

Option 3: "I don't understand, and I don't think I do" -- good, so you at least won't form opinions because you know you aren't informed, and maybe even will be motivated to look for evidence to move yourself into categories 1 or 2.

Option 4: "I don't understand, but I think I do" -- i.e. overconfidence. Your conclusions will likely be wrong, since you don't understand the data you're basing those conclusions on, but since you think you do, you will stick to those conclusions in the face of contradictory evidence or expert advice.

So, yes, I would fully expect the result this study found, from a purely logical analysis.

Now, as I said, both confidence and understanding can be thought of as spectra rather than discretized categories. In that case, you get a continuous transition between those resultant options, but generally you still get three discrete classifications in that space: "I act like I know but am wrong", "I refrain from forming an opinion", and "I act like I know and am correct."

Re: Knowledge overconfidence is associated with anti-consensus views

#107

Post by somenameforme »

mlyle wrote: post #104
somenameforme wrote: post #103

Earlier quoted context omitted.

Yes. It's a complete non sequitur to ask somebody to self rate their knowledge on e.g. climate change, and then try to objectively measure that knowledge by asking e.g. what the largest organ in the human body is. It doesn't even make any sense. It's like me asking to self estimate your knowledge of javascript, and then giving you a quiz on C# to measure that objective knowledge. The study is also filled with various…

> by asking e.g. what the largest organ in the human body is. Which is part of why they had subscores for related fields. Obviously we can't quiz about the actual controversy, because we disgaree on the facts. > It doesn't even make any sense. I think it's interesting that people who disagree with these consensus have, on average, lower overall literacy in the sciences-- both adjacent sciences and other science, but…

Ignoring your attempt at predictably headed ad hominem, you again cannot make your 'conclusion' from the study. They used a non-representative sample, removed the results of those who fully agreed with the consensus, and just generally did everything trying to massage their numbers into the conclusion they wanted to make.

Ironically, the numbers don't even support that if you do use the subscales. In all studies the overall effect from opposition was a fraction of a single point difference. In most studies opposition to the consensus in climate change was also predictive of a higher than average level of field-specific knowledge.

Re: Knowledge overconfidence is associated with anti-consensus views

#108

Post by denton-scratch »

mlyle wrote: post #63
whatshisface wrote: post #39

Earlier quoted context omitted.

That is another phrasing issue, because it would be reasonable to take the question as a practical one meant to refer to the things that actually exist, but it would also be reasonable to take it as a theoretical one like you're suggesting.

Well, most of the criticism don't apply to, e.g. golden rice, which could improve the food security and nutrition for a billion people, including preventing severe deficiencies that cause blindness. > associated with strain patenting strain patenting isn't really a GM-vs-non-GM issue. As to golden rice-- sure, it's patented, but it's also licensed in a way that most of the people cultivating it wouldn't pay any licen…

> Golden rice

I was careful to hedge myself, with phrases like "associated with" and "sold in conjunction with". I know there are exceptions.

I have no problem with golden rice, except if it's grown adjacent to some organic farmer's plot - that would potentially destroy his business. In fact I'd eat golden rice - but I've never seen it on sale.

Re: Knowledge overconfidence is associated with anti-consensus views

#109

Post by cudgy »

So, do we blindly ignore common sense and instinct, handing it over to supposed experts that have been wrong repeatedly about very fundamental concepts like diet and education? So many accepted practices promoted by experts have been and are just wrong. New information is gathered and past certainty becomes present doubt.

Of course, blindly ignoring information because it is outside of your understanding is equally incorrect. There is no easy solution to this issue, but generous skepticism is wise, especially when someone purports to be expert and certain about something.

Re: Knowledge overconfidence is associated with anti-consensus views

#110

Post by mlyle »

somenameforme wrote: post #107
mlyle wrote: post #104

Earlier quoted context omitted.

> by asking e.g. what the largest organ in the human body is. Which is part of why they had subscores for related fields. Obviously we can't quiz about the actual controversy, because we disgaree on the facts. > It doesn't even make any sense. I think it's interesting that people who disagree with these consensus have, on average, lower overall literacy in the sciences-- both adjacent sciences and other science, but…

Ignoring your attempt at predictably headed ad hominem, you again cannot make your 'conclusion' from the study. They used a non-representative sample, removed the results of those who fully agreed with the consensus, and just generally did everything trying to massage their numbers into the conclusion they wanted to make. Ironically, the numbers don't even support that if you do use the subscales. In all studies the…

> Ignoring your attempt at predictably headed ad hominem

???? ad hominem? Where? I'm saying it's very difficult to quiz about the actual controversy itself, because then we get very close to the point of disagreement and risk confounding. The research explicitly mentions this aspect of design.

> They used a non-representative sample, removed the results of those who fully agreed with the consensus, and just generally did everything trying to massage their numbers into the conclusion they wanted to make.

I don't love use of Mechanical Turk for social sciences. It's still an interesting finding. Of course, more and higher quality research should be used to confirm the effect and gain additional nuance.

> In all studies the overall effect from opposition was a fraction of a single point difference.

The overall effect from opposition was a fraction of a single point difference per unit of opposition.

Post reply on HN