Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI advice made people less accurate but more confident – sudy

thenextweb.com

Post reply on HN
Print view View original

181–190 of 234 posts

Re: AI advice made people less accurate but more confident – sudy

#181

Post by rsoto2 »

RA_Fisher wrote: post #148
encomiast wrote: post #76

Earlier quoted context omitted.

Agreed, the headline says "AI advice made people three times less accurate". But if we really want to know how accurate these people were, we need to know how accurate the AI system they use is. If the AI system is hobbled to a point where it is worse than they reasonably expect we can't blame the people or the AI system. This would be the same as claiming the listening to experts make people less accurate in a study…

Exactly, it’s unrepresentative of AI. It’s damaged AI.

All AI is flawed and prone to "hallucinations"(doing exactly what it was designed to do) that's why Microsoft considers it an "Entertainment" product.

Re: AI advice made people less accurate but more confident – sudy

#182

Post by rsoto2 »

encomiast wrote: post #134
habinero wrote: post #111

Earlier quoted context omitted.

If you ask that, you fundamentally misunderstand the point. It's not about the LLM, it's about whether people will critically evaluate what it spits out.

Here is what the study says: "The LLM used in our experiments (Step 3.5 Flash) answered such questions incorrectly almost without exception. We also checked some state-of-the-art LLMs (GPT-5.5, Claude 4.6 Sonnet, Gemini 3.5 Flash); they all failed on the hardest question (Monica’s vehicle), while being frequently correct on the other questions." So, if people's experience is with modern LLMs, they are being rational…

No, the rationality of humans is not defined by how gullible they are towards LLMs. Are yall getting your psychology degree from ChatGPT university, my god.

Re: AI advice made people less accurate but more confident – sudy

#183

Post by rsoto2 »

reticulates wrote: post #9

Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…

I mean that's this entire thread in a nutshell. Armchair experts emboldened by their $100 ChatGPT University Diploma.

Re: AI advice made people less accurate but more confident – sudy

#184

Post by rsoto2 »

imoverclocked wrote: post #62

I saw a post that some teachers are now asking students to ask ChatGPT to do their assignments… and then critique it where it is wrong. I feel like this might be an interesting method for places where the default response tends to be someone submitting copy-pasta: just give the ai-response and then have community discussion around what it missed.

nice try clammy sammy.

We're gonna continue to push for scientific based approaches to education not some lunatic's idea of an educational experiment that waste's children's education. We tried this with Bill Gates already.

Re: AI advice made people less accurate but more confident – sudy

#185

Post by [deleted] »

annzabelle wrote: post #127
reticulates wrote: post #9

Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…

Reddit has always had the dynamic of people misrepresenting their knowledge and just repeating what other people said on that subreddit. I find myself doing that when I spend too long on there - posting very authoritatively on something I'm honestly clueless about. One big pet peeve of mine on reddit is on any kind of immigration-related subreddit it is absolutely filled with people talking about how it's impossible…

[deleted]

Re: AI advice made people less accurate but more confident – sudy

#187

Post by michaelmrose »

encomiast wrote: post #134
habinero wrote: post #111

Earlier quoted context omitted.

If you ask that, you fundamentally misunderstand the point. It's not about the LLM, it's about whether people will critically evaluate what it spits out.

Here is what the study says: "The LLM used in our experiments (Step 3.5 Flash) answered such questions incorrectly almost without exception. We also checked some state-of-the-art LLMs (GPT-5.5, Claude 4.6 Sonnet, Gemini 3.5 Flash); they all failed on the hardest question (Monica’s vehicle), while being frequently correct on the other questions." So, if people's experience is with modern LLMs, they are being rational…

It still proves something much narrower. Wherein AI is insufficient to answer, people are apt to rely on it anyway.

A good real-world issue is health, where the issues are very complicated with many things poorly defined even at the state of the art where practitioners are relying on personal judgement and lots of data but patients are apt to feed ai very little data compared to what their doctor has.

Real frontier models can remain confidently incorrect in these cases.

Re: AI advice made people less accurate but more confident – sudy

#188

Post by michaelmrose »

encomiast wrote: post #170
what wrote: post #164

Earlier quoted context omitted.

> So, if people's experience is with modern LLMs, they are being rational to accept that the answers as likely correct. They are not. But also wtf is a “modern” LLM? This is totally unhinged, every complaint about an LLM is always responded to with “you’re just using one from two months ago, it’s totally different now”. Repeat every two months for the same complaints.

So then you need to ask: Why did they use a deliberately faulty LLM? They could have easily used a mainstream LLM from the past 18 months and it probably would have been less work to do so. But then they would not have that headline. The answers from the LLM would have likely made the participant's answers more accurate, not 3x less accurate. But then they would not have this juicy headline. I understand that many of…

The actual study is more circumspect than this click bait and examines pretty deliberately how people respond to inaccurate data. It's neither a trick nor a design flaw. It's literally the thrust of the study.

Re: AI advice made people less accurate but more confident – sudy

#189

Post by vineyardmike »

tsimionescu wrote: post #47
dwohnitmok wrote: post #31

Earlier quoted context omitted.

People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two. This is why I think it's a bad study. It didn't measure anything of the essential differences of how people…

> People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). What open book quizzes allow you to leave all answers blank with no penalty? An open book quiz is very different from the experimental setup tested here.

Many tests penalize incorrect answers worse than blank answers.

As a famous example, you were incentivized to leave questions blank on the American SATs (until somewhat recently).

Re: AI advice made people less accurate but more confident – sudy

#190

Post by bluefirebrand »

habinero wrote: post #125
vanuatu wrote: post #44

Earlier quoted context omitted.

+1 I'd wager you get similar results if you gave people a version of Google search that purposely gave you bad results. Like, it's framed as an assistant / lookup tool - is it so surprising that people tend to trust it more? Especially since the participants are likely used to using full-powered models and the researchers give them a purposely gimped one (lol) People are acting rationally when given AI tools to looku…

> is it so surprising that people tend to trust it more Yes and no. I think most people would agree that Wikipedia is, on the whole, a pretty great first resource on anything. It tries to be factual and accurate. Most people would also agree that Wikipedia can be wrong or manipulated and should never be used for an authoritative source. And then somehow a computer barfing up words distilled from magic internet concen…

Many middle aged people might also remember a time when every adult in the world was shouting "don't believe everything you read online" and schools wouldn't let you cite wikipedia as a primary source.

That attitude seems to have gone away for some reason

Post reply on HN