Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI advice made people less accurate but more confident – sudy

thenextweb.com

Post reply on HN
Print view View original

201–210 of 234 posts

Re: AI advice made people less accurate but more confident – sudy

#201

Post by TimByte »

michaelmrose wrote: post #187
encomiast wrote: post #134

Earlier quoted context omitted.

Here is what the study says: "The LLM used in our experiments (Step 3.5 Flash) answered such questions incorrectly almost without exception. We also checked some state-of-the-art LLMs (GPT-5.5, Claude 4.6 Sonnet, Gemini 3.5 Flash); they all failed on the hardest question (Monica’s vehicle), while being frequently correct on the other questions." So, if people's experience is with modern LLMs, they are being rational…

It still proves something much narrower. Wherein AI is insufficient to answer, people are apt to rely on it anyway. A good real-world issue is health, where the issues are very complicated with many things poorly defined even at the state of the art where practitioners are relying on personal judgement and lots of data but patients are apt to feed ai very little data compared to what their doctor has. Real frontier m…

It would be interesting to measure not just the accuracy, but how many people actually decided to double check the answer

Re: AI advice made people less accurate but more confident – sudy

#203

Post by watwut »

embedding-shape wrote: post #81
nkrisc wrote: post #24

Earlier quoted context omitted.

You could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that, while out in the world people do use LLMs like that all the time.

> You could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that Feels like this differs wildly depending on who you consider "people" to be. The average person on the street? Definitely just parrots stuff they've read somewhere, not even a "textbook". A group of software developers used to parsing semi-true information? Probably they'd get it ri…

> A group of software developers used to parsing semi-true information?

They are the first to parrot what was spewed from llm and previous even what was found on 4chan.

Re: AI advice made people less accurate but more confident – sudy

#204

Post by fxwin »

tsimionescu wrote: post #58
fxwin wrote: post #15

Earlier quoted context omitted.

You can read the study here: https://osf.io/preprints/psyarxiv/5y6m4_v1 TLDR: They studied both cases (Access to a LLM/Chat interface which gives a wrong answer when asked, and access to pregenerated (wrong) answers). Both experiments yielded similar results.

That is not a good summary. In all their versions of the experiment, they presented the tool as AI, and used actual LLM answers. In the first example, participants were directly interacting with a real, though small, LLM, but they had technical issues because of that - 10% of the time the LLM setup failed to present an answer at all. So they redid the experiment with pre-generated answers - the participants saw the s…

im not sure which part of my summary you take issue with then? the reason for study 1b is not relevant to answer the question asked in the comment i replied to

Re: AI advice made people less accurate but more confident – sudy

#205

Post by tsimionescu »

fxwin wrote: post #204
tsimionescu wrote: post #58

Earlier quoted context omitted.

That is not a good summary. In all their versions of the experiment, they presented the tool as AI, and used actual LLM answers. In the first example, participants were directly interacting with a real, though small, LLM, but they had technical issues because of that - 10% of the time the LLM setup failed to present an answer at all. So they redid the experiment with pre-generated answers - the participants saw the s…

im not sure which part of my summary you take issue with then? the reason for study 1b is not relevant to answer the question asked in the comment i replied to

It is relevant, because of (1) framing (people thought they were getting answers from AI, not a Google search, and any associated reputation works in that way); and (2) sycophancy and other similar characteristics of the answer's text, which were present in the answers presented in 1b and wouldn't be in a Google search.

Basically, the difference between 1a and 1b is not at all relevant to the question of whether the observed behavior is caused by AI or simply by faulty tools. The difference between 1a and 1b was designed specifically to be transparent to the actual test takers, and only to eliminate some confounding variable (technical issues in 1a).

Re: AI advice made people less accurate but more confident – sudy

#206

Post by ShinyLeftPad »

dwohnitmok wrote: post #18

This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…

The implication that makes this study relevant is that an LLM is vastly more likely to have factual errors and possibly wildly hallucinate than a proper textbook. If people act the same with both, that IS the actual problem.

Re: AI advice made people less accurate but more confident – sudy

#207

Post by mapontosevenths »

what wrote: post #166
mapontosevenths wrote: post #140

Earlier quoted context omitted.

I suppose that "civil" is relative. Personaly, I wouldn't speak to someone who just tried to help me that way. Instead I would restate the question to avoid bad answers. If you start by saying what you've already tried, and how it worked, it cuts off most of those "lazy answers." In my experience most lazy answers are provided as response to lazy questions.

Sending that “lazy answer” because you assumed the other person hasn’t tried solving the question themselves seems like the “uncivil” action. You should probably assume your colleague has tried before asking for help, and probably tried the same lazy route you used to respond.

> You should probably assume your colleague has tried before asking for help,

Spoken like someone who has never worked in a customer support role.

You can never assume competency, from anyone. Even the best amd brightest have bad days.

Re: AI advice made people less accurate but more confident – sudy

#208

Post by dnemmers »

slibhb wrote: post #95
tsimionescu wrote: post #46

Earlier quoted context omitted.

Why are textbooks relevant here? Even if you repeated the experiment with a textbook instead of the AI and got the same result, what conclusion would you draw from this? The general conclusion of the study seems to be "giving people access to authoritative-seeming but wrong tools for answering questions outside their area of expertise reduces their ability to say they don't know the answer, even when the answer is wr…

> but also don't give them AI for things they don't know, perhaps The study doesn't show that at all. It didn't test actual AI. They could have tested a cohort of subjects with access to actual ChatGPT. Ask yourself why they didn't.

Old AI is so bad it should be disregarded, but new AI is so good, you don't even have to verify its output....

Is that what you're selling us?

So in 18 months, we'll just rinse and repeat?

Re: AI advice made people less accurate but more confident – sudy

#209

Post by rbanffy »

rsoto2 wrote: post #181
RA_Fisher wrote: post #148

Earlier quoted context omitted.

Exactly, it’s unrepresentative of AI. It’s damaged AI.

All AI is flawed and prone to "hallucinations"(doing exactly what it was designed to do) that's why Microsoft considers it an "Entertainment" product.

I prefer to say AIs are prone to misremember things, as well as humans do. The more you read and learn, the more material you have to get confused, unfortunately.

We might want to work around the certainty with which it misremembers things. I have a very large interval of "I have no idea", or "I don't remember" and rarely caught myself misremembering things (my children are far better at that). I assume my grandchildren will eventually improve their parents' scores by a wide margin.

Re: AI advice made people less accurate but more confident – sudy

#210

Post by mapontosevenths »

Forgeties79 wrote: post #176
mapontosevenths wrote: post #172

Earlier quoted context omitted.

> I too have access to LLM’s. You should have stated what you already tried then. You didn't, so they did the obvious thing for you... to do you a favor. If you'd tried something before asking them surely you'd have told them that at the start, right? > as evidenced by all the slop dumping I see more evidence of lazy questions. A good question includes what you've already done to help understand the issue (often refe…

> You should have stated what you already tried then. You didn't, so they did the obvious thing for you... to do you a favor. If you'd tried something before asking them surely you'd have told them that at the start, right? I’m not sure what you mean here, might be a misunderstanding. My point is don’t use your LLM and simply paste it. I have an LLM, I can use it too. We all have access to them. We should all operate…

> We should all operate under that assumption

We've all had access to Google for decades. People still don't Google things.

If you go around assuming that other people are competent and did the work you would have done you will be disappointed a significant percentage of the time and waste a lot of everyone's time.

Post reply on HN