41–50 of 234 posts
> Researchers found AI advice suppressed judgment suspension from 44% to 3%, accuracy from 27% to 9%, while confidence rose from 30% to 76%. People trusted wrong AI answers.
I didn't see the article mentioned what they were actually asked, but I'm surprised confidence was only ~30%.
Earlier quoted context omitted.
That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.
The marketing and ecosystem surrounding these tools feeds into the anthropomorphization to a huge extent, though. You’re doing the opposite - throwing up your hands and going “it’s our evolutionary instinct, what can you do” while willfully ignoring how manipulated this space is. OpenAI alone has a higher advertising budget than Coca-Cola.
And especially their design. The con artist politeness and confidence is not an accident of output, but a design choice by the developers.
I tell non-technical people: It's a software program; there is always a person behind it, and they have consciously chosen this output, to serve their own interests.
Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
I'd wager you get similar results if you gave people a version of Google search that purposely gave you bad results. Like, it's framed as an assistant / lookup tool - is it so surprising that people tend to trust it more? Especially since the participants are likely used to using full-powered models and the researchers give them a purposely gimped one (lol)
People are acting rationally when given AI tools to lookup information, their first consumer use case was as a super-powered Google Search
AI pessimist view. Even if AI gets smarter it will still be agreeable, and people will use it to more confidently reinforce thier stupider ideas, especially in areas where they lack the knowledge to know if they are right. Even if this can be solved, technically, people won't want to use the model that says they are wrong, so they will choose the glib lies that reinforce their beliefs. Freedown of speech forces peopl…
Agree. AI behavior, when interacting with humans, is based on making money. Making people feel uncomfortable or challenged, will usually not be the preferred business strategy. But, it would be interesting if people could easily select the AI's personality type or to what extent it can be adversarial to the comments or ideas given as input.
I do like the idea of AI personalities ( I actually use them a fair bit for varying opposing perspectives ), but I am not entirely certain people want that.
This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…
I'll also add that even in these simple experimental conditions, I'd bet that having access to a textbook wouldn't have nearly as much of an effect, for a very simple reason: looking up an answer in a textbook is a lot more work than asking an LLM. So when you don't know and aren't forced to answer, I'd bet it's a lot less likely you'd spend the time to look up the answer in the text book. Even more so if the textbook had "this may contain wrong answers!" printed on the cover, like the AIs do.
Earlier quoted context omitted.
You could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that, while out in the world people do use LLMs like that all the time.
People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two. This is why I think it's a bad study. It didn't measure anything of the essential differences of how people…
What open book quizzes allow you to leave all answers blank with no penalty? An open book quiz is very different from the experimental setup tested here.
Earlier quoted context omitted.
People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two. This is why I think it's a bad study. It didn't measure anything of the essential differences of how people…
It would be interesting to test those LLM-specific features and issues, but I don’t see how it’s a bad study if it does reflect how people actually use LLMs, even if they could use other sources similarly. The number of people using LLMs must dwarf the number of people using textbooks for any reason.
I get why they used questions where AI models fail, but it also really reduces the value of this study. Nobody is really asking AI the color of a team's uniform in a movie, and if they do and confidently get it wrong, it just doesn't matter at all.
Asking trivial questions also feels like it would affect the rate at which people are willing to confidently say things that are wrong. If you ask me some question of pointless trivia and I ask ChatGPT, I'll probably just repeat the answer because who cares. If you ask me something even mildly important and I ask ChatGPT, I'll either verify the information before I repeat it to you, or I'll qualify that I looked it up with ChatGPT and didn't verify. But some things are just so unimportant that they don't even warrant the disclaimer.