AI assisted/induced Dunning–Kruger effect
AI advice made people less accurate but more confident – sudy
11–20 of 234 posts
Re: AI advice made people less accurate but more confident – sudy
#12Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…
I think some may think they're helping like that, but I'm sure there are quite a few who enjoy the feeling of being a notable figure in the community because they answer so many questions.
These people have always existed and gave bad answers, but now they can do it so much faster, and with so much less actual thinking.
Re: AI advice made people less accurate but more confident – sudy
#13Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…
We had to officially set the culture in the work slack. If you're just piping through raw AI response without any input of your own, you're being rude.
It goes a long way.
Re: AI advice made people less accurate but more confident – sudy
#14Very concerning - I’ve tried n+m, n*m, n^m, and even m^n, but I can’t seem to reproduce the popular “10x” claim.
Re: AI advice made people less accurate but more confident – sudy
#15This was posted earlier but didn’t get traction, and I made the following comment: Id want to know if “AI” makes a material difference vs just having access to the wrong answer. Like someone could be given search access that successfully retrieved wrong answers to questions, would that give the same results. How much do uniquely AI characteristics, like sycophancy or the conversational aspect play into this, vs peopl…
You can read the study here: https://osf.io/preprints/psyarxiv/5y6m4_v1
TLDR: They studied both cases (Access to a LLM/Chat interface which gives a wrong answer when asked, and access to pregenerated (wrong) answers). Both experiments yielded similar results.
Re: AI advice made people less accurate but more confident – sudy
#16Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…
And then the AI gets trained on the advice and information subreddits, and we see an exponential growth downwards in quality?
Re: AI advice made people less accurate but more confident – sudy
#17Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…
And then the AI gets trained on the advice and information subreddits, and we see an exponential growth downwards in quality?
Isn't AI getting more accurate over time though, as per benchmarks?
Re: AI advice made people less accurate but more confident – sudy
#18This study is pretty bad. The comment (https://news.ycombinator.com/item?id=48970182) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems.
This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a given question if they are unsure about the answer.
This is akin to giving someone a textbook on an obscure subject that has certain factual errors, letting them know they can use that textbook in a quiz on that subject, and then quizzing that person on those facts that the textbook gets wrong.
Obviously that person is both more likely to be willing to respond to the question and is more likely to get it wrong!
There are a lot of things I'm very interested in that are specific to modern LLMs and how they affect learning and confidence (sycophancy, cognitive helplessness, etc.).
This study tested none of those. Its experimental setup is not very different than simply substituting the LLM with a textbook with errors.
Re: AI advice made people less accurate but more confident – sudy
#19[flagged]
Re: AI advice made people less accurate but more confident – sudy
#20Earlier quoted context omitted.
And then the AI gets trained on the advice and information subreddits, and we see an exponential growth downwards in quality?
Isn't AI getting more accurate over time though, as per benchmarks?
Yes, but that's with mostly the same data as before, just with more of the preexisting dataset and more RLHF. We haven't seen the ouroborous eat itself yet.