Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI advice made people less accurate but more confident – sudy

thenextweb.com

Post reply on HN
Print view View original

31–40 of 234 posts

Re: AI advice made people less accurate but more confident – sudy

#31

Post by dwohnitmok »

nkrisc wrote: post #24
dwohnitmok wrote: post #18

This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…

You could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that, while out in the world people do use LLMs like that all the time.

People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz).

I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two.

This is why I think it's a bad study. It didn't measure anything of the essential differences of how people use LLMs.

Re: AI advice made people less accurate but more confident – sudy

#32

Post by skippyfish »

darkstarsys wrote: post #22
reticulates wrote: post #9

Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…

Exactly the same thing happened when Google appeared on the scene. (Not that this makes it any better!)

Sort of, but I think this was a mix of "I want to sound smart" and "I actually want to know the answer", because summarizing Google search results still required you to read the content, form an opinion, and then rephrase it.

With AI, people literally just act as token proxies and are doing it purely for internet points (upvotes, followers, etc)... I guess on some level, they're just providing a service that the "market" implicitly values. But also, in Bob Slydell's voice: "what would you say you do here?"

Re: AI advice made people less accurate but more confident – sudy

#33

Post by nkrisc »

dwohnitmok wrote: post #31
nkrisc wrote: post #24

Earlier quoted context omitted.

You could make the point that it’s no different than the textbook example you gave, but people don’t generally use textbooks like that, while out in the world people do use LLMs like that all the time.

People do use textbooks like that all the time in the experimental setup tested (essentially an open book quiz). I agree there are important differences in how textbooks and LLMs are used in real life. This study didn't explore that at all. It used a setup that essentially elided the difference between the two. This is why I think it's a bad study. It didn't measure anything of the essential differences of how people…

It would be interesting to test those LLM-specific features and issues, but I don’t see how it’s a bad study if it does reflect how people actually use LLMs, even if they could use other sources similarly.

The number of people using LLMs must dwarf the number of people using textbooks for any reason.

Re: AI advice made people less accurate but more confident – sudy

#34

Post by hinkley »

reticulates wrote: post #9

Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…

There are some people I'm not friends with anymore who didn't understand conversational bidding. They would jump straight to Why Don't You Google That and shut down any conversation we might have had.

Yes I could get some of this information off of IMDB or a DIY video but I'm looking for color commentary and regional advice, and also in some of these situations, the people I asked know me. They know the sorts of things I don't need help on and the ones I struggle with. So maybe they can tell me this movie or trick isn't for me and I should try this other thing instead. Or which store I should go to for this.

Re: AI advice made people less accurate but more confident – sudy

#35

Post by sandeepkd »

tripleee wrote: post #17
adamddev1 wrote: post #16

Earlier quoted context omitted.

And then the AI gets trained on the advice and information subreddits, and we see an exponential growth downwards in quality?

Isn't AI getting more accurate over time though, as per benchmarks?

> as per benchmarks

That seems to be the key and its risky when the measure becomes the target itself. When you already know the benchmarks, and that drives the definition of success outcome then there is every incentive to just chase them

Re: AI advice made people less accurate but more confident – sudy

#36

Post by abernard1 »

Wouldn't it be wild if there were real-world penalties for being wrong?

And people who follow bad advice will get bad results, and people who follow good advice will get good results?

I wonder if this will impact the quality and prevalence of certain AI models in the future.

Re: AI advice made people less accurate but more confident – sudy

#37

Post by emp17344 »

lll-o-lll wrote: post #29
emp17344 wrote: post #10

Earlier quoted context omitted.

The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.

That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.

The marketing and ecosystem surrounding these tools feeds into the anthropomorphization to a huge extent, though. You’re doing the opposite - throwing up your hands and going “it’s our evolutionary instinct, what can you do” while willfully ignoring how manipulated this space is. OpenAI alone has a higher advertising budget than Coca-Cola.

Re: AI advice made people less accurate but more confident – sudy

#38

Post by mmooss »

lll-o-lll wrote: post #29
emp17344 wrote: post #10

Earlier quoted context omitted.

The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.

That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.

Both claims seem to obviously contradict reality, imho:

> Humans are wired to anthropomorphise; we do it with sticks and rocks.

I do not see people anthropomorphize their TVs, books, paintings, cars, houses, rocks, sticks, ... I'm not sure what you're talking about here. I haven't seen someone call a rock or stick by a pronoun, for example.

I've seen people fancifully do it, but it's exaggeration for a cute effect, and works because everyone knows they're exaggerating.

> We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.

People learn to look at things in different ways. Humans learn, society changes. It's changed quite a bit in recent years; it's much different than a century ago, a century before that, etc.

In a way, it's more worship of technology to say humans are hopelessly, powerlessly dumb, and technology inevitably rules them. The inevitability of technology is of course a self-serving trope of the industry: you are powerless, we are inevitable. How convenient, and short-sighted.

Re: AI advice made people less accurate but more confident – sudy

#39

Post by mohamedkoubaa »

lll-o-lll wrote: post #29
emp17344 wrote: post #10

Earlier quoted context omitted.

The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.

That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.

People who should know better do it too

Re: AI advice made people less accurate but more confident – sudy

#40

Post by ffaccount2 »

wccrawford wrote: post #12
reticulates wrote: post #9

Advice and information subreddits have gone to shit because of AI usage. A large number of people seem to think that when someone asks a question, what they really want is not someone with direct knowledge, but instead someone to relay the question to ChatGPT and post the result as if it is their own hard earned knowledge and insight. I have no idea about the quality of this research but in the real world (well, real…

I think some may think they're helping like that, but I'm sure there are quite a few who enjoy the feeling of being a notable figure in the community because they answer so many questions. These people have always existed and gave bad answers, but now they can do it so much faster , and with so much less actual thinking.

I did that when I was younger and internet forums we're still a thing. The difference is that I had to do actual research, which often took me half an hour or more, to solve the problem for the asker. I got knowledge and "the feeling of being a notable figure" as you have said. The other side for the answer. I believe my answers were good.

The difference is that nowadays people can just skip the research part and copy-paste Llama. No value added.

Post reply on HN