Live data from Hacker News

AI advice made people less accurate but more confident – sudy

thenextweb.com

91–100 of 234 posts

Re: AI advice made people less accurate but more confident – sudy

#91
post #74

Earlier quoted context omitted.

Writing and videos produced before 2022 are the information equivalent of low-background steel (steel produced before the atom bomb era). Not all are precious, but they are important. AI influences are so pervasive at this point, even in informational writing from domain experts. Relentless grounding is my personal solution for my AI epistemic crisis, but it's an expensive solution in terms of time and effort, and tr…

What is "grounding"?

[deleted]

Re: AI advice made people less accurate but more confident – sudy

#92
post #74

Earlier quoted context omitted.

Writing and videos produced before 2022 are the information equivalent of low-background steel (steel produced before the atom bomb era). Not all are precious, but they are important. AI influences are so pervasive at this point, even in informational writing from domain experts. Relentless grounding is my personal solution for my AI epistemic crisis, but it's an expensive solution in terms of time and effort, and tr…

What is "grounding"?

I think in this context, they mean seeking the primary source of the information by hand.

Re: AI advice made people less accurate but more confident – sudy

#93
post #74

Earlier quoted context omitted.

What is "grounding"?

It's when the models outputs are connected to sources. So for example, you might have a company chatbot that responds with a summary of a Confluence page it was trained on and a link to the source. Doesn't eliminate hallucinations, but it does reduce them and gives you a chance to verify.

I don’t think this is correct, as we’re talking about all online discourse being influenced by AI, and you don’t know what to trust anymore, rather than generating AI output yourself (which is what you are referring to).

Re: AI advice made people less accurate but more confident – sudy

#94

Earlier quoted context omitted.

The advice subreddits were absolutely horrible before AI. The entire 2016-2024 period was utter madness there, likely as a result of globalization. As AI gets smarter, it may make the comments better, sadly!

Maybe, but isn't it likely the models are being trained on those same subreddits?

They are, and they'll also see a random reddit comment in their web search and treat it as fact.

Re: AI advice made people less accurate but more confident – sudy

#95

This study is pretty bad. The comment ( https://news.ycombinator.com/item?id=48970182 ) on the other link with the direct PDF explains the problem well, which is that nothing here being tested is specific to AI systems. This study gave people access to an LLM that the researchers knew would give incorrect answers to certain questions, and then quizzed people on those questions, with the option to not respond to a giv…

Why are textbooks relevant here? Even if you repeated the experiment with a textbook instead of the AI and got the same result, what conclusion would you draw from this? The general conclusion of the study seems to be "giving people access to authoritative-seeming but wrong tools for answering questions outside their area of expertise reduces their ability to say they don't know the answer, even when the answer is wr…

> but also don't give them AI for things they don't know, perhaps

The study doesn't show that at all. It didn't test actual AI.

They could have tested a cohort of subjects with access to actual ChatGPT. Ask yourself why they didn't.

Re: AI advice made people less accurate but more confident – sudy

#96
post #74

Earlier quoted context omitted.

What is "grounding"?

It's when the models outputs are connected to sources. So for example, you might have a company chatbot that responds with a summary of a Confluence page it was trained on and a link to the source. Doesn't eliminate hallucinations, but it does reduce them and gives you a chance to verify.

That works ok until the citation material ends up with LLM output being fed in. Already there are many examples of this happening.

Re: AI advice made people less accurate but more confident – sudy

#97

Earlier quoted context omitted.

The solution is to stop treating AI as superhuman semi-living entities, as many boosters are apt to do. We can acknowledge it as a useful tool while understanding it has limits and is not “alive” or “thinking” in the same way a human is.

That’s not a solution. Humans are wired to anthropomorphise; we do it with sticks and rocks. How much more so with LLMs. We’ve invented a technology that hijacks that part of humanity more effectively than anything ever. Asking people to “not be so stupid” is not going to help.

[dead]

Re: AI advice made people less accurate but more confident – sudy

#98

Earlier quoted context omitted.

It's when the models outputs are connected to sources. So for example, you might have a company chatbot that responds with a summary of a Confluence page it was trained on and a link to the source. Doesn't eliminate hallucinations, but it does reduce them and gives you a chance to verify.

I don’t think this is correct, as we’re talking about all online discourse being influenced by AI, and you don’t know what to trust anymore, rather than generating AI output yourself (which is what you are referring to).

In this context, it would mean building a knowledge base the model can link to that is considered authoritative. So maybe the LLM is has some slop in it's training, and that causes it to try and output some junk, but if it can't match it to it's verified ground-truth KB it doesn't end up outputting it.

The problem is figuring out what is the authoritative sources.

Re: AI advice made people less accurate but more confident – sudy

#99
post #7

AI pessimist view. Even if AI gets smarter it will still be agreeable, and people will use it to more confidently reinforce thier stupider ideas, especially in areas where they lack the knowledge to know if they are right. Even if this can be solved, technically, people won't want to use the model that says they are wrong, so they will choose the glib lies that reinforce their beliefs. Freedown of speech forces peopl…

Alas, a lot of people have always preferred a comfortable lie than an uncomfortable truth.

Re: AI advice made people less accurate but more confident – sudy

#100

Earlier quoted context omitted.

I think some may think they're helping like that, but I'm sure there are quite a few who enjoy the feeling of being a notable figure in the community because they answer so many questions. These people have always existed and gave bad answers, but now they can do it so much faster , and with so much less actual thinking.

I did that when I was younger and internet forums we're still a thing. The difference is that I had to do actual research, which often took me half an hour or more, to solve the problem for the asker. I got knowledge and "the feeling of being a notable figure" as you have said. The other side for the answer. I believe my answers were good. The difference is that nowadays people can just skip the research part and cop…

>The difference is that nowadays people can just skip the research part and copy-paste Llama. No value added.

and why would the person doing that do that? if the person asking the question wanted to know what an llm thought, wouldn't they have just asked the llm for an instant response instead of having to wait for someone else to respond? i'm at a total loss of being able to understand the point. it's obviously different if the response is a bot posting just in case that needs to be explained to a bot.

Post reply on HN