Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you deal with people who trust LLMs?

news.ycombinator.com

Post reply on HN
Print view View original

171–180 of 236 posts

Re: Ask HN: How do you deal with people who trust LLMs?

#171

Post by Jensson »

lovelearning wrote: post #168
Jensson wrote: post #165

Earlier quoted context omitted.

How does this answer the question: "how do you deal with people who trust LLMs?"? Nothing you are saying explains how to deal with such people.

I felt the question is based on some shaky assumptions that may lead to a poor answer. Since the OP trusts humans more by default, is it a problem if I point out those assumptions? Ask HN need not become another SO. I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we…

> I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources.

That just explains how to find facts for yourself, not how to deal with a person who trust LLM outputs. So you still haven't answered the question.

> Since the OP trusts humans more by default

OP never said that. OP said there is a problem with people trusting LLM instead of doing proper research and finding good sources, you explaining how to do proper research and finding good sources doesn't have anything to do with the question.

Re: Ask HN: How do you deal with people who trust LLMs?

#172

Post by mingus88 »

nomilk wrote: post #48

Simply prove them wrong (earnestly and in good faith). When they realise the LLM is fallible, they'll learn to be skeptical of it without you needing to teach them that specific lesson.

Have you had much luck with that in real life?

Even before LLMs we had a misinformation problem, and people just believe what they want to believe.

“Well, it’s exactly the kind of thing they would do” when you point out that, in fact, it did not happen.

The internet gave us all comfortable bubbles to live in, and people generally like to be comfortable.

Re: Ask HN: How do you deal with people who trust LLMs?

#173

Post by lovelearning »

basilikum wrote: post #167
lovelearning wrote: post #125

> a reputable source News reporters and editors have their biases. Book authors have their biases. Scientists and research papers have their biases. Search engines have their biases. Google too. All human-created systems have biases shaped by the environments, social norms, education, traditions, etc. of their creators and managers. So, the concepts of "objective truth" and "reputable" need to be analyzed more critic…

> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer? If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use…

> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration

I didn't understand the second part but regarding the first...

For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.

Some may consider some subset of these as reputable sources. But in my mind, the same faculties of skepticism, cynicism, distrust, and benefit-of-the-doubt calculus are activated for all of them, including LLM outputs.

So that's one possible answer to your question.

But I suggest communicating this through simple illustrative examples to help your target audience understand the problem.

Abstract terms like primary sources, secondary sources, reputable sources, objective truth, strict degeneration, etc. may not help, especially if they have time or other constraints that make frequent critical examination of sources impractical.

Re: Ask HN: How do you deal with people who trust LLMs?

#174

Post by lovelearning »

Jensson wrote: post #171
lovelearning wrote: post #168

Earlier quoted context omitted.

I felt the question is based on some shaky assumptions that may lead to a poor answer. Since the OP trusts humans more by default, is it a problem if I point out those assumptions? Ask HN need not become another SO. I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we…

> I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources. That just explains how to find facts for yourself, not how to deal with a person who trust LLM outputs. So you still haven't answered the question. > Since the OP trusts…

I gave suggestions that OP can pass on to the people they have to deal with. I didn't realize it has to be pointed out explicitly SO-style to some people.

OP implies human sources are the "good sources" or "reputable sources." This kind of confusion is exactly why I suggested using better terms than "reputable sources" or in your case "good sources."

Re: Ask HN: How do you deal with people who trust LLMs?

#175

Post by jimcollinswort1 »

I would question the person that asks the question, as they are not understanding some basic principles here. There are two types of internet/LLM users;

Sadly one type asks a question (search, prompt) using Google or an LLM and takes the first response as truth.

The other asks follow ups based on the responses and their critical thinking skills. They often even go read the linked article and make sure it's still applicable.

Pretty much the same when you're talking to a real person, critical thinking (much more than just knowing reputable sources) is key.

So very similar issues, luckily LLMs can do so much more than a simple search, and help with your critical thinking tasks. Ask the LLM to provide opposing viewpoints, historical analysis, identify sources.

Re: Ask HN: How do you deal with people who trust LLMs?

#176

Post by Jensson »

lovelearning wrote: post #174
Jensson wrote: post #171

Earlier quoted context omitted.

> I did explain the weaknesses of both LLMs and "reputable sources" and suggested people use them as complementary tools. I also suggested using the convenient self-fact-check feature of LLMs, something we can't do as easily with traditional sources. That just explains how to find facts for yourself, not how to deal with a person who trust LLM outputs. So you still haven't answered the question. > Since the OP trusts…

I gave suggestions that OP can pass on to the people they have to deal with. I didn't realize it has to be pointed out explicitly SO-style to some people. OP implies human sources are the "good sources" or "reputable sources." This kind of confusion is exactly why I suggested using better terms than "reputable sources" or in your case "good sources."

> OP implies human sources are the "good sources" or "reputable sources."

No OP did not do that, nowhere did OP mention human sources. When I search the internet I don't just find human sources, I also find automatically generated data graphs and maps and such, those are also good sources of data. If I had to choose between a map from Google maps and a map from an LLM I'd trust the map from google maps any day.

Re: Ask HN: How do you deal with people who trust LLMs?

#177

Post by Jensson »

lovelearning wrote: post #173
basilikum wrote: post #167

Earlier quoted context omitted.

> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer? If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use…

> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration I didn't understand the second part but regarding the first... For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers. Some may consid…

> For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers.

LLM just distils information from those sources and is therefore always a second hand source at best, and a liar at worst. Humans can collect real world data and write about their findings, LLM cannot do that, that makes LLM strictly worse than the best human sources.

Re: Ask HN: How do you deal with people who trust LLMs?

#178

Post by Jensson »

jimcollinswort1 wrote: post #175

I would question the person that asks the question, as they are not understanding some basic principles here. There are two types of internet/LLM users; Sadly one type asks a question (search, prompt) using Google or an LLM and takes the first response as truth. The other asks follow ups based on the responses and their critical thinking skills. They often even go read the linked article and make sure it's still appl…

> I would question the person that asks the question, as they are not understanding some basic principles here.

What principle isn't he understanding? He has a problem with people trusting LLM too much, what is it that he doesn't understand here?

To me it seems like you are missing something, not him. That some people uses LLM properly doesn't resolve his issue.

> So very similar issues

Ok, so how do you deal with that? You didn't answer.

Re: Ask HN: How do you deal with people who trust LLMs?

#179

Post by lovelearning »

dalke wrote: post #170
lovelearning wrote: post #161

Earlier quoted context omitted.

You're right that LLMs do spit out false information or wrong knowledge. I've experienced them too. But a redeeming quality is that we can ask the same LLM to fact check its own answer step by step in real time with little effort. They often identify their own hallucinations and reduce the probability of retaining that mistake in the rest of the conversation. This isn't easy with human sources. The effort to fact che…

The effort to fact check with LLMs is also high. Here's one from a few days ago. Someone used AI to generate an image in the style of a Charles Schulz Peanuts cartoon. Someone else observed that there were 5 fingers on the characters, and quoted as Google AI as saying “Charlie Brown, along with other Peanuts characters, is generally depicted with four fingers on each hand (three fingers and one thumb) ...” Yet if you…

> When do you stop the fact checking?

Exactly the same calculus as fact checking anything else from any other source. What are the social/economic/ethical consequences to me if the answer is wrong or inaccurate or incomplete? How much time do I have to check? How thorough should I be?

I imagine this calculus isn't really that different for most people. Or is it?

As for your example, I believe it. But I also feel it's a rather outlier example involving image comprehension of an obscure factoid. That isn't typical of how I use LLMs which is mostly as text-based question answering engines and not what I had in mind when writing the comment.

I guess LLMs for image comprehension need a much higher level of skepticism.

Re: Ask HN: How do you deal with people who trust LLMs?

#180

Post by rithdmc »

lovelearning wrote: post #173
basilikum wrote: post #167

Earlier quoted context omitted.

> Is Google search engine that leads to NY Times or Fox News or Wikipedia and makes us manually choose sources as per our biases "better" than Google's Gemini engine that summarizes content from all the above sources and gives an average answer? If you use just any amount of critical thinking, yes. Truth and objectivity are ideals, not practical states. LLMs are a very bad way to come close to this ideal. You may use…

> LLMs are a very bad way to come close to this ideal...the output directly is a strict degeneration I didn't understand the second part but regarding the first... For me, LLMs are just another source of information with a different UI, analogous to newspapers, TV documentaries, Wikipedia, Google search, YT talks/documentaries, even the majority of informational non-fiction books, and research papers. Some may consid…

Newspapers, TV documentaries, Wikipedia, research papers, etc tend to be edited or peer reviewed.
Post reply on HN