We need to tell people ChatGPT will lie to them, not debate linguistics

This whole conversation is speculative, obviously. The AI doomers are all speculating without evidence as well.

The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.

> Most humans do not have sufficient background to understand what they're really being presented with, they will take it at face value.

Trust in all forms of media [1], and institutions [2], is at an all time low. I'm not sure why that distrust would go away, with reading the output from a company that's clearly censuring/fudging that output.

I think any "damaging" trust would, clearly, be a transitory phenomenon, since the distrust in media and institutions is from an ability to see BS. I don't think some sentences on a screen will be as destructive as some think, because people don't appear to be as stupid as some believe.

1. https://news.gallup.com/poll/403166/americans-trust-media-re...

2. https://news.gallup.com/poll/394283/confidence-institutions-...

thing is, it's not load bearing information.

It was not a confident statement, at least not the way ChatGPT is confident.

There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close

The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold.

If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncertain statements is fine as long as it is clear the uncertainty is acknowledged. ChatGPT has no concept of uncertainty. It casually constructs false statements to same way it constructs real knowledge backed by evidence. That's the problem.

> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess.

That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think it's friendly, your intent is to deceive.

> That's equivalent to an LLM, it is mid-thought, stuck at the most recent batch of training samples it was given when you come along and you interrupt it with some new thought.

For someone who thinks hot-takes anthropomorphizing ChatGPT and LLMs shouldn't be given any serious attention, you sure are anthropomorphizing LLMs...

The thing is, LLMs aren't doing that at all. They're just pattern matching prompt data to training data and then mixing together a response from the "continuations" of the matched training data. Nothing is being "interrupted". They're sitting around waiting for prompts.

LLMs can't lie because they can't intend to deceive. They can't intend at all. They have no intent, no will. These are anthropomorphic qualities you're putting on them, which they simply don't have.

But, I think the original article IS NOT arguing that LLMs are capable of lying. It's arguing that the average human doesn't understand what's going on with LLMs, and isn't going to go through the effort to gain that understanding. That's not a judgment: the average person has better things to do with their time than learn all the necessary prerequisite knowledge to understand LLMs. It's a hard problem, because we're trying to explain something really complex to people who don't even understand the building blocks the complexity is made out of. The question the original article is answering is: what's an effective metaphor to explain LLMs to laypeople?

The approach being suggested by the OP to solving this problem is to use an anthropomorphic analogy which gives people the understanding of whether to trust information gained from LLMs. You tell them that LLMs are lying. Yes, we understand that they aren't literally lying--as I said before, LLMs don't have intent so they can't intend to deceive. It's a metaphor.

Telling people that LLMs are wrong sometimes gives the wrong impression. Most people are wrong sometimes, but people are usually trying to be honest and will usually try to not mislead you when it's important, and even when people might be wrong it's worthwhile to take the risk of trusting people because the alternative is living in fear and loneliness. So telling people LLMs are wrong sometimes, gives people the idea that LLMs make mistakes, but they're trying their best so you can trust them. Which is not accurate: LLMs are as biased and confused as the humans that created their training data, but on top of that, have a lot less variety of data, and sometimes randomly combine things in the wrong ways. LLMs can't generally be trusted: they must be generally distrusted, especially on anything where the facts matter.

Telling people that LLMs lie produces the correct general impression. People don't trust a known liar. They might listen to a liar, but they'll take what the liar says with a grain of skepticism. And that's exactly how we should be treating what LLM's say, given their current capabilities.