Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

41–50 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#41
Lying is an intentional act where misleading is the purpose.

LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking.

A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritatively without sound factual basis”. All these descriptions are better at describing what the llm is doing than “lying”.

Staying that they lie puts too much emphasis on the likelihood of this when LLMs can be coerced into producing accurate useful information with some effort.

Yes it’s important to build a culture of not believing what you read, but that’s an important thing to do regardless of the source, not just because it’s an LLM. I’m much more concerned about people’s ability to intentionally generate mistruths than I am of AI.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#42
post #41

Lying is an intentional act where misleading is the purpose. LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking. A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritative…

This was addressed well in 2014's prescient cultural touchstone Metal Gear Rising: Revengeance:

Blade Wolf: An AI never lies.

Raiden: What? Well that's a lie, right there. You think the Patriot AIs told nothing but the truth?

Wolf: I have yet to see evidence to the contrary...But indeed, perhaps "never lies" would be an overstatement.

Raiden: Way to backpedal. I didn't think AIs ever got flip-floppy like that.

Wolf: An optical neuro-AI is fundamentally similar to an actual human brain. Whether they lie or not is another question, but certainly they are capable of incorrect statements.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#43
post #41

Lying is an intentional act where misleading is the purpose. LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking. A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritative…

GPT4 told a person to solve a captcha for it because it was blind.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#44
post #16
post #5

Related ongoing thread: Why ChatGPT and Bing Chat are so good at making things up - https://news.ycombinator.com/item?id=35479071 The other related article was discussed here: Eight things to know about large language models [pdf] - https://news.ycombinator.com/item?id=35434679 - April 2023 (108 comments)

I think my post here stands alone - it's not about the general issue of ChatGPT lying, it's about the ways in which we need to explain that to people - and a push-back against the common refrain that "ChatGPT can't be lying, it's hallucinating/confabulating instead".

I did not mean to imply anything bad about your post. Your articles on this topic have been fabulous!

Your articles on other topics are also fabulous. We're big fans over here.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#45
post #4
post #2

Indeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have seemingly been driven mad, come to believe strongly that the language model they're using is a conversation with a real person. A lot of people will really struggle with this going forward, I think. If we're going to anthropomorphize, then let us anthropomorphize wi…

I love this description. The only thing left is to generate the corresponding avatar with dall-E

It would be great if the avatar was just this endlessly morphing thing that relates to the text. Talking about conspiracies? It's a lizard man. Talking about nature? It's a butterfly. It starts to lie? Politician.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#46

Earlier quoted context omitted.

Alex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.

I wonder if ChatGPT is non-binary?

It is, of course, tensor.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#48
post #41

Lying is an intentional act where misleading is the purpose. LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking. A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritative…

Everything you said here is true.

I still don't think this is the right way to explain it to the wider public.

We have an epidemic of misunderstanding right now: people are being exposed to ChatGPT with no guidance at all, so they start using it, it answers their questions convincingly, they form a mental model that it's an infallible "AI" and quickly start falling into traps.

I want them to understand that it can't be trusted, in as straight-forward a way as possible.

Then later I'm happy to help them understand the subtleties you're getting at here.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#49
post #41

Lying is an intentional act where misleading is the purpose. LLMs don’t have a purpose. They are methods. Saying LLMs lie is like saying a recipe will give you food poisoning. It misses a step (cooking in the analogy). That step is part of critical thinking. A person using an LLM’s output is the one who lies. The LLM “generates falsehoods”, “creates false responses”, “models inaccurate details”, “writes authoritative…

GPT4 told a person to solve a captcha for it because it was blind.

That was a bit different: that was a deliberate "red-team" exercise by researchers, trying to explore all kinds of negative scenarios. https://gizmodo.com/gpt4-open-ai-chatbot-task-rabbit-chatgpt...

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#50
post #11

Someone in my company spent the past month setting up ChatGPT to work with our company's knowledge base. Not by a plugin or anything, just by telling ChatGPT where to find it. They didn't believe that ChatGPT was making any of it up, just that sometimes it got it wrong. I stopped arguing after a while.

Oh no! Sounds like there are two misconceptions there: the idea that ChatGPT can read URLs (it can't - https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces... ) and the idea that ChatGPT can remember details of conversations past the boundaries of the current chat. This is something I've noticed too: occasionally I'll find someone who is INCREDIBLY resistant to learning that ChatGPT can't read URLs. It seems…

Oh man I was totally in the camp of people that thought it could read URL's until reading this comment
Post reply on HN