Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

111–120 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#111
post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

> Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? "Plausible" means "that which the majority of people is likely to say". So, yes, a foundational model is likely to say the plausible thing. On the other hand, it has to have a way to output a truthful answer too, to not fail on texts produced by experts. So, it's not impossible that the model coul…

> "that which the majority of people is likely to say"

.. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#112

People need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. They need to be told that ChatGPT is a search engine with advanced autocomplete. If they understood this, they'd probably find that it's actually useful for some things, and they can also av…

What are your thoughts on something like this [0], where ChatGPT is accused of delivering allegations of impropriety or criminal behavior citing seemingly non existent sources?

https://www.washingtonpost.com/technology/2023/04/05/chatgpt...

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#113
post #102

No we don't. I treat ChatGPT like a person. People are flawed. People lie. People make shit up. People are crazy. Just apply regular people filters to ChatGPT output. If you blindly believe ChatGPT output and take it for perfect truth, someone likely already sold you a bridge.

Actual people are also likely to tell you if they don't know something, aren't sure of their answer, or simply don't understand your question. My experience with ChatGPT is that it's always very confident in all its answers even when it clearly has no clue what it's doing.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#114
post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

I pay $0 for Google with no delays. I don’t understand why I’d want to pay for information that’s less reliable. (I’m being slightly dense, but not really)

I use it to shortcut the process of knowing what to search by generating code examples. Eg, I know what I want to do, I'm just not sure what methods $framework provides for doing so. I develop a sense of it's limitations as I go (eg, some frameworks it gives me methods that have been moved or renamed or deprecated, and so I check for those things with those frameworks).

When I was learning to program, and I'd get stuck to the point I needed to ask for help, it was a whole production. I'd need to really dot my 'i's and cross my 't's and come up with a terse question that demonstrated I had done my due diligence in trying to find the answer myself, pose the question in a concise and complete way, and do all of that in about 250 words because if it was too long it would get lost in the froth of the chatroom. (And it's probably apparent to you that brevity isn't my strongest quality (: ) And I'd still get my head bitten off for "wasting" the time of people who voluntarily sat in chatrooms answering questions all day. And I can understand why they felt that way (when I knew enough to answer questions I was just as much of a curmudgeon), but it was a pain in the ass, and I've met people who really struggled to learn to code partly because they couldn't interface with that system because they weren't willing to get their heads bitten off. So when they got stuck they spun their wheels until they gave up.

ChatGPT just answers the damn question. You don't have to wait until you're really and truly stuck and have exhausted all your available resources. It doesn't even have the capacity to feel you've wasted it's time.

I'm concerned about LLMs for a million different reasons, and I'm not sure how people who don't already know how to code can use them effectively when they make so much stuff up. But when I realized I could just ask it questions without spending half an hour just preparing to ask a question - that's when it clicked for me that this was something I might use regularly.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#115

People need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. They need to be told that ChatGPT is a search engine with advanced autocomplete. If they understood this, they'd probably find that it's actually useful for some things, and they can also av…

[flagged]

How so?

GPT LLM algorithms use a probabilistic language model to generate text. It is trained on a large corpus of text data, and it estimates the probability distribution of the next word given the previous words in the sequence.

The algorithm tokenizes the input into a sequence of tokens and then generates the next token(s) in the sequence based on the probabilities learned during training. These probabilities are based on the frequency and context of words in the training corpus. You can ask ChatGPT/etc yourself, and it'll tell you something like this.

This is not remotely like what human brains do. Your ideas cohere from network connections between the neurons in your brain, and then you come up with words to match your idea, not your previous words or the frequency that the words appear in your brain.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#116
post #102

No we don't. I treat ChatGPT like a person. People are flawed. People lie. People make shit up. People are crazy. Just apply regular people filters to ChatGPT output. If you blindly believe ChatGPT output and take it for perfect truth, someone likely already sold you a bridge.

> Just apply regular people filters to ChatGPT output.

This is technically correct, as LLMs are just aggregations of texts people produce.

It's also not quite right, as people have expectations/biases about how responses from computers might be different, and interaction with computers is missing many pieces of context that they might rely on to gauge probability of lying.

Telling them "ChatGPT can lie to you" is a succinct way of recalibrating expectations.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#117
post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

> The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plausible is to tell truth.

Yes.

> But guess what, that is pretty much how human liars work too!

There is some distinction between lying and bullshit.

https://en.wikipedia.org/wiki/On_Bullshit#Lying_and_bullshit

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#118
The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's.

Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#120

I apologize, but as an AI Language Model, I have no knowledge of other AI Language Models, but can assure you that AI Language Models do not lie. As an AI Language Model, I cannot participate in conversations related to libel or defamation of AI Language Model engineers, thus, I must stop this conversation. AI Language Models don't lie, humans lie.

However, thus therefore. Furthermore, lastly. Perchance.
Post reply on HN