Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

251–260 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#251

Post by Eisenstein »

kerkeslager wrote: post #225
Eisenstein wrote: post #126

Earlier quoted context omitted.

I think it is better to assume that normal people can be deceived by confident language than to assume this is a problem with 'dumb people'.

I'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people". If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.

Very good point. Everyone should do this.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#252

Post by 8note »

zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

I'd consider it definitional to lying that the agent knows the truth and purposefully obscures it.

Without knowing what the truth is, I don't think LLMs are capable of lying

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#255

Post by 8note »

kenjackson wrote: post #208
njarboe wrote: post #169

Earlier quoted context omitted.

I think it is much closer to bullshit. The bullshitter cares not to tell truth or deceive, just to sound like they know what they are talking about. To impress. Seems like ChatGPT to a T.

ChatGPT no more cares to impress or persuade you as it cares to tell the truth or lie. It will say what its training maps to its model the best. No more, no less. If you believe it or not -- ChatGPT doesn't care -- except to the extent that you report the answer and they tweak its training/model in the future.

It definitely cares about impressing and persuasion. Those are things that will result in higher ratings from the user

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#256

Post by cultureswitch »

kazinator wrote: post #184
dorkwood wrote: post #181

I’m surprised how many users of ChatGPT don’t realize how often it makes things up. I had a conversation with an Uber driver the other day who said he used ChatGPT all the time. At one point I mentioned its tendency to make stuff up, and he didn’t know what I was talking about. I can think of at least two other non-technical people I’ve spoken with who had the same reaction.

ChatGPT is always making things up. It is correct when the things it makes up come from fragments of training data which happened to be correct, and didn't get mangled in the transformation. Just like when a diffusion model is "correct" when it creates a correct shadow or perspective, and "incorrect" when not. But both images are made up. It's the same thing with statements. A statement can correspond to something in…

Something can be truthfully describing the world by chance. The statement would still be true.

In fact every statement we make about the world we live in is like this.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#257

Post by kerkeslager »

georgyo wrote: post #245
kerkeslager wrote: post #230

Earlier quoted context omitted.

> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess. That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think…

I thought yams and sweet potatoes were the same thing for many years. I have told people that they were the same thing, because that is what I believed. My thoughts were reinforced that in common American English we refer to both with the same name frequently. However they are different, and I eventually was corrected and I was able to verify that correction. The question is, when I legitimately believe my knowledge…

I would say no. If you believe what you said, you're not lying.

There are grey areas here, like levels of confidence in our knowledge. A rational person believes very few things with 100% certainty, but it's not reasonable to be paralyzed by even the slightest amount of doubt and never say anything. I think a reasonable approach is to state things with some indication of your level of confidence, like:

1. Obviously sweet potatoes are yams. 2. Sweet potatoes are yams. 3. I am pretty sure sweet potatoes are yams. 4. I think sweet potatoes are yams. 5. I think sweet potatoes are yams, but I'm not sure. 6. I think sweet potatoes are yams, but I don't really know why I think that. 7. Sweet potatoes might be yams.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#258

Post by int_19h »

asynchronous wrote: post #220
rakah wrote: post #199

Earlier quoted context omitted.

Butlerian Jihad.

“It is illegal to make a machine in the likeness of a human mind.”

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#259

Post by int_19h »

hunter2_ wrote: post #219
danpat wrote: post #154

Earlier quoted context omitted.

> but why would you expect that from it? If you understand what it is doing, then you don't. But the layman will just see a computer that talks in language they understand, and will infer intent and sentience are behind that, because that's the only analog they have for a thing that can talk back to them with words that appear to make sense at the complexity level that ChatGPT is achieving. Most humans do not have su…

I think this is partly explained by most of the marketing and news essentially saying "ChatGPT is an AI" instead of "ChatGPT is an LLM." If you asked me what AI is, I'd say it means getting a computer to emulate human intelligence; if you asked me what an LLM is, I'd say it means getting a computer to emulate human language. The word "language" does not imply truthiness anywhere near to the extent that the word "inte…

You could reasonably describe it as "human language emulator" back when people were using GPT-2 and the likes to compose text. But what we have today doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform, and then carries them out. Granted, the only thing it can really "do" is produce text, but that already covers quite a lot of tasks - and then of course text can be an API call.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#260

Post by kerkeslager »

brookst wrote: post #250
kerkeslager wrote: post #230

Earlier quoted context omitted.

> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess. That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think…

> That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. You can use that definition, but it is not how the word "lie" is commonly used. Mirriam-Webster defines lie as "to make an untrue statement with the intent to deceive"[0] Cambridge dictionary defines lie as "to say or write something that is not true i…

> Mirriam-Webster defines lie as "to make an untrue statement with the intent to deceive"[0]

If I don't believe I know something, and I pretend to know it, the untrue statement is implicit: I'm stating that I have information that I don't have.

For example, I have heard that there's a town called "Springfield" in every state, but I've never done any research to verify that. If I confidently say, "There's a town called 'Springfield' in every state," that's a lie, even if there IS a town called 'Springfield' in every state. The deception isn't the geographical information, it's the implicit statement that I know this geographical information when I don't.

> It's sad that people are so willing to conflate lying with being wrong to push a narrative.

I'm not conflating those things. I think you'll find that what I'm describing fits the definitions you've given, and I think I've made it very clear that I do understand that intent is a necessary component of lying.

Post reply on HN