Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

241–250 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#241

Post by zerocrates »

ZeroGravitas wrote: post #19

> This is a serious bug. Isn't describing this as a 'bug' rather than a misuse of a powerful text generation tool, playing into the framing that it's a truth telling robot brain? I saw a quote that said "it's a what text would likely come next machine", if it makes up a url pointing to a fake article with a plausible title by a person who works in that area, that's not a bug. That's it doing what it does, generating…

But look at the way these companies talk about these models they're deploying, how people talk about using them here on HN and elsewhere: so much of it is about just asking questions and getting correct answers. All the people talking about how it will destroy Google, how they're using it to learn or teach some topic, etc., Microsoft integrating one into a search engine, the gloss on them as "assistants."

The older, less-capable models were used for things more aligned to being just text-generation: "creativity" stuff. But the newer, bigger models and the human-feedback stuff to prod the models into following instructions and being more accurate have really pushed the conversation into this more "Star Trek computer" space.

I'm thinking of it kind of like the uncanny valley. On the lower end of the scale, you don't really trust the machine to do anything, but as it gets more and more capable, the places where it doesn't work well become more and more significant because you are trusting and relying on it.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#242

Post by 8note »

flanked-evergl wrote: post #196
zuminator wrote: post #187

Earlier quoted context omitted.

I don't think it intends to mislead because its answers are probabilistic. It's designed to distill a best guess out of data which is almost certain to be incomplete or conflicting. As human beings we do the same thing all the time. However we have real life experience of having our best guesses bump up against reality and lose. ChatGPT can't see reality. It only knows what "really being wrong" is to the extent that…

> Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth. ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth. I have never met a human who does that. If we reduce it down to how often most people are wrong vs how often ChatGPT is wrong, then sure, people may be on average wr…

I wouldn't be surprised if there's very successfulsoftware hustlers that do make up Google cloud apis. You may not know them but that doesn't mean they don't exist.

5 years down the line though, maybe those apis will exist because chatgpt is giving a summary of what apis Google cloud should have, and Google will listen

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#243

Post by 8note »

shagymoe wrote: post #152
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

>indistinguishable from what dumb people think smart people are like. Since about 2016, we have overwhelming evidence that even "smart people" are "fooled" by "confidently wrong".

Why 2016?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#245

Post by georgyo »

kerkeslager wrote: post #230
hotpathdev wrote: post #221

[flagged]

> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess. That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think…

I thought yams and sweet potatoes were the same thing for many years. I have told people that they were the same thing, because that is what I believed.

My thoughts were reinforced that in common American English we refer to both with the same name frequently.

However they are different, and I eventually was corrected and I was able to verify that correction.

The question is, when I legitimately believe my knowledge was correct, was I lying?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#247

Post by howenterprisey »

cultureswitch wrote: post #240
m463 wrote: post #212

Earlier quoted context omitted.

"Often wrong, never in doubt." An old saying, but frequently applies to the difficult people in your life. Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes yo…

What's terrible about some Wikipedia pages is when you personally don't know they are controversial. It should be vastly more prominent when there are tensions on an article and even admins shouldn't have the power to hide this.

Admins do not, in fact, have the power to hide tensions on an article, as even the act of deletion produces public logs. Certainly if you don't know to look for them, you won't see them, though.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#249

Post by joe_the_user »

hotpathdev wrote: post #221

[flagged]

These systems are not 'lying', they are simply 'wrong'

The meaningful distinction between lying and wrong, is that a liar doubles down on their position, making up more things and tailoring their statements to what their audience wants to hear. The article presents good arguments that LLMs actually do this. I've observed it myself interacting with ChatGPT. Sure, this seems weird but it's not. Read the article for why.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#250

Post by brookst »

kerkeslager wrote: post #230
hotpathdev wrote: post #221

[flagged]

> If you ask someone off the street a question, they won't know what you are talking about because their mind is going to be mid thought, but being friendly they will give you a low-effort first guess. That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie. Even if your intent is to deceive because you think…

> That's lying. If you don't know the answer, the correct answer is "I don't know", or to ask clarifying questions. To pretend that you know when you don't, is to lie.

You can use that definition, but it is not how the word "lie" is commonly used.

Mirriam-Webster defines lie as "to make an untrue statement with the intent to deceive"[0]

Cambridge dictionary defines lie as "to say or write something that is not true in order to deceive someone"[1]

Colllins dictionary defines lie as "A lie is something that someone says or writes which they know is untrue." and "If someone is lying, they are saying something which they know is not true."[2]

It's sad that people are so willing to conflate lying with being wrong to push a narrative.

0. https://www.merriam-webster.com/dictionary/lie

1. https://dictionary.cambridge.org/us/dictionary/english/lie

2. https://www.collinsdictionary.com/us/dictionary/english/lie

Post reply on HN