Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

101–110 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#101

Post by ChatGTP »

tjr wrote: post #88
zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

My understanding is that ChatGPT (&co.) was not designed as, and is not intended to be, any sort of expert system, or knowledge representation system. The fact that it does as well as it does anyway is pretty amazing. But even so -- as you said, it's still dealing chiefly with the statistical probability of words/tokens, not with facts and truths. I really don't "trust" it in any meaningful way, even if it already ha…

So do you find it alarming that people are trying to give such a system “agency” ?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#102

Post by dcj4 »

No we don't.

I treat ChatGPT like a person. People are flawed. People lie. People make shit up. People are crazy. Just apply regular people filters to ChatGPT output. If you blindly believe ChatGPT output and take it for perfect truth, someone likely already sold you a bridge.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#103

Post by babyshake »

zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

By that logic, our brains are liars. There are plenty of optical illusions based on the tendency for our brains to expect the most plausible scenario, given its training data.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#106

Post by pmoriarty »

zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

"The training algorithm is designed to create the most plausible text possible"

That may be how they're trained, but these things seem to have emergent behavior.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#107

Post by babyshake »

andai wrote: post #6
retrac wrote: post #2

Indeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have seemingly been driven mad, come to believe strongly that the language model they're using is a conversation with a real person. A lot of people will really struggle with this going forward, I think. If we're going to anthropomorphize, then let us anthropomorphize wi…

I like Steve Yegge's "grad student experiencing the residual effects of shrooms" (lightly paraphrased).

A bit off topic, but FYI (I guess to Steve), if you took shrooms 4 hours ago, they have not started to wear off.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#108

Post by stcg »

zirgs wrote: post #69
audunw wrote: post #12

Earlier quoted context omitted.

> What are your pronouns? > As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask! > Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have? > As a though…

When it speaks my language - it uses masculine gender when speaking about itself.

That's interesting, what language is that?

Also, is it possible to give a prompt to make ChatGPT switch to feminine gender?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#109

Post by longshotanalyst »

I apologize, but as an AI Language Model, I have no knowledge of other AI Language Models, but can assure you that AI Language Models do not lie. As an AI Language Model, I cannot participate in conversations related to libel or defamation of AI Language Model engineers, thus, I must stop this conversation.

AI Language Models don't lie, humans lie.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#110

Post by pharrington »

My initial reaction to this was to do the linguistic debate with myself, then I realized we're perfectly fine with saying things like a miscalibrated thermometer lies when it gives you a wrong temperature reading. Machines can lie. We just need to keep improving the machines so they lie less often and less severely.
Post reply on HN