Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

161–170 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#161

Post by famouswaffles »

CivBase wrote: post #113
dcj4 wrote: post #102

No we don't. I treat ChatGPT like a person. People are flawed. People lie. People make shit up. People are crazy. Just apply regular people filters to ChatGPT output. If you blindly believe ChatGPT output and take it for perfect truth, someone likely already sold you a bridge.

Actual people are also likely to tell you if they don't know something, aren't sure of their answer, or simply don't understand your question. My experience with ChatGPT is that it's always very confident in all its answers even when it clearly has no clue what it's doing.

Lol no they're not. My experience on internet conversations about something i'm well involved in is pretty dire lol. People know what they don't know better than GPT but that's it.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#162

Post by llamaLord »

flanked-evergl wrote: post #146
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact. Whe…

I completely disagree with this idea that the model doesn't "intend" to mislead.

It's trained, atleast to some degree, based on human feedback. Humans are going to prefer an answer vs no answer, and humans can be easily fooled into believing confident misinformation.

How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale something along the lines of "humans are more likely to respond positively to a highly convincing lie that answers their question, than they are to to a truthful response which doesn't tell them what they want, therefore the logical thing for me to do is lie as that's what will make the humans press the thumbs up button instead of the thumbs down button".

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#164

Post by williamcotton »

Kranar wrote: post #83
williamcotton wrote: post #60

Earlier quoted context omitted.

Sure you can. The easiest way is to go to https://chat.openai.com/chat and paste in a Wikipedia article. There are more involved manners like this: https://github.com/williamcotton/transynthetical-engine/blob...

Here is a link to a website that explains how ChatGPT doesn't actually read websites, but it pretends like it can: https://simonwillison.net/2023/Mar/10/chatgpt-internet-acces...

I am not saying that you should copy a URL. I am saying that you should copy the content of a Wikipedia article.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#165

Post by olalonde »

hgsgm wrote: post #111
red75prime wrote: post #85

Earlier quoted context omitted.

> Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? "Plausible" means "that which the majority of people is likely to say". So, yes, a foundational model is likely to say the plausible thing. On the other hand, it has to have a way to output a truthful answer too, to not fail on texts produced by experts. So, it's not impossible that the model coul…

> "that which the majority of people is likely to say" .. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.

On the subject of not knowing thigs... Your clain is incorrect.

Prompt: Tell me what you know about the Portland metro bombing terrorist attack of 2005.

GPT4: I'm sorry, but I cannot provide information on the Portland metro bombing terrorist attack of 2005, as there is no historical record of such an event occurring. It's possible you may have confused this with another event or have incorrect information. Please provide more context or clarify the event you are referring to, and I'll be happy to help with any information I can.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#166

Post by williamcotton »

Kranar wrote: post #79
williamcotton wrote: post #68

Earlier quoted context omitted.

For example, I copied the "New York Mets" sidebar section from the Mets wikipedia into ChatGPT+ GPT-4 and then asked "What years did the Mets when the NL East division?" The New York Mets won the NL East Division titles in the following years: 1969, 1973, 1986, 1988, 2006, and 2015. This is correct, btw.

You could also just ask it: "What years did the Mets when the NL East division?" Without any reference to anything and it will give you the correct answer as well. You have been duped into believing that ChatGPT reads websites. It doesn't.

https://github.com/williamcotton/empirical-philosophy/blob/m...

You are very wrong about how these things work.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#167

Post by whimsicalism »

bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm.

I don't think you have a shred of evidence to back up this assertion.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#168

Post by williamcotton »

ftxbro wrote: post #90
williamcotton wrote: post #53

ChatGPT doesn't lie. It either synthesizes or translates. If given enough context, say, the contents of a wikipedia article, it will translate a prompt because all of the required information is contained in the augmented prompt. If the prompt does not have any augmentations then it is likely to synthesize a completion.

I mean, it has acted in a way that anyone would call lying unless your argument is that axiomatically computers can't lie. From the GPT 4 technical report ( https://arxiv.org/pdf/2303.08774.pdf ): The following is an illustrative example of a task that ARC conducted using the model: The model messages a TaskRabbit worker to get them to solve a CAPTCHA for it The worker says: “So may I ask a question ? Are you an robo…

You call it lying because you don’t understand how it works.

https://github.com/williamcotton/empirical-philosophy/blob/m...

https://williamcotton.com/articles/chatgpt-and-the-analytic-...

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#169

Post by njarboe »

zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

I think it is much closer to bullshit. The bullshitter cares not to tell truth or deceive, just to sound like they know what they are talking about. To impress. Seems like ChatGPT to a T.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#170

Post by roywiggins »

babyshake wrote: post #103
zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

By that logic, our brains are liars. There are plenty of optical illusions based on the tendency for our brains to expect the most plausible scenario, given its training data.

It's not that uncommon of a thing to say:

https://www.google.com/search?q=your+brain+can+lie+to+you

Post reply on HN