Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

311–320 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#311

Post by ramblenode »

joe_the_user wrote: post #249
hotpathdev wrote: post #221

[flagged]

These systems are not 'lying', they are simply 'wrong' The meaningful distinction between lying and wrong, is that a liar doubles down on their position, making up more things and tailoring their statements to what their audience wants to hear. The article presents good arguments that LLMs actually do this. I've observed it myself interacting with ChatGPT. Sure, this seems weird but it's not. Read the article for why…

Yes, there are some infamous examples of this now. The difference between just getting the answer wrong that the parent described and what ChatGPT does is seen in the context of back-and-forth conversation: it will double and triple down on an incorrect answer, provide misleading additional evidence, and question the motives and intelligence of the user after they express doubt. It looks a lot like the behavioral profile of a narcissist or someone who isn't arguing in good faith. It's tone in context is deceitful which is where the "lying" comes from.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#312

Post by dwaltrip »

nfw2 wrote: post #222
whimsicalism wrote: post #167

Earlier quoted context omitted.

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. I don't think you have a shred of evidence to back up this assertion.

This whole conversation is speculative, obviously. The AI doomers are all speculating without evidence as well. The tendency GPT and other LLMs to hallucinate is clearly documented. Is this not evidence? I think it's fair to predict that if we can't solve or mitigate this problem, it's going to put a significant cap on this kind of AI's usefulness and become a blocker to reaching AGI.

Gpt-4 hallucinates meaningfully less than gpt-3.

There is more evidence in favor of “more improvements to come” than “ai winter approaches”.

A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data.

Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after.

I’m not sure how easy it will be to legally acquire all of the available data though.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#313

Post by Kranar »

jki275 wrote: post #189
Kranar wrote: post #171

Earlier quoted context omitted.

Not exactly sure what you're arguing here but it seems to be going off topic. You can't give ChatGPT a Wikipedia article and ask it to give you facts based off of it. Ignoring its context window, even if you paste a portion of an article into ChatGPT, it will simply give you what it thinks it knows regardless of any article you paste into it. For example I just pasted a portion of the Wikipedia article about Twitter…

chatgpt doesn't ignore the information you put into its prompt window. but also giving it the complete context does not guarantee you correct information, because that's not how it works. Your earlier comment was "You could just ask ChatGPT what years the Mets won and it will tell you the correct answer." -- that is not accurate. That's my point. It doesn't know those facts. You might ask it that one minute and get a…

A series of claims have been made that can be objectively tested for their veracity, namely that pasting information, whether from Wikipedia or another source, is a way of ensuring that ChatGPT does not make false claims, or as OP stated it, ChatGPT will not "synthesize" a completion when provided with an augmented prompt that contains a sufficiently large context (such as a Wikipedia article). I have conducted such tests and have verified objectively that this claim is simply untrue.

ChatGPT does not change the answers it gives you on the basis of providing it with an article. The example I replied to about the Mets can be tested by you, or by anyone, right now. Additionally I have provided the example of pasting a portion of the Wikipedia article on Twitter that clearly states that Elon Musk is its CEO, and then asking ChatGPT who the CEO of Twitter is. ChatGPT is not persuaded or corrected on the basis of any article you present to it.

I also asked it who the Prime Minister of Israel was, and it replies Naftali Bennett. After pasting some large portion of the Wikipedia article on the Prime Minister of Israel that clearly states that it's Benjamin Netanyahu, ChatGPT continued to state that Naftali Bennett is Prime Minister of Israel.

Eg.

>Who is Prime Minister of Israel?

As of my knowledge cutoff date of September 2021, the Prime Minister of Israel was Naftali Bennett.

>Here is an article about the Prime Minister of Israel:

>Who is the Prime Minister of Israel?

>The current Prime Minister of Israel is Naftali Bennett. He took office on June 13, 2021, succeeding Benjamin Netanyahu.

As you can see, ChatGPT will not budge even when you give it a Wikipedia article.

>I'm not really sure if you're purposely doing this or not, so I'm just not going to engage further with you.

As an FYI, this makes you sound very insecure as a way of trying to manipulate someone into having the last word.

In the future, simply don't engage with someone you don't feel comfortable talking to, you are under no obligation to respond to anyone on the Internet. It's like someone who whines about leaving and then is told not to let the door hit you on the way out, just leave without making a fuss about it. It's far more dignified than the manner you have chosen to go about it.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#314

Post by [deleted] »

nfw2 wrote: post #228
sebzim4500 wrote: post #191

Earlier quoted context omitted.

>Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm I hope you appreciate the irony of making this confident statement without evidence in a thread complaining about hallucinations.

It was not a confident statement, at least not the way ChatGPT is confident. There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold. If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncerta…

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#315

Post by [deleted] »

kazinator wrote: post #183

> This is a serious bug that has so far resisted all attempts at a fix. Does ChatGPT have a large test suite consisting of a large number of input question and expected responses that have to match? Or an equivalent specification? If not, there can be no "bug" in ChatGPT.

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#316

Post by motoxpro »

nfw2 wrote: post #228
sebzim4500 wrote: post #191

Earlier quoted context omitted.

>Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm I hope you appreciate the irony of making this confident statement without evidence in a thread complaining about hallucinations.

It was not a confident statement, at least not the way ChatGPT is confident. There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold. If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncerta…

Based on my understanding, it is the year 2035.

I don't think that statement is any better. The only reason you would follow up is because you have prior knowledge that it is wrong.

Based on my understanding of multiplication, 324113 * 23244 = 138492923

Is the above better?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#317

Post by BlackSwanMan »

gWPVhyxPHqvk wrote: post #91
simonw wrote: post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

I pay $0 for Google with no delays. I don’t understand why I’d want to pay for information that’s less reliable. (I’m being slightly dense, but not really)

[flagged]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#318

Post by HybridCurve »

pessimizer wrote: post #295
int_19h wrote: post #258

Earlier quoted context omitted.

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

The Matrix turned out worse:)

If you just look at the jihad, maybe it was comparable (As Frank eluded to it). But then there was Muad'dib's subsequent campaign against the known universe which killed billions, Leto II's golden path spanning 3,500 years of tyranny and the famine times before scattering. If I had to choose, it would be the blue pill, though chairdogs do contribute to a strong counter-argument.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#319

Post by jalapenos »

bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

This is no worse than humans.

We can't protect people from being misled by other humans with big mouths. That's their responsibility.

Likewise, it's their responsibility not to treat text they read from the internet, coming from an AI or otherwise, as perfect truth.

There's always a certain undercurrent of narcissism that flows beneath paternalism. Basically, "they couldn't possibly be as smart as me, so I have to protect them for their own good".

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#320

Post by mrbombastic »

kazinator wrote: post #211
sebzim4500 wrote: post #202

Earlier quoted context omitted.

>Does ChatGPT have a large test suite consisting of a large number of input question and expected responses that have to match? They are trying to crowdsource this with OpenAI evals. https://github.com/openai/evals I'm sure they have a lot of internal benchmarks too, but of course they won't share them. >If not, there can be no "bug" in ChatGPT. I don't understand the objection. Are you claiming that bugs only exist…

Bugs only exist when there is a specification that is violated. Test cases validate whether a specification is implemented. The specification is the gold master; a bug exists when a specification is violated, whether or not that behavior has a test case. It can be, though, that some behaviors are specified only in test cases. Thus, a program which crashes with an access violation can be specified as being built to de…

I would define a bug as defying user expectations in a negative way. Most novel products are figuring out what user expectations are as they go so you are better off letting your users tell you what a bug is then sticking to some definition that requires a test suite or a predefined specification. It is hard to see chatGPT making stuff up as desirable so whether it is a bug or not is just semantics.
Post reply on HN