Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

371–380 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#371

Post by shjake »

8note wrote: post #242
flanked-evergl wrote: post #196

Earlier quoted context omitted.

> Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth. ChatGPT makes up non-existing APIs for Google cloud and Go out of whole cloth. I have never met a human who does that. If we reduce it down to how often most people are wrong vs how often ChatGPT is wrong, then sure, people may be on average wr…

I wouldn't be surprised if there's very successfulsoftware hustlers that do make up Google cloud apis. You may not know them but that doesn't mean they don't exist. 5 years down the line though, maybe those apis will exist because chatgpt is giving a summary of what apis Google cloud should have, and Google will listen

> Google cloud should have, and Google will listen

Why? Isn’t what ChatGPT suggesting just random, essentially incoherent noise in those cases? Are there any examples at all of it coming up with something actually useful (and something humans haven’t thought of)?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#372

Post by naavis »

BlackSwanMan wrote: post #317
gWPVhyxPHqvk wrote: post #91

Earlier quoted context omitted.

I pay $0 for Google with no delays. I don’t understand why I’d want to pay for information that’s less reliable. (I’m being slightly dense, but not really)

[flagged]

It's a considerable sum for many people in many countries. Free on the other hand is equally free everywhere.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#373

Post by shjake »

motoxpro wrote: post #316
nfw2 wrote: post #228

Earlier quoted context omitted.

It was not a confident statement, at least not the way ChatGPT is confident. There are multiple ways the commenter conditioned their statement: > Based on my understanding > it is PROBABLY very close The author makes it clear that there is a uncertainty and that if their understanding is wrong, the prediction will not hold. If ChatGPT did any of the things the commenter did, the problem wouldn't exist. Making uncerta…

Based on my understanding, it is the year 2035. I don't think that statement is any better. The only reason you would follow up is because you have prior knowledge that it is wrong. Based on my understanding of multiplication, 324113 * 23244 = 138492923 Is the above better?

> Is the above better?

No. It is tangential though.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#374

Post by shjake »

furyofantares wrote: post #367
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence Is this performance art? I mean it could end up right but I think you basically just made it up and then stated it confidently.

They didn’t state it particularly confidently though

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#375

Post by kurthr »

lamontcg wrote: post #339
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

The problem is that it will be cheaper at scale. That will allow the BS to be even more targeted growing the population of acolytes of ignorance. I don't know how much bigger it is, but it seems like we're around 20% today. If it gets to a majority there could be real problems.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#376

Post by shjake »

olalonde wrote: post #165
hgsgm wrote: post #111

Earlier quoted context omitted.

> "that which the majority of people is likely to say" .. and saying "I don't know" is forbidden by the programmers. That is a huge part of the problem.

On the subject of not knowing thigs... Your clain is incorrect. Prompt: Tell me what you know about the Portland metro bombing terrorist attack of 2005. GPT4: I'm sorry, but I cannot provide information on the Portland metro bombing terrorist attack of 2005, as there is no historical record of such an event occurring. It's possible you may have confused this with another event or have incorrect information. Please pr…

There is a big difference between what they are talking about (I believe) and asking it factual questions withal straightforward answers.

Try discussing something a bit more complex and it’s very easy to get ChatGPT to start contradicting itself all the time (which is more or less the same thing).

From my experience ChatGPT is prone to come up with lists of “probable” answer when it’s not feeling very confident. Then it just starts throwing them out in a confident tone. When challenged it switches to a complete different answer and then “pretends” that I misunderstood it previously.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#377

Post by Nevermark »

dragonwriter wrote: post #370
dwaltrip wrote: post #312

Earlier quoted context omitted.

Gpt-4 hallucinates meaningfully less than gpt-3. There is more evidence in favor of “more improvements to come” than “ai winter approaches”. A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after. I’m not sure…

> There is more evidence in favor of “more improvements to come” than “ai winter approaches”. Every “AI Winter” has seen continuing improvements, its just that the pace and impact of the improvements fell short of the accelerating anticipation driven by the preceding AI hype cycle and the demands of those investing capital into AI. If there is another AI winter, it will be the same. “More improvements to come” is not…

> “More improvements to come” is not the opposite of “AI winter approaches”; its not as if expert systems stopped advancing after the second AI winter (or neural nets after the first.)

True.

And its also true that even in that nuanced form, a slowdown not a stop, there is no evidence of an imminent AI winter.

Major progress in model behavior is happening on the scale of months. Happening often based on relatively obvious or routine improvements, such as more data, more kinds of data, more computation, expanding the size of prompt windows, tuning of model architectures, etc.

And not only businesses, but consumers, are getting access to these tools in ways that are increasing demand for more improvements. And for these tools to be more available to all kinds of areas of work, creativity, and leisure.

So the resources that are being brought to bear on routine and more fundamental improvements are growing at an exceptional pace.

No sign of a saturation of progress in sight. Quite the reverse.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#378

Post by shjake »

DebtDeflation wrote: post #139
zmmmmm wrote: post #52

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained. The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plaus…

IMO, it just requires the same level of skepticism as a Google search. Just because you enter a query into the search bar and Google returns a list of links and you click one of those links and it contains content that makes a claim, doesn't mean that claim is correct. After all, this is largely what GPT has been trained on.

That would be much easier if it provided sources and some context for it’s answers

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#379

Post by shjake »

kenjackson wrote: post #208
njarboe wrote: post #169

Earlier quoted context omitted.

I think it is much closer to bullshit. The bullshitter cares not to tell truth or deceive, just to sound like they know what they are talking about. To impress. Seems like ChatGPT to a T.

ChatGPT no more cares to impress or persuade you as it cares to tell the truth or lie. It will say what its training maps to its model the best. No more, no less. If you believe it or not -- ChatGPT doesn't care -- except to the extent that you report the answer and they tweak its training/model in the future.

> ChatGPT doesn't care -- except to the extent that you report the answer and they tweak its training/model in the future.

So it does care (in a certainly non human way though)? If it’s training predisposes it to provide answers in a certain way or using a certain style that’s hardly distinguishable from ‘caring’, when that word is used to describe the behavior of a computer program.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#380

Post by nathias »

bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> unless

I don't think there is a possible unless, they are told repeatedly not to trust politicians, yet here we are ...

Post reply on HN