We need to tell people ChatGPT will lie to them, not debate linguistics • Hacker News Board

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#331

Post by kazinator » Sat, Apr 08, 2023, 3:46 AM UTC

cultureswitch wrote: post #256

kazinator wrote: post #184

Earlier quoted context omitted.

ChatGPT is always making things up. It is correct when the things it makes up come from fragments of training data which happened to be correct, and didn't get mangled in the transformation. Just like when a diffusion model is "correct" when it creates a correct shadow or perspective, and "incorrect" when not. But both images are made up. It's the same thing with statements. A statement can correspond to something in…

Something can be truthfully describing the world by chance. The statement would still be true. In fact every statement we make about the world we live in is like this.

The original assertion of such a statement has no basis, and continues not to even when the statement is independently discovered to be true.

No, not every statement we make about the world is a baseless assertion that is true by chance.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#332

Post by kazinator » Sat, Apr 08, 2023, 3:51 AM UTC

mrbombastic wrote: post #320

kazinator wrote: post #211

Earlier quoted context omitted.

Bugs only exist when there is a specification that is violated. Test cases validate whether a specification is implemented. The specification is the gold master; a bug exists when a specification is violated, whether or not that behavior has a test case. It can be, though, that some behaviors are specified only in test cases. Thus, a program which crashes with an access violation can be specified as being built to de…

I would define a bug as defying user expectations in a negative way. Most novel products are figuring out what user expectations are as they go so you are better off letting your users tell you what a bug is then sticking to some definition that requires a test suite or a predefined specification. It is hard to see chatGPT making stuff up as desirable so whether it is a bug or not is just semantics.

Defied user expectations are a bug when the purveyor of the system becomes aware of those expectations and subsequently adopts them as a requirement.

If the expectations are rejected, then the situation is resolved as a non-bug.

An expectations bug can go away if the user's expectations are "managed" in a direction away from it.

Expectations have to be formalized into something that is testable, if we are to actually implement a bug fix and close the bug.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#333

Post by sltkr » Sat, Apr 08, 2023, 3:53 AM UTC

int_19h wrote: post #282

sltkr wrote: post #177

Earlier quoted context omitted.

Just because the author predicted the objection doesn't make it invalid. It's a popular tactic to describe concepts with terms that have a strong moral connotation (“meat is murder”, “software piracy is theft”, “ChatGPT is a liar”) It can be a powerful way to frame an issue. At the same time, and for the same reason, you can hardly expect people on the other side of the issue to accept this framing as accurate. And o…

What difference does the exact phrasing make in case of "ChatGPT lies"? I don't think we have to be concerned about its reputation as a person, so the important part is making sure that people don't hurt themselves or others. "Lie" is a simple word, easy to understand, and the consequences of understanding it in the most direct and literal way are exactly as desired. Whereas if you go waxing philosophically about lac…

I understand that if you summarize a position you necessarily lose some accuracy, but if the summary is inaccurate and defamatory, that's not fair or helpful to the people involved. For example, “abortion is murder!” doesn't defame any particular person but it still casts people who had an abortion and doctors who perform them in a bad light. Similarly, I think “ChatGPT lies!” is unfair to at least the OpenAI developers, who are very open about the limitations of the tool they created.

The pearl-clutching around ChatGPT reminds me of the concerns around Wikipedia when it was new: teachers told students they couldn't trust it because anyone could edit the articles. And indeed, Wikipedia has vandals and biased editors, so you should take it with a grain of salt and never rely on Wikipedia alone as a fundamental source of truth. But is it fair to summarize these valid concerns as “Wikipedia lies!”? Would that have been fair to the Wikipedia developers and contributors, most of whom act in good faith? Would it be helpful to Wikipedia readers? I think the answers are no.

Like Wikipedia, to make effective use of (Chat)GPT you have to understand a bit about how it works, which also informs you of its limitations. It's a language model first and foremost, so it is more concerned with providing plausible-sounding answers than checking facts. If you are concerned about people being too trusting of ChatGPT, educate those people about the limitations of language models. I don't think telling people “ChatGPT lies” is what anyone should be

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#334

Post by dougb5 » Sat, Apr 08, 2023, 4:07 AM UTC

I run a word-search API and I now regularly get emails from frustrated users who complain that it doesn't work the way ChatGPT tells them it works. For example, today someone asked me why a certain request fails, and it turned out to be a fake but plausible URL to my API that ChatGPT had invented in response to "Does the Datamuse API work in French?" (It does not, and there's no indication that it does in the documentation.)

Adding up all the cases like mine out there -- the scale of the misunderstanding caused, and amount time wasted, must be colossal. What bothers me is that not only has OpenAI extracted and re-sold all of the value of the Web without providing any source attribution in return; but they do so lyingly a good chunk of the time, with someone else bearing the costs.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#335

Post by khochesh_kushat » Sat, Apr 08, 2023, 4:09 AM UTC

barbazoo wrote: post #327

HybridCurve wrote: post #318

Earlier quoted context omitted.

If you just look at the jihad, maybe it was comparable (As Frank eluded to it). But then there was Muad'dib's subsequent campaign against the known universe which killed billions, Leto II's golden path spanning 3,500 years of tyranny and the famine times before scattering. If I had to choose, it would be the blue pill, though chairdogs do contribute to a strong counter-argument.

Let me guess, Star Trek?

Dune

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#336

Post by exabrial » Sat, Apr 08, 2023, 4:19 AM UTC

“Chat GPT is just an elaborate and sophisticated bullshit generator” is a little more direct and a lot more accurate. It’s how I explained it to my non technical relatives.

If we could stop calling these dammed things AIs that’d be helpful too.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#337

Post by hunter2_ » Sat, Apr 08, 2023, 4:40 AM UTC

int_19h wrote: post #259

hunter2_ wrote: post #219

Earlier quoted context omitted.

I think this is partly explained by most of the marketing and news essentially saying "ChatGPT is an AI" instead of "ChatGPT is an LLM." If you asked me what AI is, I'd say it means getting a computer to emulate human intelligence; if you asked me what an LLM is, I'd say it means getting a computer to emulate human language. The word "language" does not imply truthiness anywhere near to the extent that the word "inte…

You could reasonably describe it as "human language emulator" back when people were using GPT-2 and the likes to compose text. But what we have today doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform, and then carries them out. Granted, the only thing it can really "do" is produce text, but that already covers quite a lot of tasks - and the…

Interesting perspective. I'm still learning about what it really is, and I'm having trouble marrying the thoughts of a parent commenter with yours:

> ... does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason ...

versus

> ... doesn't just emulate human language - it accepts tasks in that language, including such tasks that require reasoning to perform ...

Maybe a third party can jump in here: does ChatGPT use reasoning beyond the domain of language, or not?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#338

Post by bpodgursky » Sat, Apr 08, 2023, 4:43 AM UTC

antupis wrote: post #326

dwaltrip wrote: post #312

Earlier quoted context omitted.

Gpt-4 hallucinates meaningfully less than gpt-3. There is more evidence in favor of “more improvements to come” than “ai winter approaches”. A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after. I’m not sure…

I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.

None of the GPT models rely on annotated or classified data. It's unsupervised.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#339

Post by lamontcg » Sat, Apr 08, 2023, 4:46 AM UTC

bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference.

This may be the best point that you've made.

We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#340

Post by perryizgr8 » Sat, Apr 08, 2023, 4:54 AM UTC

I've started using Bing GPT more and more over the version available on OpenAI's site. I've noticed that it doesn't lie very much at all. I haven't experienced a clear cut lie yet. In fact it is much more unsure about things that seem to have conflicting information available about them on the web. And it will inform you of that, provide sources from both points of view and let you make your decision. I haven't experienced any hallucination at all.

Of course, it seems to have been artificially limited a lot. Even the slightest hint of controversy will cause it to delete its reply in front of your eyes as your reading it, which is enormously annoying. It also has a hard limit on the number of responses after which it will end the chat. I really hope this trend of stunting AI in order to satisfy some arbitrary standard of political correctness is short-lived, and we see fully functional AIs without such limitations.