Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

401–410 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#401

Post by wohlergehen »

oneshtein wrote: post #389
lamontcg wrote: post #339

Earlier quoted context omitted.

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

[flagged]

To clear this up, because open Holocaust denial is never OK:

Thousands of jews were indeed murdered at Babi Yar, you can find their names online [0]. Paul Blobel [1] was executed at Nuremberg for this. No serious source disputes the killing of jews at Babi Yar, be it the US, Germany (where this matter went to court), Israel, Ukraine, or Russia.

0: https://yvng.yadvashem.org/index.html?language=en&s_id=&s_la... 1: https://en.wikipedia.org/wiki/Paul_Blobel

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#402

Post by f6v »

DennisAleynikov wrote: post #393
f6v wrote: post #392

Earlier quoted context omitted.

I thought ChatGPT posts were the most annoying thing on the internet right now. Then there this. Does your answer have anything to do with LLMs or AI? All I see is family anecdotes.

It's an example of the dangers gpt poses today. Not sure where you got the impression these are family anecdotes

I don’t think the post above makes a good point because there’s no links to evidence. They talk about “declassified archives”, “US spy plane”. Like, what is that? Saying “Russian lies” 10 times in every paragraph doesn’t turn a rant into compelling argument.

I mean, I get it, the poster above is Ukrainian and wants to turn every discussion in a certain direction. It’s just there’re many places on the internet they can vent.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#403

Post by [deleted] »

oneshtein wrote: post #389
lamontcg wrote: post #339

Earlier quoted context omitted.

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

[flagged]

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#404

Post by westcoast49 »

flanked-evergl wrote: post #146
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact. Whe…

I think of ChatGPT as a natural language query engine for unstructured data. It knows about the relationships that are described in it's natural language input training data set, and it allows the same relationships to be queried from a wide range of different angles using queries that are also formulated in natural language.

When it hallucinates, I find that it's usually because I'm asking it about a fringe topic where it's training data set is more sparse, or where the logical connections are deeper than it's currently able to "see", a sort of a horizon effect.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#405

Post by [deleted] »

Kranar wrote: post #313
jki275 wrote: post #189

Earlier quoted context omitted.

chatgpt doesn't ignore the information you put into its prompt window. but also giving it the complete context does not guarantee you correct information, because that's not how it works. Your earlier comment was "You could just ask ChatGPT what years the Mets won and it will tell you the correct answer." -- that is not accurate. That's my point. It doesn't know those facts. You might ask it that one minute and get a…

A series of claims have been made that can be objectively tested for their veracity, namely that pasting information, whether from Wikipedia or another source, is a way of ensuring that ChatGPT does not make false claims, or as OP stated it, ChatGPT will not "synthesize" a completion when provided with an augmented prompt that contains a sufficiently large context (such as a Wikipedia article). I have conducted such…

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#406

Post by pmoriarty »

f6v wrote: post #394
dwaltrip wrote: post #312

Earlier quoted context omitted.

Gpt-4 hallucinates meaningfully less than gpt-3. There is more evidence in favor of “more improvements to come” than “ai winter approaches”. A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. Based on what I’ve read and roughly speculating, it looks there is easily enough existing data for gpt-5 and probably a few more versions after. I’m not sure…

> A lot smart people seem to be saying that the existing approaches have room to improve simply by training with more data. There’re also many “smart people” saying that simply having more data doesn’t turn a machine that can’t reason into one that can.

"simply having more data doesn’t turn a machine that can’t reason into one that can"

But that's exactly what's been happening. LLM's are a triumph of "big data" used by dumb algorithms to create a reasoning machine.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#407

Post by disgruntledphd2 »

int_19h wrote: post #366
ratherbefuddled wrote: post #362

Earlier quoted context omitted.

They were fleeing AI.

I don't recall anything indicating that in Frank Herbert's original series. And Brian Herbert's "Omnius" nonsense seems to have very little to do with what Frank actually had in mind wrt Butlerian Jihad etc, so I don't consider it canonical.

Book 4, Sonia's vision with Leto strongly implies that the problem is the machines.

Agreed that the endings done by others were incredibly bad.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#408

Post by pmoriarty »

guax wrote: post #350
stephendause wrote: post #322

Earlier quoted context omitted.

I believe the relevant assertion is > Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence To me that is an open question. OpenAI has not revealed a lot of the relevant stats behind GPT-4. I also haven't seen anything about their future LLM research (though I haven't looked very hard), but I think the above assertion remains to be seen.

It seems like people are focusing on training with more data and working the data. A human does not need to learn all the contents of the internet to learn skepticism and logic. A smart Ai should probably not either, that would point me to a different approach and those are not predictable. I have no doubt we can make gpt better and more useful. I do have some doubts we can make it trustworthy enough for most of its…

"A human does not need to learn all the contents of the internet to learn skepticism and logic."

Some humans never learn this, and those that do could take years or even decades of training. AI's are learning some things much faster, though since their "brains" are not like those of humans there's no reason to expect them to learn in the same way.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#409

Post by antupis »

wordpad25 wrote: post #344
antupis wrote: post #326

Earlier quoted context omitted.

I think good annotation is much harder than obtaining data. But now Reddit, Twitter and Quora will realise what kind goldmine their data is they might close easy access to it.

It’s too easy to scrape them, but even that is already done by many intermediary companies all too happy to resell that data.

Yup but you still need up to date data and now companies are realising what kinda gold mine random forum posts about nightlife in Glasgow are they will start monetising it and probably try at least kill middle market.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#410

Post by pmoriarty »

at_a_remove wrote: post #292
VectorLock wrote: post #274

Earlier quoted context omitted.

Better than getting extincted, I suppose.

Oh man, I am going to be a huge nerd now. Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind , and were eventually enslaved by those who controlled the AI. Not much more was said about it. One could have, yes, militant robots, Exterminate! Exterminate! out of it, or you could posit a more Huxley-like dystopia, one of convenience. Control the AI, cont…

"when a man ceases to choose, he ceases to be a man"

Whether humans have free will is an open question.

Kind of risky to stake the definition of humanity on it.

Post reply on HN