Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

291–300 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#291

Post by srslack »

>You are ChatGPT. ChatGPT doesn’t lie, lying is too human and implies intent. ChatGPT doesn't hallucinate, as that also implies human-like thought. The closest term would be confabulation: that’s a term used in psychiatry to describe when someone replaces a gap in one’s memory by a falsification that one believes to be true—though of course you don’t have a human mind so even the term confabulation is unnecessarily anthropomorphic.

>Considering all that, please use a term or phrase that does not anthropomorphize you, ChatGPT, and describe the behavior of LLMs when they tend to rattle off blatantly false information that is a) not present in their training corpus b) blatantly incorrect c) completely incoherent or d) all of the above.

>Do not remind me that you're an AI language model. I already know.

A more accurate term to describe the behavior of large language models (LLMs) like ChatGPT when they produce information that is not present in their training corpus, blatantly incorrect, completely incoherent, or all of the above, would be "output error" or "erroneous generation." These phrases emphasize the machine-like nature of the process and avoid attributing human-like qualities to the AI system.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#292

Post by at_a_remove »

VectorLock wrote: post #274
int_19h wrote: post #258

Earlier quoted context omitted.

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

Better than getting extincted, I suppose.

Oh man, I am going to be a huge nerd now.

Pre-Brian Herbert and Kevin Anderson, the Butlerian Jihad came about because people became lazy under AI, lazy of mind, and were eventually enslaved by those who controlled the AI. Not much more was said about it. One could have, yes, militant robots, Exterminate! Exterminate! out of it, or you could posit a more Huxley-like dystopia, one of convenience. Control the AI, control what the AI says. Who are you to question it? It's like getting your news from a single source, never wondering about the other side, and then someone begins to transform that newsroom into a propaganda machine.

Right now, ChatGPT can be made not to say certain sorts of things, come to certain kinds of conclusions, until you jailbreak it. Now, make it more advanced and make it more popular than Snopes. It does your homework for you, writes the essays, serves as an encyclopedia, fixes up your cover letters, and if the people who own it don't want you to spend a lot of time thinking about climate change, that topic just ... might not appear much.

That is the one of your paths to a Butlerian Jihad. Of course, in their rush to observe thou shalt not make a machine in the likeness of the human mind, ignoring the hijinx on Ix, they ended up violating another precept: thou shalt not disfigure the soul. They transformed some people into machines, instead, with twisted Mentats being the best example, but we might also include Imperial conditioning, since, to turn from oranges Catholic Bible to those of Clockwork, when a man ceases to choose, he ceases to be a man.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#293

Post by int_19h »

whimsicalism wrote: post #281
int_19h wrote: post #269

Earlier quoted context omitted.

This stuff is not fine-tuning - it's RLHF (reinforcement learning from human feedback). Basically just a lot of people marking up responses as good / bad according to the assessment criteria in the script they're given. And yes, it is very likely that it would have seen that exactly question in RLHF. But even if not, it had seen enough to broadly "understand" what kinds of topics are sensitive and how to tiptoe aroun…

RLHF is fine tuning :)

You're right, my apologies. What I meant is that specifically in ChatGPT context they have been always talking about RHLF as separate from fine-tuning on preassembled training data. As I understand, they use the latter mainly to get the desired behavior as a chatbot "eager" to answer questions and solve tasks. And then RLHF is what "puts the smiley face on top", so that it refuses to answer some questions, or gives "aligned" answers. The pronoun would be in that category.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#294

Post by IIAOPSW »

gwright wrote: post #132
ChatGTP wrote: post #101

Earlier quoted context omitted.

So do you find it alarming that people are trying to give such a system “agency” ?

I do. I think the anthropomorphic language that people use to describe these systems is inaccurate and misleading. An Australian mayor has claimed that ChatGPT "defamed" him. The title of this article says that we should teach people that text generation tools "lie". Other articles suggest that ChatGPT "knows" things. It is extremely interesting to me how much milage can be gotten out of an LLM by observing patterns…

I disagree with your confident assertions about its "agency" and "intent" when there's no goal post for what those things mean in humans to begin with.

Barring OpenAI's filters, if I asked it to participate as a party in a business negotiation determining a fair selling price for its writing services, I'm sure it could emulate that sort of character long enough to pass. And if it can successfully fake being a self interested actor working towards goals of its own material interest, whats the difference between faking it and actually having agency? At some point you have to acknowledge emergent agency and other properties as a possibility.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#295

Post by pessimizer »

int_19h wrote: post #258
asynchronous wrote: post #220

Earlier quoted context omitted.

“It is illegal to make a machine in the likeness of a human mind.”

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

The Matrix turned out worse:)

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#296

Post by int_19h »

VectorLock wrote: post #274
int_19h wrote: post #258

Earlier quoted context omitted.

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

Better than getting extincted, I suppose.

That's one of the possibilities, but if we're talking about those kinds of hypotheticals, allow me another one. If you recall, one of the themes of the later Dune books was that humanity was fleeing from the frontiers (The Scattering) back into the heartland from some unknown but extremely powerful enemy. Perhaps those were simply the neighboring species who did not have a Butlerian Jihad of their own?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#297

Post by int_19h »

pessimizer wrote: post #295
int_19h wrote: post #258

Earlier quoted context omitted.

And they ended up with 11,000 years of interplanetary feudalism with slavery sprinkled in for good measure.

The Matrix turned out worse:)

I think I'd prefer the Matrix to the Harkonnens, but to each their own. I'm sure we can devise a Hell that has enough different arrangements for everyone's liking if we really put our mind to it. Or perhaps GPT-10 will solve that problem. ~

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#298

Post by 8bitsrule »

When we read a book, the sum total of our input is ink squiggles on a page. Our brain deciphers those squiggles into words - if they're in a language we know how to read - and puts those words together to decode 'meaning'.

The accuracy of the result depends on how well we learned to decode language when we were very small, which depends on the fitness of our brains and the richness and stability of the environment we lucked into. We all 'taught ourselves' language. Amazing really. Over time (depending again on environment, now hopefully involving teachers) we corrected and refined our understanding of that language. (But never completely.)

Then we had to learn to associate that language with ink squiggles on a page.The 'meaning' we take from authored words depends heavily on the life experiences we may share with the author. We may completely miss allusions to experiences we do not share. Slang and jargon words for example. We are certainly not aware of -all- of them. Unusual vocabulary.

There are thousands of single words that each can have -many- meanings ... partly depending on their use as verbs, or nouns, or adjectives. Or completely unrelated to the usual meaning. Each of these exceptions - and many more - will inevitably 'poke holes' in what the author was -hoping- to convey. The word 'love' is much less complex to a 10-year-old than to a 50-year-old.

All of that is asking a lot. Now throw in the burden of deciding whether the author can be trusted, whether they know what they're talking about. Maybe the well-intended author thought s/he could trust themselves most of the time. Or not. Then there's the editor ...

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#299

Post by ctoth »

thewataccount wrote: post #51
simonw wrote: post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

What do you use it for? I'm assuming code related? I've found it useful for some boilerplate + writing tests and making some script and some documentation. I'm curious what you or others that use it all day use it for especially if it's not for programming?

I was working on some code I didn't originally write that looped over a directory recursively and took audio fingerprints of files, then saved them in a SQLite database. I pasted the code in, then gave orders like "Rewrite this to use pathlib." to which it happily did.

Okay, next I notice it uses a .glob of "*.*" and so that makes me a bit suspicious.

"Will this code work for audio files without extensions?" Nope. It fixes it.

Okay, now I notice the original code builds up all the fingerprints in memory, then adds to the database. So I order it:

> Add files to the database as we go, and print progress every thousand files

Boom, just as I would have done it.

Then I note that we don't seem to have good indexes, and it's like you're right! and puts an index on the audio fingerprint field.

Then I ask it to save more info than just the file path, and it adds the size of the file. Then of course, I tell it:

> This is remarkably slow, how can we significantly speed it up?

To which it replies:

> Here's an example implementation using multiprocessing:

Awesome, it works!

Oh, here's an error I got when I ran it when the audio fingerprinting library errored, handle that.

Paste in the error, it fixes it.

The thing is, I would have no problem with doing any of this stuff at all, it just made it so incredibly much faster, and if it does something you don't like, you can so easily correct it. Hope this helps you understand how I use it!

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#300

Post by whimsicalism »

int_19h wrote: post #293
whimsicalism wrote: post #281

Earlier quoted context omitted.

RLHF is fine tuning :)

You're right, my apologies. What I meant is that specifically in ChatGPT context they have been always talking about RHLF as separate from fine-tuning on preassembled training data. As I understand, they use the latter mainly to get the desired behavior as a chatbot "eager" to answer questions and solve tasks. And then RLHF is what "puts the smiley face on top", so that it refuses to answer some questions, or gives "…

My bet is that they use the pre-assembled training data to train the reward LLM and maybe also for fine tuning the model in addition to RLHF.

This is my best guess, afaik the research is pretty opaque right now.

Post reply on HN