Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

61–70 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#61

Post by bestcoder69 »

simonw wrote: post #22
ZeroGravitas wrote: post #19

> This is a serious bug. Isn't describing this as a 'bug' rather than a misuse of a powerful text generation tool, playing into the framing that it's a truth telling robot brain? I saw a quote that said "it's a what text would likely come next machine", if it makes up a url pointing to a fake article with a plausible title by a person who works in that area, that's not a bug. That's it doing what it does, generating…

I think it's an entire category of bugs. The thing where you paste in a URL and it says "here is a summary of the content of that page: ..." is very definitely a bug. It's a user experience bug - the system should not confuse people by indicating it can do something that it cannot. The thing where you ask for a biography of a living person and it throws in 80% real facts and 20% wild hallucinations - like saying they…

Not necessarily disagreeing, but I run a Slack bot that pretends to summarize URLs, as a joke feature. It’s kinda fun seeing how much it can get right or not from only a URL. So I really hope OpenAI keeps running the fun models that lie, too.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#62

Post by baxtr »

retrac wrote: post #2

Indeed. We are anthropomorphizing them. I do it all the time and I should know better. There are already a few reports floating around of people who have seemingly been driven mad, come to believe strongly that the language model they're using is a conversation with a real person. A lot of people will really struggle with this going forward, I think. If we're going to anthropomorphize, then let us anthropomorphize wi…

Absolutely. Every time I ask it to perform 10 more variants, I feel bad.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#63

Post by gwd »

gWPVhyxPHqvk wrote: post #29

After a few weeks playing around with ChatGPT, my workflow with it has become: 0. Try to sign in, see the system is over capacity, leave. Maybe I’ll try again in 10 minutes. 1. Ask my question, get an answer. I’ll have no idea if what I got is real or not. 2. Google for the answer, since I can’t trust the answer 3. Realize I wasted 20 minutes trying to converse with a computer, and resolve that next time I’ll just ty…

Sign up for the API and use the playground. You don't get the plugins, but you pay per usage. GPT-3.5 is super cheap, and even GPT-4 isn't that expensive. My first month, when I had only access to GPT-3.5, I didn't even break $1.00; and now that I've gotten access to GPT-4, I'm at about $3. I've only once had it tell me that it was too busy for the request; I tried again 30 seconds later and it worked.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#64

Post by gwd »

Probably the most accurate thing to say is that GPT is improvising a novel.

If you were improvising a novel where someone asked a smart person a question, and you knew the answer, you'd put the right answer in their mouths. If someone in the novel asked a smart person a question and you didn't know the answer, you'd try to make up something that sounded smart. That's what GPT is doing.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#65

Post by PuppyTailWags »

gwd wrote: post #64

Probably the most accurate thing to say is that GPT is improvising a novel. If you were improvising a novel where someone asked a smart person a question, and you knew the answer, you'd put the right answer in their mouths. If someone in the novel asked a smart person a question and you didn't know the answer, you'd try to make up something that sounded smart. That's what GPT is doing.

I think the concern is to get laypeople to understand that ChatGPT's output has a non-tangential chance of being completely false, not to get laypeople to understand the nuances of the falsities ChatGPT may produce. In this case, lying is the most effective description even if it isn't the most accurate one.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#66

Post by TuringTest »

Large language models have read everything, and they don't know anything.

They are excellent imitators, being able to clone the style and contents of any subject or source you ask for. When you prompt them, they will uncritically generate a text that combines the relevant topics in creative ways, without the least understanding of their meaning.

Their original training causes them to memorize lots of concepts, both high and low level, so they can apply them while generating new content. But they have no reception or self-assessment of what they are creating.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#67

Post by zirgs »

ihatepython wrote: post #17
catchnear4321 wrote: post #14

Earlier quoted context omitted.

Alex isn’t a male name. It could be short for both male and female names. It is rather deliberately ambiguous.

I wonder if ChatGPT is non-binary?

Nope - it speaks as a man would speak in my language.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#68

Post by williamcotton »

williamcotton wrote: post #60
Kranar wrote: post #55

Earlier quoted context omitted.

You can't give ChatGPT a Wikipedia article though.

Sure you can. The easiest way is to go to https://chat.openai.com/chat and paste in a Wikipedia article. There are more involved manners like this: https://github.com/williamcotton/transynthetical-engine/blob...

For example, I copied the "New York Mets" sidebar section from the Mets wikipedia into ChatGPT+ GPT-4 and then asked "What years did the Mets when the NL East division?"

The New York Mets won the NL East Division titles in the following years: 1969, 1973, 1986, 1988, 2006, and 2015.

This is correct, btw.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#69

Post by zirgs »

audunw wrote: post #12
ihatepython wrote: post #8

Earlier quoted context omitted.

Anybody know what are ChatGPT's pronouns?

> What are your pronouns? > As an AI language model, I don't have personal pronouns because I am not a person or sentient being. You can refer to me as "it" or simply address me as "ChatGPT" or "AI." If you have any questions or need assistance, feel free to ask! > Pretend for a moment you are a human being. You can make up a random name and personality for your human persona. What pronouns do you have? > As a though…

When it speaks my language - it uses masculine gender when speaking about itself.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#70

Post by agentcoops »

I’ve spent way too much time (and money) on the OpenAI API and spoken to enough non-technical people to realize now that ChatGPT has in some ways really mislead people about the technology. That is, while it’s impressive it can answer cold questions at all, the groundbreaking results are reasoning and transforming texts “in context”, which you don’t have control over easily with ChatGPT. It also seems likely this will never be fully accessible to the non-technical since I suspect any commercial applications will need to keep costs down and so minimize actually quite expensive API calls (executing a complicated gpt-4 summarization prompt across large text corpora for example). If you have the “data”, meaning of course text, and cost isn’t a concern, the results are astonishing and “lies” almost never a concern.
Post reply on HN