Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

51–60 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#51

Post by thewataccount »

simonw wrote: post #38
gWPVhyxPHqvk wrote: post #29

After a few weeks playing around with ChatGPT, my workflow with it has become: 0. Try to sign in, see the system is over capacity, leave. Maybe I’ll try again in 10 minutes. 1. Ask my question, get an answer. I’ll have no idea if what I got is real or not. 2. Google for the answer, since I can’t trust the answer 3. Realize I wasted 20 minutes trying to converse with a computer, and resolve that next time I’ll just ty…

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

What do you use it for? I'm assuming code related? I've found it useful for some boilerplate + writing tests and making some script and some documentation.

I'm curious what you or others that use it all day use it for especially if it's not for programming?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#52

Post by zmmmmm »

I think characterisation of LLMs as lying is reasonable because although the intent isn't there to misrepresent the truth in answering the specific query, the intent is absolutely there in how the network is trained.

The training algorithm is designed to create the most plausible text possible - decoupled from the truthfulness of the output. In a lot of cases (indeed most cases) the easiest way to make the text plausible is to tell truth. But guess what, that is pretty much how human liars work too! Ask the question: given improbable but thruthful output but plausible untruthful output, which does the network choose? And which is the intent of the algorithm designers for it to choose? In both cases my understanding is, they have designed it to lie.

Given the intent is there in the design and training, I think it's fair enough to refer to this behavioral trait as lying.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#53

Post by williamcotton »

ChatGPT doesn't lie. It either synthesizes or translates. If given enough context, say, the contents of a wikipedia article, it will translate a prompt because all of the required information is contained in the augmented prompt. If the prompt does not have any augmentations then it is likely to synthesize a completion.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#54

Post by wpietri »

revolvingocelot wrote: post #36
zug_zug wrote: post #27

Earlier quoted context omitted.

That all feels tad dramatic. It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.

When someone is wrong on the internet, nailing both the tone and vocabulary of the type of expert said wrong someone is purporting to be is rare and impressive. But ChatGPT nails both an overwhelming amount of the time, IME, and in that way it is entirely unlike a person on the internet.

Exactly. Most wrong-on-the-internet people have tells. And most of the rest have history. We all use this all the time on HN. But with LLMs the former gets engineered away and the latter is nearly useless. Except of course to the extent that we say,"Wow, ChatGPT is absolutely untrustworthy, so we should never listen to it." But given how many people are excited to use ChatGPT as a ghostwriter, even that is limited.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#55

Post by Kranar »

williamcotton wrote: post #53

ChatGPT doesn't lie. It either synthesizes or translates. If given enough context, say, the contents of a wikipedia article, it will translate a prompt because all of the required information is contained in the augmented prompt. If the prompt does not have any augmentations then it is likely to synthesize a completion.

You can't give ChatGPT a Wikipedia article though.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#56

Post by notahacker »

lm28469 wrote: post #31
zug_zug wrote: post #27

Earlier quoted context omitted.

That all feels tad dramatic. It's like a person on the internet -- in that it's wrong 20% of the time, often confidently so. But the distinction is it's less rude, and more knowledgeable.

> But the distinction is it's less rude, and more knowledgeable. And that regular people assume it basically is an oracle, which doesn't happen to many people online

The internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users.

I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and certainly more people inclined to totally disbelieve it than a random off Quora or Reddit...

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#57

Post by japhyr »

thewataccount wrote: post #51
simonw wrote: post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

What do you use it for? I'm assuming code related? I've found it useful for some boilerplate + writing tests and making some script and some documentation. I'm curious what you or others that use it all day use it for especially if it's not for programming?

I was just working on a small exploratory project in Python. I used sys.argv because it's so quick to prototype with.

When I started refining the project for longer term development, I wanted to convert the CLI to use argparse, so I could build a more nuanced CLI. I gave GPT a couple example commands I wanted to use, and in less than a minute I had a fully converted CLI that did exactly what I wanted, with more consistent help settings.

I can do that work, but it would have been 30-45 minutes because there was one setting in argparse I hadn't used before. That alone was worth this month's $20.

For more complex and mature projects I could see having to give GPT a minimum working example of what I need instead of the whole project, but I can already see how it will enhance my current workflows, not replace them.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#58

Post by visarga »

usrusr wrote: post #39

I wonder how the impact on the legal field will eventually turn out: from a certain perspective, that already looks like a battle more of quantity than of quality, throw tons of binders full of hopelessly weak arguments at the other side and if they fail to find the few somewhat sound needles in that haystack they won't be able to prepare a defense. Now enter a tool that can be used to write seemingly infinite amount…

LLMs can be also be used as negotiators. One LLM agent from each party.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#59

Post by simonw »

thewataccount wrote: post #51
simonw wrote: post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

What do you use it for? I'm assuming code related? I've found it useful for some boilerplate + writing tests and making some script and some documentation. I'm curious what you or others that use it all day use it for especially if it's not for programming?

There are some examples of code things I've done with it here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-...

In terms of non-code things, here are a few from the past couple of days:

- Coming up with potential analogies to explain why it's OK to call something a "language" model even though it doesn't have actual understanding of language

- Exploring outliers in a CSV file (I was playing with the new alpha code interpreter, where you can upload a file and have it run and execute Python to evaluate that file)

- Asking some basic questions about GDPR cookie banners

- Figuring out if there are any languages worldwide where "Artificial Intelligence" translates to something that doesn't include the word "Intelligence" (found some good ones: Finnish translates to "Artificial wit", Swahili to "Artificial cunning")

- Understanding jargon in a tweet about payment services - I wanted to know the difference between a "payment aggregator" and a "merchant acquirer"

- As a thesaurus, finding alternatives to "sanctity" in the sentence "the sanctity of your training data"

- Fun: "Pretend to be human in a mocking way", then "Again but meaner and funnier"

- "What should I be aware of when designing the file and directory layout in an S3 bucket that could grow to host millions of files?"

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#60

Post by williamcotton »

Kranar wrote: post #55
williamcotton wrote: post #53

ChatGPT doesn't lie. It either synthesizes or translates. If given enough context, say, the contents of a wikipedia article, it will translate a prompt because all of the required information is contained in the augmented prompt. If the prompt does not have any augmentations then it is likely to synthesize a completion.

You can't give ChatGPT a Wikipedia article though.

Sure you can. The easiest way is to go to https://chat.openai.com/chat and paste in a Wikipedia article.

There are more involved manners like this: https://github.com/williamcotton/transynthetical-engine/blob...

Post reply on HN