Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

Post reply on HN
Print view View original

381–390 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#381

Post by shjake »

notahacker wrote: post #56
lm28469 wrote: post #31

Earlier quoted context omitted.

> But the distinction is it's less rude, and more knowledgeable. And that regular people assume it basically is an oracle, which doesn't happen to many people online

The internet is full of people that deliver answers confidently and eloquently enough to be widely believed, especially if they have been right on other topics. They even have similar feedback loops to GPT in learning what sort of answers impress other forum users. I'm not saying the people that think ChatGPT is an oracle don't exist, but I think it probably has more people in the surprised it works at all camp, and…

> The internet is full of people that deliver answers confidently and eloquently enough to be widely believed

Why should that be the baseline? After all you usually have hundreds or thousands of other people who have made their opinions on the subject publicly accessible, which more or less solves this problem most of the time (and I’m not talking about random people on Quora and their bizarrely absurd answers..)

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#382

Post by thinkingemote »

joe_the_user wrote: post #249
hotpathdev wrote: post #221

[flagged]

These systems are not 'lying', they are simply 'wrong' The meaningful distinction between lying and wrong, is that a liar doubles down on their position, making up more things and tailoring their statements to what their audience wants to hear. The article presents good arguments that LLMs actually do this. I've observed it myself interacting with ChatGPT. Sure, this seems weird but it's not. Read the article for why…

The liar knows the truth and speaks falsely. Being wrong is speaking false things without knowing the truth.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#384

Post by whiddershins »

bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

The real problem is that smart people are even easier to program than stupid people and they do even more damage once they are programmed.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#385

Post by [deleted] »

zirgs wrote: post #354
stcg wrote: post #108

Earlier quoted context omitted.

That's interesting, what language is that? Also, is it possible to give a prompt to make ChatGPT switch to feminine gender?

It's Latvian. But, I guess it acts the same when speaking other gendered languages. Both "chatbot" and "large language model" are masculine - so this is why it picked the masculine gender, I guess. I was asking it about its training dataset and when it said "I've been trained on..." it picked the masculine form of the word "trained". This is why machine translation is a hard problem to solve. Things like this can eas…

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#386

Post by root_axis »

kurthr wrote: post #375
lamontcg wrote: post #339

Earlier quoted context omitted.

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

The problem is that it will be cheaper at scale. That will allow the BS to be even more targeted growing the population of acolytes of ignorance. I don't know how much bigger it is, but it seems like we're around 20% today. If it gets to a majority there could be real problems.

There's already more information generated each day than any individual can ever hope to process, adding more crap to that pile doesn't move the needle much if at all.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#387

Post by topspin »

gWPVhyxPHqvk wrote: post #91
simonw wrote: post #38

Earlier quoted context omitted.

I pay for ChatGPT Plus, and use it with no delays at all dozens of times a day. The more I use it the better I get at predicting if it can be useful for a specific question or not.

I pay $0 for Google with no delays. I don’t understand why I’d want to pay for information that’s less reliable. (I’m being slightly dense, but not really)

> I don’t understand why I’d want to pay for information that’s less reliable.

It's faster. Much faster in a lot of cases. Think of ChatGPT not as a precision source of answers, but rather a universal clue generator.

Yes, ChatGPT frequently (perhaps even usually) produces subtly defective answers. Most of the code it generates won't interpret, compile, etc. without errors, frequently exhibiting simple minded mistakes. That doesn't make the output useless. Indeed, it is a great time saver. That may seem counterintuitive, but I assure you it holds.

I had to write some Perl today. I end up dealing with that about once a year. I have a deep understanding of the language (going back to serious use in the 90's,) but it's been 20+ years since I dealt with it frequently enough to achieve any sort of "flow." ChatGPT and its imperfect answers were extremely helpful in getting through this efficiently. If I had to guess I was 3-4x faster than same exercise using convention search engines.

Google, now more then ever, feels like a ghetto. Useful results from Google are always buried among low quality bait that takes time to wade through, sorting wheat from chaff. ChatGPT obviates that nonsense, producing distraction free, ad free results. Also, Google isn't some fount of truth; lots of wrong stuff makes it to the top of Google.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#388

Post by prox »

kerkeslager wrote: post #225
Eisenstein wrote: post #126

Earlier quoted context omitted.

I think it is better to assume that normal people can be deceived by confident language than to assume this is a problem with 'dumb people'.

I'd actually go further and say it's better to assume I can be deceived by confident language, than to assume that this is a problem with "dumb people". If I see other people making a mistake, I want my first question to be "am I making the same mistake?". I don't live up to that aspiration, certainly.

Yeah this is a common sales tactic, and talented/ charismatic people can certainly employ confidence for an uptick in sales.

The way to combat this is the same for all hoaxes/disinfo and so on, epistemology. The attitude of the Socratic method.

As I write this I wonder if ChatGPT can use the Socratic method on itself.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#389

Post by oneshtein »

lamontcg wrote: post #339
bdw5204 wrote: post #118

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> The only danger is that stupid people might get their brains programmed by AI rather than by demagogues which should have little practical difference. This may be the best point that you've made. We're already drowning in propaganda and bullshit created by humans, so adding propaganda and bullshit created by AI to the mix may just be a substitution rather than any tectonic change.

[flagged]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#390

Post by zem »

root_axis wrote: post #386
kurthr wrote: post #375

Earlier quoted context omitted.

The problem is that it will be cheaper at scale. That will allow the BS to be even more targeted growing the population of acolytes of ignorance. I don't know how much bigger it is, but it seems like we're around 20% today. If it gets to a majority there could be real problems.

There's already more information generated each day than any individual can ever hope to process, adding more crap to that pile doesn't move the needle much if at all.

but most of that crap will go entirely disregarded, because it's not coated in something that makes it attractive. what ai really helps scale up is finding millions of ways to dress up fake news, hoaxes, propaganda, etc in a container - a Trojan horse, if you will - that encourages people to consume it.

as one example, imagine if you could churn out a stream of "foreigners are taking your jobs!" articles, but every one talking about a different sector or profession. people would be far more likely to stop and read the article that was talking about their particular line of work, and thanks to ai you can now practically write a custom article for anyone you can think of in as much time as it takes to identify the demographic.

Post reply on HN