Live data from Hacker News

We need to tell people ChatGPT will lie to them, not debate linguistics

simonwillison.net

231–240 of 485 posts

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#232

Earlier quoted context omitted.

Anth- Antrom- Antropo- wait, no I think it's Anthropomorphizing I agree w your take.

It's common: https://www.google.com/search?client=firefox-b-d&q=%22anthro... 5000 hits

Ah, learn sum new everyday

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#233

[flagged]

I think 'lying' is perfectly appropriate if there's a structured orientation towards the mistruth.

'Broken' or 'wrong' would be fine if that were simply just making an error.

But if there's something in the system, even the corpus, that permeates a falsehood then I think 'lying' is the right term.

ChatGPT has a morality, and it's a bit specific to the Anglosphere. It talks like the perfect LinkedIn corporate bot. This means that it will effectively misrepresent ideas on a lot of sensitive subjects from the purview of many people in the world. That's a form of 'structured misrepresentation' or maybe just bias, but it can feel like lying.

'Hallucinating' is a cool sounding but inappropriate anthropomorphization.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#234
ChatGPT (often called Geptile in Russian - from “heptile”, which is a very powerful but very dangerous rocket fuel) can well lie when debating linguistics, lol. Like:

Например, в слове "bed" ударение на первом слоге, а в слове "get" - на втором.

Here, Geptile (in a good Russian) insists that English word “get” has two syllables! When pointed to the error, Geptile apologizes - and then repeats the error again.

But I guess it is not the program lying, but it’s sellers. It should have version 0.35, not 3.5…

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#235

[flagged]

The thing is, LLMs are not-even-wrong.

They are detached from correctness because they're answering a question that is loosely correlated with correctness. It's not like they're attempting to be correct, or care about correctness

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#236

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

GPT-2, 3 and 4 keep on showing that increasing the size of the model keeps on making the results better without slowing down.

This is remarkable, because usually in practical machine learning applications there is a quickly reached plateau of effectiveness beyond which a bigger model doesn't yield better results. With these ridiculously huge LLMs, we're not even close yet.

And this was exciting news in papers from years ago talking about the upcoming GPT3 btw.

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#237
post #199

Earlier quoted context omitted.

… with confident language no less xD

Butlerian Jihad.

That doesn't help the root problem though, confidently wrong people are just as convincing as confidently wrong robots

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#238
post #212

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

"Often wrong, never in doubt." An old saying, but frequently applies to the difficult people in your life. Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes yo…

Re wikipedia, I think it's most interesting how you choose which pages are trustworthy and which ones are not, without doing a detailed analysis on all the sources listed.

What are the working heuristics?

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#239

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. I don't think you have a shred of evidence to back up this assertion.

[deleted]

Re: We need to tell people ChatGPT will lie to them, not debate linguistics

#240
post #212

The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…

"Often wrong, never in doubt." An old saying, but frequently applies to the difficult people in your life. Related, I remember when wikipedia first started up, and teachers everywhere were up-in-arms about it, asking their students not to use it as a reference. But most people have accepted it as "good enough", and now that viewpoint is non-controversial. (some wikipedia entries are still carefully curated - makes yo…

What's terrible about some Wikipedia pages is when you personally don't know they are controversial.

It should be vastly more prominent when there are tensions on an article and even admins shouldn't have the power to hide this.

Post reply on HN