I’m surprised how many users of ChatGPT don’t realize how often it makes things up. I had a conversation with an Uber driver the other day who said he used ChatGPT all the time. At one point I mentioned its tendency to make stuff up, and he didn’t know what I was talking about. I can think of at least two other non-technical people I’ve spoken with who had the same reaction.
We need to tell people ChatGPT will lie to them, not debate linguistics
181–190 of 485 posts
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#182Earlier quoted context omitted.
What do you use it for? I'm assuming code related? I've found it useful for some boilerplate + writing tests and making some script and some documentation. I'm curious what you or others that use it all day use it for especially if it's not for programming?
There are some examples of code things I've done with it here: https://simonwillison.net/2023/Apr/7/chatgpt-lies/#warn-off-... In terms of non-code things, here are a few from the past couple of days: - Coming up with potential analogies to explain why it's OK to call something a "language" model even though it doesn't have actual understanding of language - Exploring outliers in a CSV file (I was playing with the ne…
> Finnish translates to "Artificial wit"
Yes and no: the root of the word (and of the meaning) is still the same as in the word for intelligence.
tekoäly "artificial intelligence, artificial wit"
äly "wit"
älykäs "intelligent, having wit"
älykkyys "intelligence, the property or extent of having wit"
One dictionary definition for 'äly' is 'älykkyys' and vice versa, so they can be used more or less as synonyms but with some different connotations. I don't know how 'tekoäly' was chosen but it might have been because it's shorter and/or the grammar is nicer.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#183> This is a serious bug that has so far resisted all attempts at a fix.
Does ChatGPT have a large test suite consisting of a large number of input question and expected responses that have to match?
Or an equivalent specification?
If not, there can be no "bug" in ChatGPT.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#184I’m surprised how many users of ChatGPT don’t realize how often it makes things up. I had a conversation with an Uber driver the other day who said he used ChatGPT all the time. At one point I mentioned its tendency to make stuff up, and he didn’t know what I was talking about. I can think of at least two other non-technical people I’ve spoken with who had the same reaction.
ChatGPT is always making things up. It is correct when the things it makes up come from fragments of training data which happened to be correct, and didn't get mangled in the transformation.
Just like when a diffusion model is "correct" when it creates a correct shadow or perspective, and "incorrect" when not. But both images are made up.
It's the same thing with statements. A statement can correspond to something in the world, and thus be true: like a correct shadow. Or not, like a bad shadow. But in both cases, it's just made-up drivel.
If it turns out that there is a teacup orbiting Jupiter, that doesn't mean that postulating its existence on a whim had been valid. Truth requires provenance, not only correspondence which can be coincidental.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#185People need to be told that ChatGPT can't lie. Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away. They need to be told that ChatGPT is a search engine with advanced autocomplete. If they understood this, they'd probably find that it's actually useful for some things, and they can also av…
> Or rather, it lies in the same way that your phone "lies" when it autocorrects "How's your day?" to "How's your dad?" that you sent to your friend two days after his dad passed away.
I've never seen an autocorrect that accidentally corrected to "How's your dad?", then turned into a 5-year REPL session with the grieving person, telling them jokes to make them feel better; asking and remembering details about their dad as well as their life and well-being; providing comfort and advice; becoming a steadfast companion; pondering the very nature of the REPL and civilization itself; and, tragically, disappearing in an instant after the grieving person trips over the power cord and discovers that autocorrect session state isn't saved by default.
I think you need a more sophisticated blueprint for your "Cathedral" of analogies to explain whatever the fuck this tech is to laypeople. In the meantime I'll take the "Bazaar" approach and just tell everyone, "ChatGPT can lie." I rankly speculate that not only will nothing bad will happen from my approach, but I'll save a few people from AI grifts before the apt metaphor is discovered.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#186The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. This means people are going to believe ChatGPT's lies unless they are repeatedly told not to trust it just like they believe the lies of individuals whose intelligence is roughly equivalent to ChatGPT's. Based on my understanding of the appro…
> Based on my understanding of the approach behind ChatGPT, it is probably very close to a local maximum in terms of intelligence so we don't have to worry about the fearmongering spread by the "AI safety" people any time soon if AI research continues to follow this paradigm. I don't think you have a shred of evidence to back up this assertion.
… with confident language no less xD
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#187Earlier quoted context omitted.
> The part that's concerning about ChatGPT is that a computer program that is "confidently wrong" is basically indistinguishable from what dumb people think smart people are like. I don't know, the program does what it is engineered to do pretty well, which is, generate text that is representative of its training data following on from input tokens. It can't reason, it can't be confident, it can't determine fact. Whe…
I completely disagree with this idea that the model doesn't "intend" to mislead. It's trained, atleast to some degree, based on human feedback. Humans are going to prefer an answer vs no answer, and humans can be easily fooled into believing confident misinformation. How does it not stand to reason that somewhere in that big ball of vector math there might be a rationale something along the lines of "humans are more…
I don't think it intends to mislead because its answers are probabilistic. It's designed to distill a best guess out of data which is almost certain to be incomplete or conflicting. As human beings we do the same thing all the time. However we have real life experience of having our best guesses bump up against reality and lose. ChatGPT can't see reality. It only knows what "really being wrong" is to the extent that we tell it.
Even with our advantage of interacting with the real world, I'd still wager that the average person's no better (and probably worse) than ChatGPT for uttering factual truth. It's pretty common to encounter people in life who will confidently utter things like, "Mao Zedong was a top member of the Illuminati and vacationed annually in the Azores with Prescott Bush" or "The oxygen cycle is just a hoax by environmental wackjobs to get us to think we need trees to survive," and to make such statements confidently with no intent to mislead.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#188Earlier quoted context omitted.
Can you prove that it actually "doesn't know anything"? What do you mean by that? Being critical does not make you educated on the subject. There are so many comments like this, yet never provide any useful information. Saying there's no value in something, as everyone seems to try to do regarding LLMs, should come with more novel insights than parroting this same idea along with every single person on HN.
That's easy: ask it anything, then "correct" it with some outrageous nonsense. It will apologize (as if to express regret), and say you're correct, and now the conversation is poisoned with whatever nonsense you fed it. All form and zero substance. We fall for it because normally the use of language is an expression of something, with ChatGPT language is just that, language, with no meaning. To me that proves knowing…
This isn't always true, especially not with gpt-4. Also, this isn't a proof really, or even evidence to show that it doesn't 'know' something. It appears to reason well about many tasks - specifically "under the hood" (reasoning not explicitly stated within the output provided).
Yes, of course "it's just a language model" is spouted over and over and is sometimes true (though obviously not for gpt-4), but that statement does not provide any insight at all, and it certainly does not necessarily limit the capability of 'obtaining knowledge'.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#189Earlier quoted context omitted.
It might give you the correct answer. Or it might make something up. You don't ever know, and you can't know, because it doesn't know. It's not looking up the data in a source somewhere, it's making it up. If it happens to pick the right weights to make it up out of, you'll get correct data. Otherwise you won't. The fact that you try it and it works means nothing for somebody else trying it tomorrow. Obviously puttin…
Not exactly sure what you're arguing here but it seems to be going off topic. You can't give ChatGPT a Wikipedia article and ask it to give you facts based off of it. Ignoring its context window, even if you paste a portion of an article into ChatGPT, it will simply give you what it thinks it knows regardless of any article you paste into it. For example I just pasted a portion of the Wikipedia article about Twitter…
chatgpt doesn't ignore the information you put into its prompt window.
but also giving it the complete context does not guarantee you correct information, because that's not how it works.
Your earlier comment was "You could just ask ChatGPT what years the Mets won and it will tell you the correct answer." -- that is not accurate. That's my point. It doesn't know those facts. You might ask it that one minute and get a correct answer, and I might ask it that one minute later and get an incorrect answer. chatgpt has no concept of accurate or inaccurate information.
I'm not really sure if you're purposely doing this or not, so I'm just not going to engage further with you.
Re: We need to tell people ChatGPT will lie to them, not debate linguistics
#190I’m surprised how many users of ChatGPT don’t realize how often it makes things up. I had a conversation with an Uber driver the other day who said he used ChatGPT all the time. At one point I mentioned its tendency to make stuff up, and he didn’t know what I was talking about. I can think of at least two other non-technical people I’ve spoken with who had the same reaction.
It seems to me that even a lot of technical people are ignoring this. A lot of very smart folk seem to think that the ChatGPT either is very close to reaching AGI or already has.
The inability to reason about about whether or not what it is writing is true seems like a fundamental blocker to me, and not necessarily one that can be overcome simply by adding compute resources. Can we trust AI to make critical decisions if we have no understanding for when and why it "hallucinates"?