It does acknowledge it's wrong, but it doesn't really understand.
Ask HN: ChatGPT doesn't ackowledge being wrong?
11–20 of 23 posts
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#12This is why it should be outlawed to call every ML model we have today "AI". Those things are NOT artificial intelligence. They are specific noise generators. They generate noise that is as similar as possible to noise it learned from, which match the input. THats it. nothing more.
While I agree that these large language models are not 'artificially intelligent' - I call them machine learning models, no more and no less - the question does rise what intelligence actually is. In what way does our mental meandering differ from that of a deep model, and what makes our mental processes be described as 'thoughts' while those of the server-under-the-stairs are mere 'calculations' producing 'shaped noise'?
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#13This is why it should be outlawed to call every ML model we have today "AI". Those things are NOT artificial intelligence. They are specific noise generators. They generate noise that is as similar as possible to noise it learned from, which match the input. THats it. nothing more.
I agree in principle, but the reality is that we're facing a near future where the output of ML models is indistinguishable from the output of an AI. So if we just replace "AI" with "ML" in all these discussions about ethics, there will be some questions worth considering.
That said, the question of "why doesn't ChatGPT caveat every single response with a reminder that it might be inaccurate" doesn't seem like a meaningful or important discussion.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#14It always acknowledged it to me. Something like: > AI: Answer that's clearly factually wrong (e.g. a function that doesn't exist or a completely wrong numerical figure). > Me: that's not right, the function doesn't exist (etc.) > AI: you are right, that function doesn't exist. The answer is blah And repeat, since if it didn't get it right first time, it seems unlikely to be able to get there at all (and you'd not kno…
I always wondered why it bothers returning the first result if it can tell it's wrong 30 seconds later if you ask it again "I think that's wrong". Why not do make the function that outputs answers also feed itself "is this actually right/are you sure/is this not wrong"? Too expensive? Giant loop?
It can't tell that it's wrong. You tell it that it's wrong.
Then you reset your session, you ask the same question and you get the wrong answer again.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#15It always acknowledged it to me. Something like: > AI: Answer that's clearly factually wrong (e.g. a function that doesn't exist or a completely wrong numerical figure). > Me: that's not right, the function doesn't exist (etc.) > AI: you are right, that function doesn't exist. The answer is blah And repeat, since if it didn't get it right first time, it seems unlikely to be able to get there at all (and you'd not kno…
I always wondered why it bothers returning the first result if it can tell it's wrong 30 seconds later if you ask it again "I think that's wrong". Why not do make the function that outputs answers also feed itself "is this actually right/are you sure/is this not wrong"? Too expensive? Giant loop?
It's a language model trained to return the most likely answer given a specific prompt. The goal of the answer is, to simulate a conversation and to sound realistic. ChatGPT is not grounded in reality, it does not know where its "knowledge" comes from, nor does it know whether the things it is saying are correct or wrong.
It can't tell it's wrong, it just reacts in a plausible well to you telling it it's wrong (acknowledging it and giving another explanation). Since ChatGPT is an expansion of GPT-3, those things won't be really solved since being somewhat of a knowledge base is a nice side effect, not the main goal of the model.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#16The way it's constructed is very impressive but it doesn't (to the best of my knowledge and understanding) have the concept of "this is correct and this is incorrect" that you have - or any kind of analogy to it whatsoever. (NB this does not mean it has no internal analogy to a concept or to correctness. It's just under no obligation to be like yours if it does.)
There seem to be a fair number of humans of whom I could say the same. As with the humans, the question is "which jobs am I happy to see this one doing and which ones worry me".
As GPT has endless ability to produce flowing words and precisely zero concept-comparable-to-will to verify them, unfortunately, the job for which it is most apt is probably politics. That I must admit worries me a little because of all the gpt-like humans we tolerate and encourage.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#17Earlier quoted context omitted.
I always wondered why it bothers returning the first result if it can tell it's wrong 30 seconds later if you ask it again "I think that's wrong". Why not do make the function that outputs answers also feed itself "is this actually right/are you sure/is this not wrong"? Too expensive? Giant loop?
It's a language model trained to return the most likely answer given a specific prompt. The goal of the answer is, to simulate a conversation and to sound realistic. ChatGPT is not grounded in reality, it does not know where its "knowledge" comes from, nor does it know whether the things it is saying are correct or wrong. It can't tell it's wrong, it just reacts in a plausible well to you telling it it's wrong (ackno…
Thank you, this is a really clear way of explaining it and a careful reading would be helpful to lots of people who think this is something it isn't.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#18It always acknowledged it to me. Something like: > AI: Answer that's clearly factually wrong (e.g. a function that doesn't exist or a completely wrong numerical figure). > Me: that's not right, the function doesn't exist (etc.) > AI: you are right, that function doesn't exist. The answer is blah And repeat, since if it didn't get it right first time, it seems unlikely to be able to get there at all (and you'd not kno…
I tried to get it to highlight the row with today's date in Google sheets. It got it wrong, I told it it was wrong, acknowledged and tried again. After 3 wrong answers I gave up and went to Google where the answer was the first result. I went back to chatgpt and told it "the way to do this is actually {blah}" and it went "yes, to do this you do such and such..." and launched into the regular speil where it explains the code it just gave you to you.
Mansplained by a robot...
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#19It sure can acknowledge being wrong, but it's not what it's there for.
GPT is the ultimate cosplayer. It pretends to be who you want it to be. If you want it to answer a question, it will make up something that looks like a valid answer to that question. Sometimes it may actually be the right answer (after all, the right answer has a pretty good chance to look like the right answer), but ultimately being right is not the goal it's trying to achieve - all it "wants" is to autocomplete your prompt in a plausible way.
Instead of making it try to come up with the answer, you can, for example, ask it (either explicitly or indirectly) to cosplay a scientist who is unsure of their position and tries to evaluate various options - and it will be happy to oblige.
What's worrying is not whether the model "acknowledges being wrong" or not - it's rather how it's being marketed to people and, in turn, what people expect from it. We've got plenty of submissions here on HN with people being surprised that the model has made up exactly what they asked it for - for example [0], which shouldn't happen if there wasn't a dissonance between what people think it's doing and what it's actually doing.
Re: Ask HN: ChatGPT doesn't ackowledge being wrong?
#20ChatGPT isn't wrong.
Imagine putting two newspaper headings on the shelf at your local drugstore that happen to spell out an incorrect headline when combined. (Or put two browser windows together with a similar effect).
This is not wrong, your understanding of what it means is wrong. Sort your perspective out and you're fine.
1. LLMs are working as expected.
2. https://beta.openai.com/docs/guides/fine-tuning/case-study-i...
3. More complicated - probably not. It mostly just needs more data with specific training on domain specific areas. Take any field and generate some wrong completions from the data available and some right completions and the output tends to get better. Train it in the opposite fashion and it gets worse.
The meat of this comes back to people see ChatGPT as changing its response as an indication of understanding, but it's not. Users are just adding extra constraints on what parts of the language model the model uses to generate text. This is not the same thing.
An (perhaps poor) example of that might be where you're deciding to buy a tool that might solve a problem that you have. Evaluating whether it does solve the problem you decide that it doesn't. But then you evaluate whether you can afford the tool based on your budget and you decide you can, so you buy the tool. You haven't changed your mind, you've just constrained the reasons that you're using to evaluate the choice (electronic musicians often refer to this as GAS - Gear Acquisition Syndrome).
TL;DR: Stop Anthropomorphizing Language Models