Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

421–430 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#421

Post by beardedwizard »

ninetyninenine wrote: post #394
Jensson wrote: post #330

Earlier quoted context omitted.

> What is the difference between a machine that for all intents and purposes appears to understand something to a degree of 100 percent versus a human? There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, thus we say the model doesn't understand. We do that for humans as well. For example, for Sora in the video wit…

>There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, thus we say the model doesn't understand. We do that for humans as well. Two things. We also see the model make things that are correct. In fact the mistakes are a minority in comparison to what it got correct. That is in itself an indicator of understanding to…

How can it possibly understand physics when the training data does not teach it or contain the laws of physics?

Re: Hallucination is inevitable: An innate limitation of large language models

#422

Post by naasking »

renegade-otter wrote: post #381
godelski wrote: post #351

Earlier quoted context omitted.

Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV , https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE…

You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world. I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly…

> They can't reason about it - they just need to see enough data of people handling books.

I'd like to see you reason about something before you've seen any data about it. What a silly argument. LLMs understand the world as presented to them via the training data, and the training data to date has been biased in unnatural ways and so sometimes produces unnatural results. That does not prove they cannot reason, it proves that they cannot reason in a vacuum, and neither can people.

Re: Hallucination is inevitable: An innate limitation of large language models

#423

Post by naasking »

godelski wrote: post #416
xetplan wrote: post #398

Earlier quoted context omitted.

There is no evidence. This entire line of thought is really stupid when chatGPT itself will tell you it doesn't understand the responses it gives. Try the prompt in chatGPT4 "Can you explain to a human how you don't understand anything in the response you give to this question?" For those that don't have chatGPT4. chatGPT4: "Imagine you're using a very advanced calculator that can help you solve all kinds of math pro…

To save people time, here's the inverse > Can you explain to a human how you understand things and respond to this question? GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response…

> It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat.

Doesn't understand what exactly? That seems like a fairly open ended statement and almost certainly wrong as a result. GPT doesn't understand certain things because it hasn't seen those things or anything like it in its training data. How much do you understand about something you've never experienced before?

Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole? It seems fairly obvious that LLMs have been exposed to a very limited slice of the world as projected through natural language, so their understanding will necessarily be limited, but not zero.

Multimodal LLMs noticeably improve reasoning here, and no doubt there are some model improvements yet to come, but people running around making unqualified claims like "LLMs don't understand anything" are just wrong.

Re: Hallucination is inevitable: An innate limitation of large language models

#424

Post by naasking »

sandworm101 wrote: post #163
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns.

Sorry, how do you know that "thinking minds" are not also just "complex pattern-fitting supercomputers hovering over a massive table of precomputed patterns"?

Re: Hallucination is inevitable: An innate limitation of large language models

#425

Post by naasking »

beardedwizard wrote: post #214
fl7305 wrote: post #204

Earlier quoted context omitted.

If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.

And your post was the most probable output of your mind process given your experiences. The only self-evident difference is the richness of your experience as compared to LLMs.

Re: Hallucination is inevitable: An innate limitation of large language models

#426

Post by ninetyninenine »

beardedwizard wrote: post #421
ninetyninenine wrote: post #394

Earlier quoted context omitted.

>There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, thus we say the model doesn't understand. We do that for humans as well. Two things. We also see the model make things that are correct. In fact the mistakes are a minority in comparison to what it got correct. That is in itself an indicator of understanding to…

How can it possibly understand physics when the training data does not teach it or contain the laws of physics?

Video data contains physics. Objects in motion obey the laws of physics. Sora understand physics the same way you understand it.

Re: Hallucination is inevitable: An innate limitation of large language models

#427

Post by grantcas »

avidphantasm wrote: post #42
avidphantasm wrote: post #24

Earlier quoted context omitted.

Perhaps solving hallucinations at the LLM level alone is impossible, hence the inevitability. I reckon that lots of human “hallucination” is simply caught by higher-level control loops operating over the output of the generative mechanism. Basically, our conscious mind says, “nah, that doesn’t look right” enough that most of the time most of us don’t “hallucinate”.

So this implies that instead of spending resources on training bigger and bigger LLMs, AI practitioners need to shift focus to developing “ontological” and “epistemological” control loops to run on top of the LLM. I suspect they already have rudimentary such control loops. In a sense, the “easier” part of AI may be a largely “solved” problem, leaving the development of “consciousness” to be solved, which is obviously…

[dead]

Re: Hallucination is inevitable: An innate limitation of large language models

#428

Post by ninetyninenine »

mannykannot wrote: post #419
ninetyninenine wrote: post #396

Earlier quoted context omitted.

The author of the post is saying that understanding something can't be defined because we can't even know how the human brain works. It is a black box. The author is saying at best you can only set benchmark comparisons. We just assume all humans have the capability of understanding without even really defining the meaning of understanding. And if a machine can mimic human behavior to it must also understand. That is…

What the author of the post actually said - and I am quoting, to make it clear that I'm not putting my spin on someone else's opinion - was "There's no difference between doing something that works without understanding and doing the exact same thing with understanding."

I'm the author. To be clear. I referred to myself as "the author."

And no I did not say that. Let me be clear I did not say that there is "no difference". I said whether there is or isn't a difference we can't fully know because we can't define or know about what "understanding" is. At best we can only observe external reactions to input.

Re: Hallucination is inevitable: An innate limitation of large language models

#429

Post by ninetyninenine »

mannykannot wrote: post #420
ninetyninenine wrote: post #406

Earlier quoted context omitted.

Without a metric no position can be made. All conversation about this topic is just conjecture with no path to a conclusion.

This is a common misunderstanding, one also seen with regard to definitions. When applied to knowledge acquisition, it suffers from a fairly obvious bootstrapping problem, which goes away when you realize that metrics and definitions are rewritten and refined as our knowledge increases. Just look at what has happened to concepts of matter and energy over the last century or so. You are free to disagree with this, but…

No it's not a misunderstanding. Without a concrete definition on a metric comparisons are impossible because everything is based off of wishy washy conjectures on vague and fuzzy concepts. Hard metrics bring in quantitative data. It shows hard differences.

Even if the metric is some side marker where in the future is found to have poor correlation or causation with the the thing being measured the hard metric is still valid.

Take IQ. We assume iq measures intelligence. But in the future we may determine that no it doesn't measure intelligence well. That doesn't change the fact that iq tests still measured something. The score still says something definitive.

My test is similar to the Turing test. But so is yours. In the end there's a human in the loop making a judgment call.

Re: Hallucination is inevitable: An innate limitation of large language models

#430

Post by Anotheroneagain »

p1esk wrote: post #208
Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

Your understanding is distorted by dealing mostly with psychotic people. Iron poisoning (well within the supposed healthy levels) and lead deficiency cause schizophrenia; normal people look childlike or "not intelligent" to the affected.
Post reply on HN