I think part of understanding is filling in the gaps between facts. AIs can't recognize when that gap is too large
Hallucination is inevitable: An innate limitation of large language models
271–280 of 491 posts
Re: Hallucination is inevitable: An innate limitation of large language models
#272Earlier quoted context omitted.
The solution is older non-AI tech. Google search can say "no good results found" because it returns actual data rather than creating anything new. If you want a hard answer about the presence or absence of something, AI isnt the correct tool.
Can, but doesn't. I can't remember the last time google actually returned no results.
ah, i get no results pages often when i search for quoted error strings from many different sources. Thing is, I have a hard time believing that no one has actually talked about at least some of those errors :(
Re: Hallucination is inevitable: An innate limitation of large language models
#273I think understanding comes from filling the gaps between the facts. AIs can't tell when the gap between points of knowledge is too large to interpolate.
Re: Hallucination is inevitable: An innate limitation of large language models
#274I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…
They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…
> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match
Can you tell that's not how you yourself function?
Re: Hallucination is inevitable: An innate limitation of large language models
#275Earlier quoted context omitted.
Humans have some amount of ability to recognize they hit a wall and adjust accordingly. On the other hand this (completeness theorems, Kolmogorov complexity, complexity theory) was only arrived at what, in the 20th century?
'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?
One thing a human might do that I’ve never seen an LLM do is ask followup and clarifying questions to determine what is actually being requested.
Re: Hallucination is inevitable: An innate limitation of large language models
#276Earlier quoted context omitted.
You cannot approximate NP-complete functions. If you could approximate them with a practically useful limited error and at most P effort you would have solved P=NP. (disclaimer my computer science classes have been a long time ago)
Perhaps I'm not using the vocabulary correctly here. What I mean is, if you ask a human to solve a travelling salesman problem and they find it too hard to solve exactly, they will still be able to come up with a better than average solution. This is what I called approximation (but maybe this is incorrect?). Hallucination would be to choose a random solution and claim that it's the optimum.
I may be misunderstanding the way LLM practitioners use the word “hallucination,” but I understood it to describe it as something different from the kind of “random” nonsense-word failures that happen, for example, when the temperature is too high [0].
Rather, I thought hallucination, in your example, might be something closer to a grizzled old salesman-map-draftsman’s folk wisdom that sounds like a plausibly optimal mapping strategy to a boss oblivious to the mathematical irreducibility of the problem. Imagining a “fact” that sounds plausible and is rhetorically useful, but that’s never been true and nobody ever said was true.
It’ll still be, like your human in the example, better than average (if “average” means averaged across the universe of all possible answers), and maybe even useful enough to convince the people reading the output, but it will be nonetheless false.
Re: Hallucination is inevitable: An innate limitation of large language models
#277Earlier quoted context omitted.
Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…
I do not understand these comments at all. Sora was trained on billions of frames from video and images - they were tagged with words like "ballistic missile launch" and "cinematic shot" and it simply predicts the pixels like every other model. It stores what we showed it, and reproduces it when we ask - this has nothing to do with understanding and everything to do with parroting. The fact that it's now a stream of…
What is the difference between a machine that for all intents and purposes appears to understand something to a degree of 100 percent versus a human?
Both the machine and the human are a black box. The human brain is not completely understood and the LLM is only trivially understood at a high level through the lens of stochastic curve fitting.
When something produces output that imitates the output related to a human that we claim "understands" things that is objectively understanding because we cannot penetrate the black box of human intelligence or machine intelligence to determine further.
In fact in terms of image generation the LLM is superior. It will generate video output superior to what a human can generate.
Now mind you the human brain has a classifier and can identify flaws but try watching a human with Photoshop to try to even draw one frame of those videos.. it will be horrible.
Does this indicate that humans lack understanding? Again, hard to answer because we are dealing with black boxes so it's hard to pinpoint what understanding something even means.
We can however set a bar. A metric. And we can define that bar as humans. all humans understand things. Any machine that approaches human input and output capabilities is approaching human understanding.
Re: Hallucination is inevitable: An innate limitation of large language models
#278Earlier quoted context omitted.
Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…
If by “understand” you mean “can model reasonably accurately much of the time” then maybe you’ll find consensus. But that’s not a universal definition of “understand”. For example, if I asked you whether you “understand” ballistic flight, and you produced a table that you interpolate from instead of a quadratic, then I would not feel that you understand it, even though you can kinda sorta model it. And even if you do…
Well what would you need to see to prove understanding? That's the metric here. Both the LLM and the human brain are black boxes. But we claim the human brain understands things while the LLM does not.
Thus what output would you expect for either of these boxes to demonstrate true understanding to your question?
Re: Hallucination is inevitable: An innate limitation of large language models
#279Earlier quoted context omitted.
Ok, but LLMs are just tools, and I'm just asking how a tool can be made more useful. It doesn't really matter why an LLM tells you to go look elsewhere, it's simply more useful if it does than if it hallucinates. And usefulness isn't binary, getting the error rate down is also an improvement.
> Ok, but LLMs are just tools, and I'm just asking how a tool can be made more useful. I think I know what you're after (notice my self-awareness to qualify what I say I know): that the tool's output can be relied upon without applying layers of human judgement (critical thinking, logical reasoning, common sense, skepticism, expert knowledge, wisdom, etc.) There are a number of boulders in that path of clarity. One o…
Yeah I don't think it will ever make sense to think about Transformer models as 'understanding' something. The approach that I suggested would replace that with rather simple logic like answer_variance > arbitrary_threshold ? return 'I don't know' : return $original_answer
It's not a fundamental fix, it doesn't even change the model itself, but the output might be more useful. And then there was just some speculation how you could try to train a new AI mimicking the more useful output. I'm sure smarter people than me can come up with way smarter approaches. But it wouldn't have to do with understanding - when I said the tool should return 'I don't know' above, I literally meant it should return that string (maybe augmented a bit by some pre-defined prompt), like a meaningless symbol, not any result of anything resembling introspection.
Re: Hallucination is inevitable: An innate limitation of large language models
#280Fiction and story writing is hallucination. It is the opposite of a stochastic parrot. We've achieved both extremes of AI. Computers can be both logical machines and hallucinators. Our goal is to create a machine that can be both at the same time and can differentiate between both. That's the key. Hallucination is important but the key is for the computer to be self aware about when it's hallucinating. Of course it's…
Comparing religion to LLM mistakes is a case of the very prevalent anthropomorphism in society currently. I fear this misunderstanding and conflation will prevent us actually improving the tech. Coming to an incorrect explanation such as, “Helios pulls the Sun across the sky every day,” is categorically different than a math program incorrectly returning the most likely next token in a sequence. LLMs don’t have belie…
No. I never said we figured out cognition.
The LLM is still a black box feed forward network. It is the intricacies of how signals interact with each other in this network that we don't fully understand.
Word prediction and curve fitting are high level concepts we used to build an intelligence we don't fully understand.
Also belief and understanding are orthogonal concepts. I can believe something I don't understand and I can understand something I don't belief.
My claim here is that LLMs understand things from a certain aspect because LLMs can produce output indistinguishable from understanding.
Also because both the human brain and the LLM are black boxes there is no other metric we can use to gauge the level of understanding an LLM has other than comparing it's inputs and outputs to the human brain.