31–40 of 274 posts
> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…
Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction.
In fact this is what learning in general is doing. It means this whole thing about incomplete data applies to human intelligence and learning as well.
Everything this theory is going after basically has application learning and intelligence in general.
So sure you can say that LLMs will always hallucinate. But humans will also always hallucinate.
The real problem that needs to be solved is: how do we get LLMs to hallucinate in the same way humans hallucinate?
OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mi…
Plus, why do we care about that degree? If we could make it so humans don't hallucinate too that would be great, but it ain't happening. Humans memory gets polluted the moment you feed them new information, as evidence by how much care we have to give when trying to extract information when it matters, like law enforcement. People rag on LLMs constantly and i get it, but they then give humans way too much credit imo.…
I'm of the opinion that the current architectures are fundamentally ridden with "hallucinations" that will severely limit their practical usage (including very much what the hype thinks they could do). But this article puts an impossible limit to what it is to "not-hallucinate". It essentially restates well known fundamental limitations of formal systems and mechanistic computation and then presents the trivial resul…
> fundamentally ridden with "hallucinations" that will severely limit their practical usage On the other hand, a LLM that got rid of "hallucinations" is basically just a thing that copy-paste at that point. The interesting properties from LLMs comes from the fact that it can kind of make things up but still make them believable.
OK - there's always a nonzero chance of hallucination. There's also a non-zero chance that macroscale objects can do quantum tunnelling, but no one is arguing that we "need to live with this" fact. A theoretical proof of the impossibility of reaching 0% probability of some event is nice, but in practice it says little about whether we can exponentially decrease the probability of it happening or not to effectively mi…
Exactly. LLMs will sometimes be inaccurate. So are humans. When LLMs are clearly better than humans for specific use cases, we don't need 100% perfection. Autonomous cars will sometimes cause accidents. So do humans. When AVs are clearly safer than humans for specific driving scenarios, we don't need 100% perfection.
Yet any arbitrary degree of error can be dismissed in LLMs because "humans do it too." It's weird.
A human does not do this.
First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them.
Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing it, we relate that thought to other things we know. We may do “sanity checks“ internally, often habitually without even realizing it.
Therefore, we should not expect an LLM to generate the correct answer immediately without giving it space for reflection.
In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. Rarely do you answer a question from just one persona. Instead, most of your answers are the result of internal discussion and compromise.
We also create additional context, such as imagining the consequences of saying the answer we have in mind. Thoughts like that are only possible once an initial “draft” answer is formed in your head.
So, to evaluate the intelligence of an LLM based on its first “gut reaction” to a prompt is probably misguided.
Let me know if you need any further revisions!
It's crazy to me that we managed to get such an exciting technology both theoretically and as a practical tool and still managed to make it into a bubbly hype wave because business people want it to be an automation technology, which is just a poor fit for what they actually do It's kind of cool that we can make mathematical arguments for this, but the idea that generative models can function as universal automation…
They want it to be a wage reduction technology. Everything else you've noticed is a direct consequence of this, and only this, so the analysis doesn't actually need to be any deeper than that.
In that sense, a hallucinating system seems like a promising step towards stronger AI. AI systems simply are lacking a way to test their beliefs against a real world in the way we can, so natural laws, historical information, art and fiction exist on the same epistemological level. This is a problem when integrating them into a useful theory because there is no cost to getting the fundamentals wrong.