Hallucination is inevitable: An innate limitation of large language models
1–10 of 491 posts
Re: Hallucination is inevitable: An innate limitation of large language models
#2> hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function.
That's simply inaccuracy or fabrication.
Labelling it hallucination simply panders to the idea these programs are intelligent.
Re: Hallucination is inevitable: An innate limitation of large language models
#3The models are just generating probable text. What’s amazing of how often the text is correct. It’s no surprise at all when it’s wrong
Their bold confidence to be flat out wrong may be their most human trait
Re: Hallucination is inevitable: An innate limitation of large language models
#4The models are just generating probable text. What’s amazing of how often the text is correct. It’s no surprise at all when it’s wrong Their bold confidence to be flat out wrong may be their most human trait
This again.
They’re trained to generate probable text. The mechanisms created in the parameter blob during training to do that are basically a mystery and have to be pulled out of the model with digital brain surgery. E.g. LLMs are reasonable at chess and turns out somewhere in the blob there’s a chessboard representation, and you can make the model believe the board is in a different state by tweaking those parameters.
So yeah they generate probable text, sure. Where they get the probabilities is a very good research problem.
Re: Hallucination is inevitable: An innate limitation of large language models
#5The models are just generating probable text. What’s amazing of how often the text is correct. It’s no surprise at all when it’s wrong Their bold confidence to be flat out wrong may be their most human trait
I hope to remember your last line for the rest of my life when I think about AI.
Re: Hallucination is inevitable: An innate limitation of large language models
#6The models are just generating probable text. What’s amazing of how often the text is correct. It’s no surprise at all when it’s wrong Their bold confidence to be flat out wrong may be their most human trait
paulsutter said:
> Note that this is the single most human attribute of LLMs
It might be if LLM hallucinations looked like or occurred at the same frequency as human hallucinations do, but they don’t.
Re: Hallucination is inevitable: An innate limitation of large language models
#7You have to very carefully ask your question for it to not make things up. For example don't ask "how do I do this in in x?". Ask "can I do this with x?"
These "AI" s are like "yes men". They will say anything to please you even if it's untrue or impossible.
I have met people like that and they are very difficult to work with. You can't trust that they will deliver the project they promised and you always have to double check everything. You also can't trust them that what they promised is even possible.
Re: Hallucination is inevitable: An innate limitation of large language models
#8This is just another diagonalization against some finite thing, like the no free lunch theorem. An "LLM" in their definition is essentially any finite thing which predicts the next token. The same result applies to your brain too, for example.
Like all such diagonalization results, it is not really relevant for real world considerations. The reason is that it does not matter if your model fails on none, finitely many or infinitely many inputs. In reality the space of possible inputs is equipped with a probability measure, and the size of the hallucinating inputs set w.r.t. that measure is relevant. Diagonalization arguments usually, make no claim to the size of that set, and it is most likely negligible in the real world.
Re: Hallucination is inevitable: An innate limitation of large language models
#9You have to very carefully ask your question for it to not make things up. For example don't ask "how do I do this in in x?". Ask "can I do this with x?" These "AI" s are like "yes men". They will say anything to please you even if it's untrue or impossible. I have met people like that and they are very difficult to work with. You can't trust that they will deliver the project they promised and you always have to dou…
[deleted]
Re: Hallucination is inevitable: An innate limitation of large language models
#10The models are just generating probable text. What’s amazing of how often the text is correct. It’s no surprise at all when it’s wrong Their bold confidence to be flat out wrong may be their most human trait
paulsutter said: > Note that this is the single most human attribute of LLMs It might be if LLM hallucinations looked like or occurred at the same frequency as human hallucinations do, but they don’t.
You could make an argument that what we currently see are effectively internal monologues. It is extremely hard to evaluate how much subconscious or conscious filtering happens between a human's internal state and the eventual outbound communications, but I wouldn't be at all surprised if the upstream hallucination rate in humans was much higher than you'd think.
By analogy to Kahneman and Tversky's System 1 and System 2, the whole field of Prospect Theory is about how often System 1 is wrong. This feels connected.