Someone smart once said:
If it is good, we call it "creativity."
If it is bad, we call it "hallucination."
This isn't a bug (or limitation, as the authors say). It's a feature.
Hallucination is inevitable: An innate limitation of large language models
31–40 of 491 posts
Re: Hallucination is inevitable: An innate limitation of large language models
#32Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.
Asking it to write code for you is basically asking it to hallucinate.
Re: Hallucination is inevitable: An innate limitation of large language models
#33> hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. With this definition, you can trivially prove the titular sentence - "hallucination is inevitable" - is untrue. Let your LLM have a fixed input context length of one byte. Continue training the LLM until such a time as it replies to the input "A" with "yes" and all other inputs with "no". Define your computa…
> Continue training the LLM until such a time as it replies to the input "A" with "yes" and all other inputs with "no". This is basically the same as saying "train your LLM until they never hallucinate", which reduces your claim to a tautology: an LLM trained not to hallucinate does not hallucinate. The trick is making that happen.
As I always tell my students: the solution to unreliable code is not to put the bugs in there in the first place.
Re: Hallucination is inevitable: An innate limitation of large language models
#34Ah, the AI hype is now entering the "let's be real" phase. Haven't seen a frenzied post on alignment in a while now.
Re: Hallucination is inevitable: An innate limitation of large language models
#35Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.
I imagine the gold is in knowing whether the LLM understands when it's doing either?
Isn't this the difference between a human and an LLM?
A human knows it's making an educated guess and (should) say so. Or it knows when it's being creative, and can say so.
If it doesn't know which is which, then it really does bring it home that LLM's are not that much more than (very sophisticated) mechanical input-output machines.
Re: Hallucination is inevitable: An innate limitation of large language models
#36> hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. That's simply inaccuracy or fabrication. Labelling it hallucination simply panders to the idea these programs are intelligent.
That statement is also a bit easy on the "computable ground truth." There is no such thing for the tasks we use an LLM for (unless you make up some definition that mangles the definition of each word).
Re: Hallucination is inevitable: An innate limitation of large language models
#37I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…
I think there is no easy way to make an LLM answer "I don't know". For that, it should learn among all the stuff ingested when people effectively don't know. But most people on internet write down irrelevant stuff even when they don't know instead of simply writing "I don't know".
That's a very good point.
Re: Hallucination is inevitable: An innate limitation of large language models
#38Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.
Asking it to write code for you is basically asking it to hallucinate.
I don't think so. I think it's asking it to repeat code it has been trained on
Re: Hallucination is inevitable: An innate limitation of large language models
#39I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…
I can assure you it has no relevance for people working with LLMs, as the result includes your brain, for example.
Re: Hallucination is inevitable: An innate limitation of large language models
#40I’m sorry… does this paper just point out that LLMs by definition are not as good at holding data as a direct database? Cause A) duh and b) who cares, they’re intuitive language transformers, not knowledge models.
Maybe I’m missing something obvious? This seems like someone torturing math to imply outlandish conclusions that fit their (in this case anti-“AI”) agenda.