Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

41–50 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#41

Post by golol »

Sparkyte wrote: post #29

Definitely a given, it isn't like AI has an actual brain capable of resolving and forming new connections. The LLM and human brains is that LLMs are interactive compendiums and our brains organize and sort information that ensures survival as an organism. There is no survival of whether or not LLMs are accurate and a machine wouldn't understand what is good or bad without weighted context. Its good for analyze, proce…

The result in the theorem applies to your brain. Your brain can be modelled as an LLM in the sense of the paper up to arbitrarily small error.

The result is a diagonalization argument that is not very relevant for the real world.

Re: Hallucination is inevitable: An innate limitation of large language models

#42

Post by avidphantasm »

avidphantasm wrote: post #24
FeepingCreature wrote: post #11

It seems a stretch to call it "inevitable". "Inevitable given the current architecture without modifications" at most. Also, I'm missing a section on how (if) human brains manage to avoid hallucinations in this. Also, it doesn't have to never hallucinate, it just has to hallucinate less than we do.

Perhaps solving hallucinations at the LLM level alone is impossible, hence the inevitability. I reckon that lots of human “hallucination” is simply caught by higher-level control loops operating over the output of the generative mechanism. Basically, our conscious mind says, “nah, that doesn’t look right” enough that most of the time most of us don’t “hallucinate”.

So this implies that instead of spending resources on training bigger and bigger LLMs, AI practitioners need to shift focus to developing “ontological” and “epistemological” control loops to run on top of the LLM. I suspect they already have rudimentary such control loops. In a sense, the “easier” part of AI may be a largely “solved” problem, leaving the development of “consciousness” to be solved, which is obviously the hard part.

Re: Hallucination is inevitable: An innate limitation of large language models

#44

Post by audunw »

carlossouza wrote: post #31

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.

True, but I think we can fall into the trap of expecting too much of LLMs. Their knowledge can seem perfect. They can answer almost anything, so it’s easy to get the illusion that they can answer anything truthfully.

In terms of what we can expect of future improvements, I think it’s overly optimistic to expect any kind of super intelligence beyond what we see today (that is, having access to all the worlds publicly available information, or rapidly generating texts/images/videos that fall into existing creative patterns).

I suspect that more creative intelligence requires an extremely fine balance to not “go crazy”.. that is, producing output we’d consider creative rather than hallucinations.

I think getting this balance right will get exponentially harder as we create feedback loops within the AI that let its intelligence evolve.

And it’s entirely possible that humans have already optimised this creative intelligence feedback loop as much as the universe allows. Having a huge amount of knowledge can obviously benefit from more neurons/storage. But we simply don’t know if that’s true for creative intelligence yet

Re: Hallucination is inevitable: An innate limitation of large language models

#45

Post by timini »

az09mugen wrote: post #37
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

I think there is no easy way to make an LLM answer "I don't know". For that, it should learn among all the stuff ingested when people effectively don't know. But most people on internet write down irrelevant stuff even when they don't know instead of simply writing "I don't know". That's a very good point.

I think its fairly simple, it needs a certain level of proof e.g references to authoritative sources, if not say "i don't know".

Re: Hallucination is inevitable: An innate limitation of large language models

#46

Post by exe34 »

resolutebat wrote: post #28
FeepingCreature wrote: post #11

It seems a stretch to call it "inevitable". "Inevitable given the current architecture without modifications" at most. Also, I'm missing a section on how (if) human brains manage to avoid hallucinations in this. Also, it doesn't have to never hallucinate, it just has to hallucinate less than we do.

Humans do hallucinate, there's lots of literature on how memories are distorted, we see and hear things we want to see and hear, etc. The particular pathology of LLMs is that they're literally incapable of distinguishing facts from hallucinations even in the most mundane circumstances: if a human is asked to summarize the quarterly results of company X, unlike an LLM they're highly unlikely to recite a convincing but…

And yet if you ask a random person at a rally about their favourite cause of the day, they usually spew sound bites that are factually inaccurate, and give all impressions of being as earnest and confident as the LLM making up quarterly results.

Re: Hallucination is inevitable: An innate limitation of large language models

#48

Post by vladms »

LightBug1 wrote: post #35
carlossouza wrote: post #31

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.

I imagine the gold is in knowing whether the LLM understands when it's doing either? Isn't this the difference between a human and an LLM? A human knows it's making an educated guess and (should) say so. Or it knows when it's being creative, and can say so. If it doesn't know which is which, then it really does bring it home that LLM's are not that much more than (very sophisticated) mechanical input-output machines.

You mean "some humans know and could say so". And this reflection process is not captured in the data we fed to LLM-s (like let's say a lesson in which teacher asks "do you know X?", and students first answer "I don't know", etc.)

Also, LLM-s could report more statistical measures for each answer and external tools could interpret them.

Re: Hallucination is inevitable: An innate limitation of large language models

#49

Post by az09mugen »

timini wrote: post #45
az09mugen wrote: post #37

Earlier quoted context omitted.

I think there is no easy way to make an LLM answer "I don't know". For that, it should learn among all the stuff ingested when people effectively don't know. But most people on internet write down irrelevant stuff even when they don't know instead of simply writing "I don't know". That's a very good point.

I think its fairly simple, it needs a certain level of proof e.g references to authoritative sources, if not say "i don't know".

I'm really curious about one would implement that. By pondering weigths from certain sources ?

Re: Hallucination is inevitable: An innate limitation of large language models

#50

Post by Certhas »

timini wrote: post #45
az09mugen wrote: post #37

Earlier quoted context omitted.

I think there is no easy way to make an LLM answer "I don't know". For that, it should learn among all the stuff ingested when people effectively don't know. But most people on internet write down irrelevant stuff even when they don't know instead of simply writing "I don't know". That's a very good point.

I think its fairly simple, it needs a certain level of proof e.g references to authoritative sources, if not say "i don't know".

LLMs don't have a concept of sources for their statements.

Ask them to give you some literature recommendations on something it has explained to you. You'll get plenty of plausible sounding papers that don't exist.

Humans know to some extent why they know (read it in a text book, colleague mentioned it). LLMs don't seem to.

Post reply on HN