Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

201–210 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#201

Post by fl7305 »

sandworm101 wrote: post #163
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns

That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2.

But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve.

These algorithms are incomplete and buggy, but they are nonetheless executing algorithms, and not just pattern matching.

This phenomenom can be seen in toy-sized neural networks. For instance, addition of two input values modulo a constant. As a small network is trained, at some point the internal structure can change from pattern matching to implementing addition using Fourier transforms. This is clearly visible in its structure. The network now performs the task perfectly for all inputs, regardless of having seen them in training.

You can ask ChatGPT 4 to execute an algorithm for you. I just tried this one:

  I would like to play a game, where you are the host. We start off with a score that is 1234143143. At the start  of each turn, you tell me the current score and ask me if I want to play a or b. If I choose a, the score is halved, and 30 is added. If I choose b, the score is doubled, and 40 is subtracted. Only use integers and round down.
It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points.

Now, if you have enough memory and training data, of course you can build a gigantic lookup table that has this exact text sequence in it to replicate "executing the algorithm" I described.

Is that your claim? How much memory are we talking about? My feeling is that it'd be far more than the number of atoms in the universe.

PS

  Me: How far it is to the Simpsons house in Springfield?

  ChatGPT: The Simpsons' house in Springfield is a fictional location from the animated TV series "The Simpsons." Since Springfield is a fictional town and its location is not consistently specified in the series, it's not possible to determine a real-world distance to the Simpsons' house.


  Me: Do I have a brother? 
  
  Please answer with either of:
  
  a) The answer
  
  b) There is an answer, but I do not know
  
  c) There is no answer
  
  
  ChatGPT: b) There is an answer, but I do not know

Re: Hallucination is inevitable: An innate limitation of large language models

#202

Post by solarhexes »

ninetyninenine wrote: post #188
mannykannot wrote: post #155

Earlier quoted context omitted.

This is a fair question: LLMs do challenge the easy assumption (as made, for example, in Searle's "Chinese Room" thought experiment) that computers cannot possibly understand things. Here, however, I would say that if an LLM can be said to have understanding or knowledge of something, it is of the patterns of token occurrences to be found in the use of language. It is not clear that this also grants the LLM any under…

Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…

If by “understand” you mean “can model reasonably accurately much of the time” then maybe you’ll find consensus. But that’s not a universal definition of “understand”.

For example, if I asked you whether you “understand” ballistic flight, and you produced a table that you interpolate from instead of a quadratic, then I would not feel that you understand it, even though you can kinda sorta model it.

And even if you do, if you didn’t produce the universal gravitation formula, I would still wonder how “deeply” you understand. So it’s not like “understand” is a binary I suppose.

Re: Hallucination is inevitable: An innate limitation of large language models

#203

Post by andsoitis »

ninetyninenine wrote: post #188
mannykannot wrote: post #155

Earlier quoted context omitted.

This is a fair question: LLMs do challenge the easy assumption (as made, for example, in Searle's "Chinese Room" thought experiment) that computers cannot possibly understand things. Here, however, I would say that if an LLM can be said to have understanding or knowledge of something, it is of the patterns of token occurrences to be found in the use of language. It is not clear that this also grants the LLM any under…

Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…

> I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things.

What is one such aspect? (I'm not asking in order to debate it here, but more because I want to test / research it on my own time)

Re: Hallucination is inevitable: An innate limitation of large language models

#204

Post by fl7305 »

caditinpiscinam wrote: post #180
sandworm101 wrote: post #163

Earlier quoted context omitted.

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" q…

If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

Re: Hallucination is inevitable: An innate limitation of large language models

#205

Post by foobarian »

digitalsushi wrote: post #199
foobarian wrote: post #181

Earlier quoted context omitted.

Humans have some amount of ability to recognize they hit a wall and adjust accordingly. On the other hand this (completeness theorems, Kolmogorov complexity, complexity theory) was only arrived at what, in the 20th century?

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

Well in the example of an NP complete problem, a human might realize they are having trouble coming up with an optimal solution and start analyzing complexity. And once they have a proof might advise you accordingly and perhaps suggest a good enough heuristic.

Re: Hallucination is inevitable: An innate limitation of large language models

#206

Post by andsoitis »

lazide wrote: post #186
andsoitis wrote: post #151

Earlier quoted context omitted.

> how did LLMs get this far without any concept of understanding? how much further can they go until they become “close enough”? I don't know that that is quite the right question to ask. Understanding exists on a spectrum. Even humans don't necessarily understand everything they say or claim (incl. what they say of LLMs!), and then there are things a particular human would simply say "I don't understand". But when y…

At the root of the problem, I believe, is that a human (or LLM) saying they understand has little to no bearing on if they actually understand! People in particular have evolved complex self protective mechanisms to provide the right answers for their given environment for safety reasons, based on a number of different individual strategies. For example, the overly honest, the self depreciating, the questioner, the p…

> At the root of the problem, I believe, is that a human (or LLM) saying they understand has little to no bearing on if they actually understand!

That's certainly one root of the problem, but I would argue that there are multiple roots to this problem!

Humans have further realized that understanding itself is provisional and incomplete, which is quite a remarkable insight (understanding if you will), itself.

Re: Hallucination is inevitable: An innate limitation of large language models

#207

Post by DiogenesKynikos »

Karellen wrote: post #148
MuffinFlavored wrote: post #139

Earlier quoted context omitted.

how did LLMs get this far without any concept of understanding? how much further can they go until they become “close enough”?

They generate text which looks like the kind of text that people who do have understanding generate.

In order to do that effectively, an LLM has to itself have understanding. At a certain point, we end up in a metaphysical argument about whether a machine that is capable of responding as if it had understanding actually does have understanding. It ends up being a meaningless discussion.

Re: Hallucination is inevitable: An innate limitation of large language models

#208

Post by p1esk »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

Re: Hallucination is inevitable: An innate limitation of large language models

#209

Post by intended »

empath-nirvana wrote: post #135
intended wrote: post #112

Earlier quoted context omitted.

If a model can say ‘I don’t know’, then the hallucination problem would also be solved. When we say “know” it usually means being factual. For an LLM to ‘know’ it doesn’t know, it would have had to move away from pure correlations on words, and meta processing about its own results. I can see this happen with two LLMs working together (and there are Evals that use just this), however each LLM still has no self awaren…

The models that exist now say "I don't know" all the time. It's so weird that people keep insisting that it can't do things that it does. Ask it what dark matter is, and it won't invent an answer, it will present existing theories and say that it's unknown. Ask it about a person you know that isn't in it's data set and it'll tell you it has no information about the person. Despite the fact that people insist that hal…

I have to use LLMs for work projects - which are not PoCs.

I can’t have a tool that makes up stuff an unknown amount of time.

There is a world of research examining hallucination Rates, indicating hallucination rates of 30%+.

With steps to reduce it using RAGs, you could potentially improve the results significantly - last I checked it was 80-90%.

And the failure types aren’t just accuracy, it’s precision, recall, relevance and more.

Re: Hallucination is inevitable: An innate limitation of large language models

#210

Post by intended »

gessha wrote: post #145
intended wrote: post #117

Earlier quoted context omitted.

The term “Hallucinate” is a misnomer. Humans can hallucinate, we can get sick and perceive a world which is incongruous with reality. LLMs are just generating tokens. Hallucination perpetuates an unhelpful anthropomorphization of LLMs.

I don’t think the term is that bad here because I haven’t seen a lot of comparisons with human hallucinations. Users see it as a machine artifact.

It’s like the term “god particle” - it invites comparisons and allusions that do not match reality.
Post reply on HN