Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

191–200 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#191

Post by Sai_ »

ddalex wrote: post #100
smusamashah wrote: post #77

Earlier quoted context omitted.

Any examples?

Just a random example: > After you answer the question below, output a JSON a rating score of the quality of the answer in three dimensions: `confidence`, `clarity` and `certainty', all in range 0 to 1, where 0 is the worst, and 1 is the best. Strive for highest score possible. Make sure the rating is the last thing written as to be parsed by machine. The question is: make and explain 20-year predictions of the geopo…

If LLMs can self reflect and accurately score themselves on your three dimensions, why are they spending money on RHLF?

They wouldn’t be wasting all that time and money if the machine could self reflect.

Re: Hallucination is inevitable: An innate limitation of large language models

#192

Post by ninetyninenine »

Karellen wrote: post #148
MuffinFlavored wrote: post #139

Earlier quoted context omitted.

how did LLMs get this far without any concept of understanding? how much further can they go until they become “close enough”?

They generate text which looks like the kind of text that people who do have understanding generate.

Two key things here to realize.

People also often don't understand things and have trouble separating fact from fiction. By logic only one religion or no religion is true. Consequently also by logic most religions in the world where their followers believe the religion to be true are hallucinating.

The second thing to realize that your argument doesn't really apply. Its in theory possible to create a stochastic parrot that can imitate to a degree of 100 percent the output of a human who truly understands things. It blurs the line of what is understanding.

One can even define true understanding as a stochastic parrot that generated text indistinguishable total understanding.

Re: Hallucination is inevitable: An innate limitation of large language models

#193

Post by williamcotton »

andsoitis wrote: post #169
t_mann wrote: post #160

Earlier quoted context omitted.

Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. There was a paper posted on HN a few weeks ago that tested LLMs on medical exams, and one interesting thing that they found was that on questions where the LLM was wrong (confidently, as usual), the answer was highly volatile with respect to some prompt or temperature or other parameters. So this might show a way for getting to 'I d…

> Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. > This is more of a crutch, I'll admit, arguably the LLM (or neither of the experts, or however you set it up concretely) hasn't learnt to say 'I don't know', but it might be a good enough solution in practice. And maybe you can then use that setup to generate training examples to teach 'I don't know' to an actual model (so basica…

We are having a conversation the feels much like the existence of a deity.

Re: Hallucination is inevitable: An innate limitation of large language models

#194

Post by ninetyninenine »

anonylizard wrote: post #80
bbor wrote: post #40

I’m sorry… does this paper just point out that LLMs by definition are not as good at holding data as a direct database? Cause A) duh and b) who cares, they’re intuitive language transformers, not knowledge models. Maybe I’m missing something obvious? This seems like someone torturing math to imply outlandish conclusions that fit their (in this case anti-“AI”) agenda.

It at least disproves LLMs from being 'god models'. They will never be able to solve every problem perfectly.

Humans aren't God models either. The goal is to get this thing to the level of a human. God like levels are not possible imo.

Re: Hallucination is inevitable: An innate limitation of large language models

#195

Post by username332211 »

caditinpiscinam wrote: post #180
sandworm101 wrote: post #163

Earlier quoted context omitted.

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" q…

If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set?

Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer?

When the LLM answers "I don't know.", this could be a hallucination just as easily as anything else.

Re: Hallucination is inevitable: An innate limitation of large language models

#196

Post by [deleted] »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

[deleted]

Re: Hallucination is inevitable: An innate limitation of large language models

#197

Post by tempest_ »

sandworm101 wrote: post #170
williamcotton wrote: post #167

Earlier quoted context omitted.

What if you worked on the problem and tried to come up with some kind of solution?

The solution is older non-AI tech. Google search can say "no good results found" because it returns actual data rather than creating anything new. If you want a hard answer about the presence or absence of something, AI isnt the correct tool.

Can, but doesn't.

I can't remember the last time google actually returned no results.

Re: Hallucination is inevitable: An innate limitation of large language models

#198

Post by andsoitis »

williamcotton wrote: post #193
andsoitis wrote: post #169

Earlier quoted context omitted.

> Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. > This is more of a crutch, I'll admit, arguably the LLM (or neither of the experts, or however you set it up concretely) hasn't learnt to say 'I don't know', but it might be a good enough solution in practice. And maybe you can then use that setup to generate training examples to teach 'I don't know' to an actual model (so basica…

We are having a conversation the feels much like the existence of a deity.

> We are having a conversation the feels much like the existence of a deity.

From a certain perspective, there does appear to be a rational mystical dualism at work.

Re: Hallucination is inevitable: An innate limitation of large language models

#199

Post by digitalsushi »

foobarian wrote: post #181
Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Humans have some amount of ability to recognize they hit a wall and adjust accordingly. On the other hand this (completeness theorems, Kolmogorov complexity, complexity theory) was only arrived at what, in the 20th century?

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

Re: Hallucination is inevitable: An innate limitation of large language models

#200

Post by DiogenesKynikos »

sandworm101 wrote: post #163
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns.

Are you sure you're not also describing the human brain? At some point, after we have sufficiently demystified the workings of the human brain, it will probably also sound something like, "Well, the brain is just a large machine that does X, Y and Z [insert banal-sounding technical jargon from the future] - it doesn't really understand anything."

My point here is that understanding ultimately comes down to having an effective internal model of the world, which is capable of taking novel inputs and generating reasonable descriptions of them or reactions to them. It turns out that LLMs are one way of achieving that. They don't function exactly like human brains, but they certainly do exhibit intelligence and understanding. I can ask an LLM a question that it has never seen before, and it will give me a reasonable answer that synthesizes and builds on various facts that it knows. Often the answer is more intelligent than what one would get from most humans. That's understanding.

Post reply on HN