171–180 of 491 posts
Based on what we know about different systems in the brain it might be a stretch to expect hallucination free AGI, using only a single LLM.
I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…
They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…
They define an LLM as "a probabilistic model of a string that conditions the output at time t based on all the tokens that come before it in the string". I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans.
In other words, it can be a good feature request or idea.
Earlier quoted context omitted.
The result in the theorem applies to your brain. Your brain can be modelled as an LLM in the sense of the paper up to arbitrarily small error. The result is a diagonalization argument that is not very relevant for the real world.
>your brain can be modelled as an LLM in the sense of the paper The vast majority of people actually writing LLMs don’t claim this, and in fact, actually claim the very opposite: that LLMs do not accurately model a human brain in any capacity. The fact is that science has no clue what happens in the nucleus of a neuron, so claiming that computer scientists must is… well. You fill in the word.
For an arbtrarily large duration and an arbitrarily small error with respect to any definition of error you choose, there exists an LLM in the above sense which models the dynamics of your brain with that small error.
Earlier quoted context omitted.
Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. There was a paper posted on HN a few weeks ago that tested LLMs on medical exams, and one interesting thing that they found was that on questions where the LLM was wrong (confidently, as usual), the answer was highly volatile with respect to some prompt or temperature or other parameters. So this might show a way for getting to 'I d…
> Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. > This is more of a crutch, I'll admit, arguably the LLM (or neither of the experts, or however you set it up concretely) hasn't learnt to say 'I don't know', but it might be a good enough solution in practice. And maybe you can then use that setup to generate training examples to teach 'I don't know' to an actual model (so basica…
I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in this paper, and I don't see an obvious connection, since commonsense hallucinations (eg inventing chapters of a novel when asked to produce summaries, inventing specific details when asked) don't seem to be NP-complete problems but rather are hallucinatory for some other interesting reason. (I apologize if I have not captured the paper correctly and would welcome correction on that, I read it quickly)
The statement that the formal world (the world of math and logic and formal grammars) is a subset of the "real" world (or perhaps, the world of natural language) is really interesting to me as well. Most humans can't solve formal logic problems and parse formal grammars but don't suffer from a (strong) hallucination effect, and can work in natural language in great proficiency. Is hallucination inevitable in humans since we also can't solve certain NP-complete problems? We have finite lifespans, after all, so even with the capabilities we might never complete a certain problem.
Complete AI layman here but it seems to me that the human mind's architecture has an overarching "executive" module that is involved in managing the numerous "expert" systems that do other stuff for us. (split brain experiments demonstrate multiple "minds" in the brain) Based on what we know about different systems in the brain it might be a stretch to expect hallucination free AGI, using only a single LLM.
Based on what we know about brains, it might be a stretch to expect hallucination free AGI. I’ve yet to meet a general intelligence that doesn’t hallucinate.
Lots of fun examples from chickens who turn off when they see a straight line to cats who get spooked by cucumbers and the endless fun cognitive biases humans fall for.
Earlier quoted context omitted.
Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. There was a paper posted on HN a few weeks ago that tested LLMs on medical exams, and one interesting thing that they found was that on questions where the LLM was wrong (confidently, as usual), the answer was highly volatile with respect to some prompt or temperature or other parameters. So this might show a way for getting to 'I d…
> Maybe it requires understanding, maybe there are other ways to get to 'I don't know'. > This is more of a crutch, I'll admit, arguably the LLM (or neither of the experts, or however you set it up concretely) hasn't learnt to say 'I don't know', but it might be a good enough solution in practice. And maybe you can then use that setup to generate training examples to teach 'I don't know' to an actual model (so basica…
I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…
They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…
Maybe LLMs don't truly "understand" questions, but they're good at looking like they understand questions. If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.