Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

341–350 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#341

Post by samatman »

Almondsetat wrote: post #102

Of course it's inevitable. Things can be facts or deductions of facts (or both). If I ask an LLM the date of birth of Napoleon and it doesn't have it in its dataset there are only 2 options: either it has other facts from which Napoleon's birthday can be deduced or it doesn't. If it does then by improving the LLM we will be able to make more and more deductions, it if doesn't then it can only hallucinate. Since there…

I don't see why that's inevitable at all. I immediately recognize that I don't happen to know Napoleon's birthday, and act on that basis.

Re: Hallucination is inevitable: An innate limitation of large language models

#342

Post by cultureswitch »

DiogenesKynikos wrote: post #207
Karellen wrote: post #148

Earlier quoted context omitted.

They generate text which looks like the kind of text that people who do have understanding generate.

In order to do that effectively, an LLM has to itself have understanding. At a certain point, we end up in a metaphysical argument about whether a machine that is capable of responding as if it had understanding actually does have understanding. It ends up being a meaningless discussion.

That is the argument we're having though.

The central claim is that a machine which answers exactly the same thing a human would answer given the same input does not have understanding, while the human does.

This claim is religious, not scientific. In this worldview, "understanding" is a property of humans which can't be observed but exists nonetheless. It's like claiming humans have a soul.

Re: Hallucination is inevitable: An innate limitation of large language models

#343

Post by bonzaidrinkingb »

dragonwriter wrote: post #338
bonzaidrinkingb wrote: post #333

Earlier quoted context omitted.

Around 12k fatal outcomes have been reported in the EU after vaccination, but it is not certain in all cases that vaccines were the cause. The vaccine tracking chips come from two Microsoft (-affiliate) patents, one about using chips to track body activity to reward in cryptocurrency, and another about putting a vaccine passport chip in the hands of African immigrants. That vaccines contain tracking chips is a fabric…

> Real humans do hallucinate all the time. No, they don't hallucinate “all the time”, but LLM “hallucination” is a bad metaphor, as the phenomenon is more like confabulation than hallucination. Humans also don’t confabulate all the time, either, though.

> "Everyone experiences hallucinations," Anil Seth, a neuroscientist at the University of Sussex in the UK, told DW.

> "It's important to recognize hallucinations can come and go during our lives at points of stress or tiredness," Seth said. "There is a bit of a stigma around hallucinations. It comes from people associating them with mental illness and being called crazy."

> But it's actually very common and happens even daily. The itching Yarwood experiences is particularly common, especially after drinking alcohol.

> "It's also common for people with reduced hearing or vision function to get hallucinations in that ear or eye," said Rick Adams, a psychiatrist at University College London. "These are non-clinical hallucinations because they are not associated with a psychiatric diagnosis."

https://www.dw.com/en/hallucinations-are-more-common-than-yo...

Confabulation is more like making something up when you don't have sufficient knowledge. Seems to happen regularly :)

Re: Hallucination is inevitable: An innate limitation of large language models

#344

Post by navane »

samatman wrote: post #325
Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and…

When you talk to your mom and you remember something happening one way, and she remembers it another way, but you both insist you remember it correctly, one of you is doing what the LLM is doing (filling up gaps of knowledge with bull shit). And even when later you talk about this on meta level, no one calls this confabulation because no one uses that word. Also this is not a psychiatric syndrome, it's just people making shit up, inadvertently, to tell a coherent story without holes. It very much sounds you did the same. Everyone does this all the time.

Re: Hallucination is inevitable: An innate limitation of large language models

#345

Post by cultureswitch »

nkrisc wrote: post #131
mike_hearn wrote: post #83

Earlier quoted context omitted.

Ask a human to provide accurate citations for any random thing they know and they won't be able to do a good job either. They'd probably have to search to find it, even if they know they got it from a document originally and have some clear memory of what it said.

But they could, if they needed to. But most people don’t need to, so they don’t keep that information in their brains. I can’t tell you the date of every time I clip my toenails, but if I had to could remember it.

LLMs can remember their sources. It's just additional knowledge, there's nothing special about it.

When you ask an LLM to tell you the height of Mount Everest, it clearly has a map of mountains to heights, in some format. Using exactly the same mapping structure, it can remember a source document for the height.

Re: Hallucination is inevitable: An innate limitation of large language models

#346

Post by bitwize »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

> but rather are hallucinatory for some other interesting reason.

In improv theater, the actor's job is to come up with plausible interactions. They are free to make shit up as they go along, hence improv, but they have to keep their inventions plausible to what had just happened before. So in improv if someone asks you "What is an eggplant?" it is perfectly okay to say "An eggplant is what you get when you genetically splice together an egg and a cucumber" or similar. It's nonsense but it's nonsense that follows nicely from what just came before.

Large language models, especially interactive ones, are a kind of improv theater by machine: the machine outputs something statistically plausible to what had just come before; what "statistically plausible" means is based on the data about human conversations that came from the internet. But if there are gaps in the data, or the data lacks a specific answer that seems to statistically dominate, it seems like giving a definitive answer is more plausible in the language model than saying "I don't know", so the machine selects definitive, but wrong, answers.

Re: Hallucination is inevitable: An innate limitation of large language models

#347

Post by solarhexes »

cultureswitch wrote: post #336
solarhexes wrote: post #202

Earlier quoted context omitted.

If by “understand” you mean “can model reasonably accurately much of the time” then maybe you’ll find consensus. But that’s not a universal definition of “understand”. For example, if I asked you whether you “understand” ballistic flight, and you produced a table that you interpolate from instead of a quadratic, then I would not feel that you understand it, even though you can kinda sorta model it. And even if you do…

Are you telling me that WW1 artillery crews didn't understand ballistics? Because they were using tables. There's no difference between doing something that works without understanding and doing the exact same thing with understanding.

You’ve decided that your definition of “understanding” is correct. Ok.

Re: Hallucination is inevitable: An innate limitation of large language models

#348

Post by nedt »

bonzaidrinkingb wrote: post #333
nedt wrote: post #310

Well humans believe that vacination either kills people or gives them chips for tracking and the top politicians are lizard people drinking the blood of children kept in caves and they had to fake a pandemic to get them out. I'd say an A.I. hallucinating isn't that far off from real humans. It's rather the recipient that needs to interpret any response from either.

Around 12k fatal outcomes have been reported in the EU after vaccination, but it is not certain in all cases that vaccines were the cause. The vaccine tracking chips come from two Microsoft (-affiliate) patents, one about using chips to track body activity to reward in cryptocurrency, and another about putting a vaccine passport chip in the hands of African immigrants. That vaccines contain tracking chips is a fabric…

In Germany and Austria we have those Querdenker telegram channels. All examples I‘ve given are coming from there. I‘d really like to say I‘ve made it up. But all you did with my message is also what I‘d do with AI output. It can be trained on wrong data, not understanding the question or make stuff up. Just like a human.

Re: Hallucination is inevitable: An innate limitation of large language models

#349

Post by steveBK123 »

Jensson wrote: post #331
steveBK123 wrote: post #323

Earlier quoted context omitted.

Everyone assumes the AI is going to replace their employees but not replace them.. fascinating.

Uber proves we can replace Taxi management with simple algorithms, that was apparently much easier than replacing the drivers. I hope these bigger models can replace management in more industries, I'd love to have an AI as a manager.

Yeah on the one hand people think AI management id dystopian (probably lol), on the other hand probably fewer than 50% of ICs promoted to management are good at it.

North of 25% are genuinely bad at it. We've all worked for several of these.

Many of us have tried our hand at management and then moved back to senior IC tracks. Etc.

Re: Hallucination is inevitable: An innate limitation of large language models

#350

Post by jncfhnb »

jijijijij wrote: post #334
jncfhnb wrote: post #308

Earlier quoted context omitted.

> They cannot say "I dont know" because they dont actually know anything. print(“I don’t know”) You don’t need proper cognition to identify that the answer is not stored in source data. Your conception of the model is incomplete as is easily demonstrable by testing such cases now. Chat gpt does just fine on your simpsons test. You, however, have made up an answer of how something works that you don’t actually know de…

> to identify that the answer is not stored in source data How would an LLM do that?

They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic.

How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning.

I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the answer isn’t known.

Post reply on HN