Hallucination is inevitable: An innate limitation of large language models • Hacker News Board

Re: Hallucination is inevitable: An innate limitation of large language models

#361

Post by Fripplebubby » Sun, Feb 25, 2024, 9:08 PM UTC

Animats wrote: post #301

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Yes. It looks like they introduce infinities and then run into the halting problem for infinities. That may not be helpful. The place where this argument gets into trouble is where it says "we define hallucination in a formal world where all we care about is a computable ground truth function f on S." This demands a reliable, computable predicate for truth. That alone is probably not possible. If, however, we are wil…

Well put. Overall this paper feels very Gödel Incompleteness for LLMs which is _interesting_ and perhaps even valuable to somebody, but because it attaches itself to this hot query 'hallucination', I think some people are finding themselves searching this paper for information it does not contain.

Re: Hallucination is inevitable: An innate limitation of large language models

#362

Post by nedt » Sun, Feb 25, 2024, 9:09 PM UTC

nedt wrote: post #310

Well humans believe that vacination either kills people or gives them chips for tracking and the top politicians are lizard people drinking the blood of children kept in caves and they had to fake a pandemic to get them out. I'd say an A.I. hallucinating isn't that far off from real humans. It's rather the recipient that needs to interpret any response from either.

Ok I take it back. Hallucination is a problem. Seeing the downvotes and comments here it does seem to be hard to see what is made up and what is just fuxxed up humans. The will to believe that humans can't be that stupid is bigger than I thought and same evaluation might lead to an AI response taken as truth if it's just calm enough and sounds plausible.

Re: Hallucination is inevitable: An innate limitation of large language models

#363

Post by breck » Sun, Feb 25, 2024, 9:12 PM UTC

Sparkyte wrote: post #29

Definitely a given, it isn't like AI has an actual brain capable of resolving and forming new connections. The LLM and human brains is that LLMs are interactive compendiums and our brains organize and sort information that ensures survival as an organism. There is no survival of whether or not LLMs are accurate and a machine wouldn't understand what is good or bad without weighted context. Its good for analyze, proce…

> capable of resolving and forming new connections

> There is no survival of whether or not LLMs are accurate

I agree that today's LLMs are still missing important components like these needed for breakout intelligence, but I would not be surprised if researchers discover how to add them (and other important things) within 0-5 years.

Re: Hallucination is inevitable: An innate limitation of large language models

#364

Post by pixl97 » Sun, Feb 25, 2024, 9:13 PM UTC

dkjaudyeqooe wrote: post #355

p1esk wrote: post #352

Earlier quoted context omitted.

Ok, so how do humans solve a problem? Isn’t it also a sequential, step by step process, even if not expressed explicitly in words? What if instead of words a model would show you images to solve a problem? Would it change anything?

No, I don't know how other people think but I just focus on something and the answer pops into my head. I generally only use a step by step process if I'm following steps given to me.

>but I just focus on something and the answer pops into my head.

It's perfectly valid to say "I don't know", because no one really understand these parts of the human mind.

The point here is saying "Oh the LLM thinks word by word, but I have a magical black box that just works" isn't good science, nor is it a good means of judging what LLMs are capable or not capable of.

Re: Hallucination is inevitable: An innate limitation of large language models

#365

Post by nedt » Sun, Feb 25, 2024, 9:16 PM UTC

bonzaidrinkingb wrote: post #358

nedt wrote: post #348

Earlier quoted context omitted.

In Germany and Austria we have those Querdenker telegram channels. All examples I‘ve given are coming from there. I‘d really like to say I‘ve made it up. But all you did with my message is also what I‘d do with AI output. It can be trained on wrong data, not understanding the question or make stuff up. Just like a human.

I think you are (subconsciously) strawmanning the anti-vax movements like Querdenker. Most of these believe that mandatory vaccination (or reducing freedom of unvaccinated, or making it economically infeasible/required to work) is bad and goes against individual human rights, and that the risks and benefits of vaccines were not clearly communicated. So, even if you did not make it up, it is twisting the viewpoints to…

Source is Die Zeit as written here: https://news.ycombinator.com/item?id=39504716

You could also find it in other sources like Science Busters etc. Most of it will be German, because Germany and Austria does have a real problem with some (dis-)believes in the medical system.

Pretty sure other sources of human halicunations could be given (WMD in Iraq, lot of bad things because of religon, ... ). Point is not the strawman itself, but rather that any message needs evaluation. AI or not.

Re: Hallucination is inevitable: An innate limitation of large language models

#366

Post by pixl97 » Sun, Feb 25, 2024, 9:20 PM UTC

alwa wrote: post #276

fauigerzigerk wrote: post #253

Earlier quoted context omitted.

Perhaps I'm not using the vocabulary correctly here. What I mean is, if you ask a human to solve a travelling salesman problem and they find it too hard to solve exactly, they will still be able to come up with a better than average solution. This is what I called approximation (but maybe this is incorrect?). Hallucination would be to choose a random solution and claim that it's the optimum.

I may be misunderstanding the way LLM practitioners use the word “hallucination,” but I understood it to describe it as something different from the kind of “random” nonsense-word failures that happen, for example, when the temperature is too high [0]. Rather, I thought hallucination, in your example, might be something closer to a grizzled old salesman-map-draftsman’s folk wisdom that sounds like a plausibly optimal…

> but it will be nonetheless false.

Only if you're assuming all questions have binary answers.

For example in the traveling salesman problem you don't have to compute all answers to start converging on an average. A random sampling of solutions can start setting a bounds for average, and your grizzled salesmans guesses would fall somewhere on that plot. If they are statistically better than average then they are far more than good enough. Unless of course you think burning up the observable universe in finding the best solution is the only way to solve the problem of which trip uses the least gas?

Re: Hallucination is inevitable: An innate limitation of large language models

#367

Post by jiggawatts » Sun, Feb 25, 2024, 9:25 PM UTC

navane wrote: post #344

samatman wrote: post #325

Earlier quoted context omitted.

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and…

When you talk to your mom and you remember something happening one way, and she remembers it another way, but you both insist you remember it correctly, one of you is doing what the LLM is doing (filling up gaps of knowledge with bull shit). And even when later you talk about this on meta level, no one calls this confabulation because no one uses that word. Also this is not a psychiatric syndrome, it's just people ma…

Just ask any criminal attorney or police detective. Normal people can’t get their facts straight even if they all witnessed something memorable first-hand just hours ago.

Re: Hallucination is inevitable: An innate limitation of large language models

#368

Post by NateEag » Sun, Feb 25, 2024, 9:25 PM UTC

ninetyninenine wrote: post #182

Fiction and story writing is hallucination. It is the opposite of a stochastic parrot. We've achieved both extremes of AI. Computers can be both logical machines and hallucinators. Our goal is to create a machine that can be both at the same time and can differentiate between both. That's the key. Hallucination is important but the key is for the computer to be self aware about when it's hallucinating. Of course it's…

> Only one religion can be right or none, that must mean,logically speaking all other religions are hallucinations.

There are some mistakes in this sentence.

It is possible (if unlikely) that multiple religions accurately describe some aspects of the world, while being mistaken about others. That is, treating rigorous complete "correctness" as the only useful state a religion could have is very misleading. Newtonian physics and special relativity both fail to predict some observed phenomena, but they're still both useful (and not every religion claims rigorous perfect correctness, even if some do).

Even if some religions can be shown to be wrong, that doesn't automatically mean that they're hallucinations. People can believe things for plausible reasons and be wrong about them.

People can also have reasonable stances like "I cannot prove this is true, and would not try to, but my subjective personal experience of visions of God persuade me it's probably real."

That seems very different to me from an LLM hallucinating a paper from whole cloth out of the blue.

Re: Hallucination is inevitable: An innate limitation of large language models

#369

Post by pixl97 » Sun, Feb 25, 2024, 9:31 PM UTC

someplaceguy wrote: post #261

dragonwriter wrote: post #249

Earlier quoted context omitted.

> Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I cou…

> Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I couldn't solve”. Are you saying that LLMs can't learn to discriminate between which questions they should answer "I don't know"…

>Are you saying that LLMs can't learn to discriminate between which questions they should answer "I don't know" vs which questions they should try to provide an accurate answer?

This is highly problematic and highly contextualized statement.

Imagine you're an accountant with the piece of information $x. The answer you give for the statement "What is $x" is going to be highly dependent on who is answering the question. For example

1. The CEO asks "What is $x"

2. A regulator at the SEC asks "What is $x

3. Some random individual or member of the press asks "What is $x"

An LLM doesn't have the other human motivations a person does when asked questions, pretty much at this point with LLMs there are only one or two 'voices' it hears (system prompt and user messages).

Whereas a human will commonly lie and say I don't know, it's somewhat questionable if we want LLMs intentionally lying.

In addition human information is quite often compartmentalized to keep secrets which is currently not in vogue with LLMs as we are attempting to make oracles that know everything with them.

Re: Hallucination is inevitable: An innate limitation of large language models

#370

Post by ebcode » Sun, Feb 25, 2024, 9:37 PM UTC

p1esk wrote: post #321

ebcode wrote: post #316

Earlier quoted context omitted.

>> And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. I think the mistake lies in the belief that the LLM "knows" things. As humans, we have a strong tendency to anthropomorphize. And so, when we see something behave in a certain way, we imagine that thing to be doing the same thing that we do when we behave that way.…

I think when I write, so the machine must also think when it writes. What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.

That's a difficult question to answer, since I must be doing a lot of very different things while thinking. For one, I'm not sure I'm never not thinking. Is thinking different from "brain activity"? We can shut down the model, store it on disk, and boot it back up. Shut down my brain and I'm a goner.

I'm open to saying that the machine is "thinking", but I do think we need more clear language to distinguish between machine thinking and human thinking.

EDIT: I chose the wrong word with "thinking", when I was trying to point out the logical fallacy of anthropomorphizing the machine. It would have been more clear if I had used the word "breathing": When I write I'm breathing, so the machine must also be breathing.