Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

321–330 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#321

Post by p1esk »

ebcode wrote: post #316
lazide wrote: post #236

Earlier quoted context omitted.

I think it's more subtly misleading - to be smarter, I need more knowledge. But knowledge != smart, knowledge == informed, or educated. And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. Which arguably isn't about being smart, and is only tangentially about less or more (external) knowledge really. It's about self-know…

>> And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. I think the mistake lies in the belief that the LLM "knows" things. As humans, we have a strong tendency to anthropomorphize. And so, when we see something behave in a certain way, we imagine that thing to be doing the same thing that we do when we behave that way.…

I think when I write, so the machine must also think when it writes.

What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.

Re: Hallucination is inevitable: An innate limitation of large language models

#322

Post by Jensson »

fl7305 wrote: post #287
sandworm101 wrote: post #282

Earlier quoted context omitted.

The simpsons example is for a navigation system, not any AI. It is an analogy, not a test to be put to chatgpt.

So which test can you put to ChatGPT to prove your claim that it is a lookup table, and that it doesn't perform any logic on facts?

There is no such stable test, just like humans can memorize and create simple heuristics to pass any test without understanding so can an LLM. You have probably seen humans that has perfect grades but can't do much in practice, that is how these LLMs work.

The creators of the LLM just feeds it a bunch of edge questions, and whenever people invent new ones they just feed those as well, so proving it doesn't understand will always be a moving target just like making tests that tests peoples understanding is also a moving target since those people will just look at the old tests and practice those otherwise.

Re: Hallucination is inevitable: An innate limitation of large language models

#323

Post by steveBK123 »

digitalsushi wrote: post #199
foobarian wrote: post #181

Earlier quoted context omitted.

Humans have some amount of ability to recognize they hit a wall and adjust accordingly. On the other hand this (completeness theorems, Kolmogorov complexity, complexity theory) was only arrived at what, in the 20th century?

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

Everyone assumes the AI is going to replace their employees but not replace them.. fascinating.

Re: Hallucination is inevitable: An innate limitation of large language models

#324

Post by bonzaidrinkingb »

sschueller wrote: post #7

You have to very carefully ask your question for it to not make things up. For example don't ask "how do I do this in in x?". Ask "can I do this with x?" These "AI" s are like "yes men". They will say anything to please you even if it's untrue or impossible. I have met people like that and they are very difficult to work with. You can't trust that they will deliver the project they promised and you always have to dou…

Current AIs are RLHFd to avoid being a "yes man"/sycophant.

The point about employing better prompting is well taken. Don't ask "Who was the first female president?", ask "Was there ever a female president?". Much like on StackOverflow you want to ask the right question and not assume things (since you don't know enough to make assumptions).

Imagine if every time on early Google you found a spam result and then blame the search engine for that (and not your choice of keywords, or ignoring that you always want to return something, even if remotely related). Like a user banging a slab of concrete with a chisel and complaining that this does not produce a beautiful statue.

Re: Hallucination is inevitable: An innate limitation of large language models

#325

Post by samatman »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology.

When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom.

And no, confabulation isn't a normal thing which humans do, and I don't see how that fact could have anything to do with P != NP. A normal person is aware of the limits of their knowledge, for whatever reason, LLMs are not.

Re: Hallucination is inevitable: An innate limitation of large language models

#326

Post by Jensson »

p1esk wrote: post #321
ebcode wrote: post #316

Earlier quoted context omitted.

>> And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. I think the mistake lies in the belief that the LLM "knows" things. As humans, we have a strong tendency to anthropomorphize. And so, when we see something behave in a certain way, we imagine that thing to be doing the same thing that we do when we behave that way.…

I think when I write, so the machine must also think when it writes. What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.

There are probably many, but the most glaring one is that LLMs has to write a word every time it thinks, meaning it can't solve a problem before it starts to write down the solution. That is an undeniable limitation of current architectures, it means that the way the LLM answers your question also matches its thinking process, meaning that you have to trigger a specific style of response if you want it to be smart with its answer.

Re: Hallucination is inevitable: An innate limitation of large language models

#327

Post by mistermann »

ben_w wrote: post #220
mistermann wrote: post #127

Earlier quoted context omitted.

Humans also have various culturally flavored, implicit "you know what I mean" algorithms on each end to smooth out "irrelevant" misunderstandings and ensure a cordial interaction, a cultural prime directive.

Sure. I think LLMs are good at that kind of thing. My final example demonstrates how those cultural norms cause errors, it was from a logical thinking session at university, where none of the rest of my group could accept my (correct) claim that the answer was "not enough information to answer" even when I gave a (different but also plausible) non-robbery scenario and pointed out that we were in a logical thinking tr…

Don't you think it's strange that humans have little to no interest when root causes to their problems are found?

Re: Hallucination is inevitable: An innate limitation of large language models

#328

Post by samatman »

ninetyninenine wrote: post #294
MauranKilom wrote: post #247

Earlier quoted context omitted.

> even humans hallucinate massively Simpler example: Dreams.

Yeah good point. But dreams are easily distinguishable from reality. Religion is often indistinguishable from truth and reality to those who hallucinate it.

Confusing sincere but incorrect belief with hallucination is categorically wrong.

Re: Hallucination is inevitable: An innate limitation of large language models

#330

Post by Jensson »

ninetyninenine wrote: post #277
beardedwizard wrote: post #217

Earlier quoted context omitted.

I do not understand these comments at all. Sora was trained on billions of frames from video and images - they were tagged with words like "ballistic missile launch" and "cinematic shot" and it simply predicts the pixels like every other model. It stores what we showed it, and reproduces it when we ask - this has nothing to do with understanding and everything to do with parroting. The fact that it's now a stream of…

What is the difference between a machine that for all intents and purposes appears to understand something to a degree of 100 percent versus a human? Both the machine and the human are a black box. The human brain is not completely understood and the LLM is only trivially understood at a high level through the lens of stochastic curve fitting. When something produces output that imitates the output related to a human…

> What is the difference between a machine that for all intents and purposes appears to understand something to a degree of 100 percent versus a human?

There is no such difference, we evaluate that based on their output. We see these massive model make silly errors that nobody who understands it would make, thus we say the model doesn't understand. We do that for humans as well.

For example, for Sora in the video with the dog in the windos, we see the dog walk straight through the window shutters, so Sora doesn't understand physics or depth. We also see it drawing the dogs shadow on the wall very thin, much smaller than the dog itself, it obviously drew that shadow as if it was cast on the ground and not a wall, it would have been very large shadow on that wall. The shadows from the shutters were normal, because Sora are used to those shadows being on a wall.

Hence we can say Sora doesn't understand physics or shadows, but it has very impressive heuristics about those, the dog accurately places its paws on the platforms etc, and the paws shadows were right. But we know those were just basic heuristics since the dog walked through the shutters and its body cast shadow in the wrong way meaning Sora only handles very common cases and fails as soon as things are in an unexpected envionment.

Post reply on HN