Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

311–320 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#311

Post by DinaCoder99 »

PopePompus wrote: post #185
Culonavirus wrote: post #111

Earlier quoted context omitted.

The hype is insane. Listen, I think LLMs still have a lot of room to grow and they're already very useful, but like some excellent researchers say, they're not the holy grail. If we want AGI, LLMs are not it. A lot of people seem to think this is an engineering issue and that LLMs can get us there, but they can't, because it is not an engineering issue.

I don't think you can say with confidence that the LLM approach will not lead to AGI, unless you understand in detail how human intelligence operates, and can show that no modification to current LLM architectures can achieve the same or superior results. I think the fact that adding "attention" to LLMs made a huge difference means that we are probably still in the low hanging fruit stage of LLM architecture developm…

Well, both the cognitive scientists and linguists seem very doubtful we can apply this model to human cognition and yield much of value, so I'd say the idea that this model can yield behavior analogous to human cognition without other mechanisms seems rather far-fetched.

Of course, we should absolutely pursue better understanding of both as to not throw the baby out with the bath water, but I'm not personally placing much hope in finding AGI any time soon.

Re: Hallucination is inevitable: An innate limitation of large language models

#312

Post by earthwalker99 »

DinaCoder99 wrote: post #307

"Hallucination" implies perception of non-real things, not generation of phrases that map poorly to reality (or are simply incoherent). It seems like a really bad term for this phenomenon.

It makes being wrong sound impressive and mysterious, so it's here to stay.

Re: Hallucination is inevitable: An innate limitation of large language models

#313

Post by katehikes88 »

nedt wrote: post #310

Well humans believe that vacination either kills people or gives them chips for tracking and the top politicians are lizard people drinking the blood of children kept in caves and they had to fake a pandemic to get them out. I'd say an A.I. hallucinating isn't that far off from real humans. It's rather the recipient that needs to interpret any response from either.

Have you considered that parts of what you said might be true but you ridicule it only because you associate with the others might be untrue and maybe even ridiculous?

Re: Hallucination is inevitable: An innate limitation of large language models

#314

Post by username332211 »

caditinpiscinam wrote: post #229
username332211 wrote: post #195

Earlier quoted context omitted.

If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as ea…

> Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velo…

The thing is, the usefulness of a question answering system is in answering questions people don't generally know. We don't need an answering system for things that are common knowledge.

And it's not uncommon that certain knowledge would be, well uncommon even among experts. Experts specialize.

Since the usefulness of ornithological examples is getting exhausted, let's say one out of a hundred lawyers works in bankruptcy. If you ask a million lawyers about the provisions of 11 USC § 1129 and only ten thousand know the answer, is the answer untrustworthy, just because bankruptcy lawyers are far rarer than civil and criminal lawyers?

Re: Hallucination is inevitable: An innate limitation of large language models

#315

Post by c22 »

DiogenesKynikos wrote: post #263
c22 wrote: post #248

Earlier quoted context omitted.

You claim that logical and reasonable responses "require understanding" therefore LLMs must understand . But I see LLMs as evidence that understanding is not required to produce logical and reasonable responses. Thinking back to when I used to help tutor some of my peers in 101-level math classes there were many times someone was able to produce a logical and reasonable response to a problem (by rote use of an algori…

Then your definition of understanding is meaningless. If a physical system is able to accurately simulate understanding, it understands.

My definition of understanding is not meaningless, but it appears you do not understand it.

Re: Hallucination is inevitable: An innate limitation of large language models

#316

Post by ebcode »

lazide wrote: post #236
FpUser wrote: post #222

Earlier quoted context omitted.

You post amounts to: in order to be smarter I need to increase my smartness. Great insight.

I think it's more subtly misleading - to be smarter, I need more knowledge. But knowledge != smart, knowledge == informed, or educated. And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. Which arguably isn't about being smart, and is only tangentially about less or more (external) knowledge really. It's about self-know…

>> And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers.

I think the mistake lies in the belief that the LLM "knows" things. As humans, we have a strong tendency to anthropomorphize. And so, when we see something behave in a certain way, we imagine that thing to be doing the same thing that we do when we behave that way.

I'm writing, and the machine is also writing, but what I'm doing when I write is very different from what the machine does when it writes. So the mistake is to say, or think, "I think when I write, so the machine must also think when it writes."

We probably need to address the usage of the word "hallucination", and maybe realize that the LLM is always hallucinating.

Not: "When it's right, it's right, but when it's wrong, it's hallucinating." It's more like, "Sweet! Some of these hallucinations are on point!"

Re: Hallucination is inevitable: An innate limitation of large language models

#317

Post by bonzaidrinkingb »

DinaCoder99 wrote: post #307

"Hallucination" implies perception of non-real things, not generation of phrases that map poorly to reality (or are simply incoherent). It seems like a really bad term for this phenomenon.

We use "confabulation".

It is a feature, not a bug.

"confabulation" could be "solved" when LLMs realize they are uncertain on a reply and making things up. But that requires the LLM saying "I don't know" and rewarding that more than a wrong guess. That requires a change in loss functions and not even sure if all users desire that.

Re: Hallucination is inevitable: An innate limitation of large language models

#318

Post by Jensson »

DiogenesKynikos wrote: post #263
c22 wrote: post #248

Earlier quoted context omitted.

You claim that logical and reasonable responses "require understanding" therefore LLMs must understand . But I see LLMs as evidence that understanding is not required to produce logical and reasonable responses. Thinking back to when I used to help tutor some of my peers in 101-level math classes there were many times someone was able to produce a logical and reasonable response to a problem (by rote use of an algori…

Then your definition of understanding is meaningless. If a physical system is able to accurately simulate understanding, it understands.

A human that mimics the speech of someone that does understand usually doesn't understand himself. We see that happen all the time with real humans, you have probably seen that as well.

To see if a human understands we ask them edge questions and things they probably haven't seen before, and if they fail there but just manage for common things then we know the human just faked understanding. Every LLM today fails this, so they don't understand, just like we say humans don't understand that produces the same output. These LLM has superhuman memory so their ability to mimic smart humans is much greater than a human faker, but other than that they are just like your typical human faker.

Re: Hallucination is inevitable: An innate limitation of large language models

#319

Post by [deleted] »

tonnydourado wrote: post #126
ddalex wrote: post #71

Earlier quoted context omitted.

Actually it seems to me that they do... I asked via custom prompts the various GPTs to give me scores for accuracy, precision and confidence for its answer (in range 0-1), and then I instructed them to stop generating when they feel the scores will be under .9, which seems to pretty much stop the hallucination. I added this as a suffix to my queries.

People really need to understand that your single/double digit dataset of interactions with an inherently non-deterministic process is less than irrelevant. It's saying that global warming isn't real because it was really cold this week. I don't even know enough superlatives to express how irrelevant it is that "it seems to you" that an LLM behaves this way or that. And even the "protocol" in question is weak. Self r…

[deleted]

Re: Hallucination is inevitable: An innate limitation of large language models

#320

Post by fauigerzigerk »

alwa wrote: post #276
fauigerzigerk wrote: post #253

Earlier quoted context omitted.

Perhaps I'm not using the vocabulary correctly here. What I mean is, if you ask a human to solve a travelling salesman problem and they find it too hard to solve exactly, they will still be able to come up with a better than average solution. This is what I called approximation (but maybe this is incorrect?). Hallucination would be to choose a random solution and claim that it's the optimum.

I may be misunderstanding the way LLM practitioners use the word “hallucination,” but I understood it to describe it as something different from the kind of “random” nonsense-word failures that happen, for example, when the temperature is too high [0]. Rather, I thought hallucination, in your example, might be something closer to a grizzled old salesman-map-draftsman’s folk wisdom that sounds like a plausibly optimal…

If a driver is tasked with visiting a number of places, they will probably choose a reasonably good route. If the driver claims to have found the optimal route, it may not be true, but it's still not a hallucination and it's still a pretty good route.

The driver certainly cannot be relied on to always find an exact solution to an NP-complete problem. But failure modes matter. For practical purposes, the driver's solution is not simply "false". It's just suboptimal.

If we could get LLMs to fail in a similarly benign way, that would make them far more robust without disproving what the posted paper claims.

Post reply on HN