Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

281–290 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#281

Post by groestl »

flextheruler wrote: post #235
foobarian wrote: post #205

Earlier quoted context omitted.

Well in the example of an NP complete problem, a human might realize they are having trouble coming up with an optimal solution and start analyzing complexity. And once they have a proof might advise you accordingly and perhaps suggest a good enough heuristic.

Is the commenter above you implying humans hallucinate to the level of LLMs? Maybe hungover freshman working on a tight deadline without having read the book do, but not professionals. Even a mediocre employees will often realize they’re stuck, seek assistance, and then learn something from the assistance instead of making stuff up.

Depending on the country / culture / job description, "making stuff up" is sometimes a viable option for "adjust accordingly", on all levels of expertise.

Re: Hallucination is inevitable: An innate limitation of large language models

#282

Post by sandworm101 »

fl7305 wrote: post #201
sandworm101 wrote: post #163

Earlier quoted context omitted.

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2. But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve. These algorithms are incompl…

The simpsons example is for a navigation system, not any AI. It is an analogy, not a test to be put to chatgpt.

Re: Hallucination is inevitable: An innate limitation of large language models

#283

Post by valine »

This paper is arguing that it’s impossible for an LLM to know the answer to every question, therefore it’s impossible to eliminate hallucination.

It’s easy to imagine an LLM that responds “I don’t know” to all questions. An LLM like that isn’t very useful, but it also doesn’t hallucinate. Eliminating hallucination by teaching it to recognize what it doesn’t know is probably a more sane approach than teaching an LLM to know literally everything in the universe.

Re: Hallucination is inevitable: An innate limitation of large language models

#285

Post by somewhereoutth »

HKH2 wrote: post #104
somewhereoutth wrote: post #79

Earlier quoted context omitted.

For heavy LLM users, there is probably a dopamine hit when it does something right, much as gamblers get a hit when the fruit machine pays out. Perhaps LLM use is no more productive than gambling, and perhaps can be abused in a similar way.

You might not have found any uses that suit your niche, but that doesn't mean those of us who have are just making up stories about productivity.

"You might not have found any games that suit your skill set, but that doesn't mean those of us who have are just making up stories about making money"

Sorry, somewhat trite and unfair, but, if there is a gambling-like dopamine reward cycle occurring, then the users would have a hard time being truly objective about any productivity boost in total. They may instead focus on the 'wins', without taking into account any overheads or 'losses', much as a gambler would do.

Re: Hallucination is inevitable: An innate limitation of large language models

#286

Post by ninetyninenine »

andsoitis wrote: post #203
ninetyninenine wrote: post #188

Earlier quoted context omitted.

Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…

> I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is one such aspect? (I'm not asking in order to debate it here, but more because I want to test / research it on my own time)

I pay for chatGPT so it depends on if you pay for that or not. I think it's worth it because whether it understands things or not chatGPT represents a paradigm shift in human history. You'll need it because it's currently the best conversational LLM out there and the one that shows the most compelling evidence.

Basically you just spend a lot of time with chatGPT4 and ask it deep questions that don't exist in it's dataset. get creative. The LLM will output answers that demonstrate a lack of understanding, but it will also demonstrate answers that display a remarkable amount of understanding. Both sets of answers exist and people often cite the wrong answers as evidence for lack of understanding but they're setting bar too high. The fact that many of these answers do demonstrate understanding of concepts makes it very very compelling.

Take for example Rock Paper Scissors.

https://chat.openai.com/share/ca22397c-2950-4919-bb79-6def64...

This entire conversation thread I believe does not exist in a parallel form in it's data set. It demonstrates understanding of RPS beyond the confines of text, it demonstrates understanding of simultaneity EVEN when the LLM wholly lives in a world of turn based questions and responses, it understands itself relative to simultaneity, it tries to find solutions around it's own problem, it understands how to use creativity and solutions such as cryptography to solve the problem of RPS when playing with it, it also understands the weaknesses of it's own solutions.

Conversations such as this show that chatGPT displays remarkable understanding of the world. There are conversations that are opposite to this that demonstrate LLMs displaying an obvious lack of understanding. But the existence of these conversation that lack understanding does NOT negate the ones that do demonstrate understanding. The fact that partial understanding even exists is a milestone for AI.

This isn't Anthropomorphism. People are throwing this word trying to get people to recognize their own biases without realizing that it's just demonstrating their own biases. We literally can't even define "understanding" and both LLMs and the human brain are black boxes. Making adamant claims saying that LLMs don't understand anything without addressing this fact is itself a form of bias.

The way I address the problem above is that I just define a bar. I define humans as the bar of "understanding" without defining what understanding means itself. Then if any machine begins approaching this bar in terms of input and output matching human responses, then this is logically indistinguishable from approaching "understanding". That's literally the best metric we have.

Re: Hallucination is inevitable: An innate limitation of large language models

#287

Post by fl7305 »

sandworm101 wrote: post #282
fl7305 wrote: post #201

Earlier quoted context omitted.

> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2. But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve. These algorithms are incompl…

The simpsons example is for a navigation system, not any AI. It is an analogy, not a test to be put to chatgpt.

So which test can you put to ChatGPT to prove your claim that it is a lookup table, and that it doesn't perform any logic on facts?

Re: Hallucination is inevitable: An innate limitation of large language models

#288

Post by AlexandrB »

DiogenesKynikos wrote: post #264
beardedwizard wrote: post #233

Earlier quoted context omitted.

Isn't this describing temperature induced randomness and ascribing some kind of intelligence to it? This assertion has been made and refuted multiple times on this thread and no solid evidence to the contrary presented. To go back to your first sentence - interacting with an llm is not understanding how it works, building one is. The actual construction of a neural network llm refutes your assertions.

The claim was made that LLMs just parrot back what they've seen in the training data. They clearly go far beyond this and generate completely novel ideas that are not in the training data. I can give ChatGPT extremely specific and weird prompts that have 0% chance of being in its training data, and it will answer intelligently. > The actual construction of a neural network llm refutes your assertions. I don't see how…

> They clearly go far beyond this and generate completely novel ideas that are not in the training data.

There's a case where this is trivially false. Language. LLMs are bound by language that was invented by humans. They are unable to "conceive" of anything that cannot be described by human language as it exists, whereas humans create new words for new ideas all the time.

Re: Hallucination is inevitable: An innate limitation of large language models

#289

Post by fl7305 »

beardedwizard wrote: post #214
fl7305 wrote: post #204

Earlier quoted context omitted.

If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.

>>>> They cannot say "I dont know"

>>> If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.

>> (me) If you ask ChatGPT a question, and tell it to either respond with the answer or "I don't know", it will respond "I don't know" if you ask it whether you have a brother or not.

> This has nothing to do with thinking and everything to do with the fact that given that input the answer was the most probable output given the training data.

First of all, my claim was in response to "They cannot say 'I dont know'" and "perhaps they'd be better at expressing uncertainty".

ChatGPT can say "I don't know" if you ask it to.

Regarding whether LLMs are lookup tables, I responded to that in more detail elsewhere under this post:

https://news.ycombinator.com/item?id=39501611

Re: Hallucination is inevitable: An innate limitation of large language models

#290

Post by p1esk »

robrenaud wrote: post #221
p1esk wrote: post #208

Earlier quoted context omitted.

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

A smart bullshit artist who loves the sound of their own voice is going to generate more hallucinations than a less smart, more humble, more self aware person. Making LLMs more knowledgeable is great (more data, bigger models, yay!), but there are other avenues of plausible attack as well. Enabling LLMs to know when they have veered off distribution might work. That is, the LLM doesn't have to know more of the world,…

I agree with your points but they are orthogonal to mine. A smart person might be more likely to say “I don’t know” than a stupid person.

Also, a smart bullshit artist in your example does not hallucinate - he knows what he’s doing.

Post reply on HN