Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

261–270 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#261

Post by someplaceguy »

dragonwriter wrote: post #249
someplaceguy wrote: post #243

Earlier quoted context omitted.

> their function is to produce text output which forms a plausible seeming response to the question posed Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. And it would not be a hallucination.

> Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I cou…

> Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I couldn't solve”.

Are you saying that LLMs can't learn to discriminate between which questions they should answer "I don't know" vs which questions they should try to provide an accurate answer?

Sure, there will be an error rate, but surely you can train an LLM to minimize it?

Re: Hallucination is inevitable: An innate limitation of large language models

#263

Post by DiogenesKynikos »

c22 wrote: post #248
DiogenesKynikos wrote: post #228

Earlier quoted context omitted.

> It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that re…

You claim that logical and reasonable responses "require understanding" therefore LLMs must understand . But I see LLMs as evidence that understanding is not required to produce logical and reasonable responses. Thinking back to when I used to help tutor some of my peers in 101-level math classes there were many times someone was able to produce a logical and reasonable response to a problem (by rote use of an algori…

Then your definition of understanding is meaningless. If a physical system is able to accurately simulate understanding, it understands.

Re: Hallucination is inevitable: An innate limitation of large language models

#264

Post by DiogenesKynikos »

beardedwizard wrote: post #233
DiogenesKynikos wrote: post #228

Earlier quoted context omitted.

> It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that re…

Isn't this describing temperature induced randomness and ascribing some kind of intelligence to it? This assertion has been made and refuted multiple times on this thread and no solid evidence to the contrary presented. To go back to your first sentence - interacting with an llm is not understanding how it works, building one is. The actual construction of a neural network llm refutes your assertions.

The claim was made that LLMs just parrot back what they've seen in the training data. They clearly go far beyond this and generate completely novel ideas that are not in the training data. I can give ChatGPT extremely specific and weird prompts that have 0% chance of being in its training data, and it will answer intelligently.

> The actual construction of a neural network llm refutes your assertions.

I don't see how. There's a common view that I see expressed in these discussions, that if the workings of an LLM can be explained in a technical manner, then it doesn't understand. "It just uses temperature induced randomness, etc. etc." Once we understand how the human brain works, it will then be possible to argue, in the exact same way, that humans do not understand. "You see, the brain is just mechanically doing XYZ, leading to the vocal cords moving in this particular pattern."

Re: Hallucination is inevitable: An innate limitation of large language models

#265

Post by sshumaker »

renegade-otter wrote: post #254
p1esk wrote: post #208

Earlier quoted context omitted.

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

You’re being downvoted because this is a hot take that isn’t supported by evidence.

I just tried exactly that with dalle-3 and it worked well.

More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they reason about things. There was some good experiments on this a while back - check out the Othello experiment.

https://thegradient.pub/othello/

Re: Hallucination is inevitable: An innate limitation of large language models

#266

Post by timeon »

ninetyninenine wrote: post #182

Fiction and story writing is hallucination. It is the opposite of a stochastic parrot. We've achieved both extremes of AI. Computers can be both logical machines and hallucinators. Our goal is to create a machine that can be both at the same time and can differentiate between both. That's the key. Hallucination is important but the key is for the computer to be self aware about when it's hallucinating. Of course it's…

> Just look at religion.

This is bit off-topic but what I see as one of driving force behind existence of religions is need for personification. It seems easier for human to interact with the world and its elements by communicating with it as it was familiar parson-like entity.

Now when we talk about LLMs and AI in general, there is often personification as well.

Re: Hallucination is inevitable: An innate limitation of large language models

#267

Post by kenjackson »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Last I’d heard it was still open if P != NP. And most questions I’ve seen hallucinations on are not NP-Complete.

Re: Hallucination is inevitable: An innate limitation of large language models

#268

Post by moyix »

kalkin wrote: post #251
thargor90 wrote: post #241

Earlier quoted context omitted.

You cannot approximate NP-complete functions. If you could approximate them with a practically useful limited error and at most P effort you would have solved P=NP. (disclaimer my computer science classes have been a long time ago)

This isn't correct. What you may be remembering is that some (not all) NP complete problems have limits on how accurately they can be approximated (unless P = NP). But approximation algorithms for NP complete problems form a whole subfield of CS.

The theorem that proves this is the PCP Theorem, in case anyone wants to read more about it: https://en.wikipedia.org/wiki/PCP_theorem#PCP_and_hardness_o...

Re: Hallucination is inevitable: An innate limitation of large language models

#269

Post by ninetyninenine »

andsoitis wrote: post #211
ninetyninenine wrote: post #192

Earlier quoted context omitted.

Two key things here to realize. People also often don't understand things and have trouble separating fact from fiction. By logic only one religion or no religion is true. Consequently also by logic most religions in the world where their followers believe the religion to be true are hallucinating. The second thing to realize that your argument doesn't really apply. Its in theory possible to create a stochastic parro…

> People also often don't understand things and have trouble separating fact from fiction. That's not the point being argued. Understanding, critical thinking, knowledge, common sense, etc. all these things exist on a spectrum - both in principle and certainly in humans. In fact, in any particular human there are different levels of competence across these dimensions. What we are debating, is whether or not, an LLM c…

Of course it can. Simply ask the LLM about itself. chatGPT4 can answer.

In fact. That question is one of the more trivial questions it will most likely not hallucinate on.

The reason why I alluded to humans here is because I'm saying we are setting the bar too high. It's like everyone is saying it hallucinates and therefore it can't understand anything. I'm saying that we hallucinate too and because of that LLMs can approach humans and human level understanding.

Re: Hallucination is inevitable: An innate limitation of large language models

#270

Post by HDThoreaun »

renegade-otter wrote: post #254
p1esk wrote: post #208

Earlier quoted context omitted.

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.
Post reply on HN