Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

241–250 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#241

Post by thargor90 »

fauigerzigerk wrote: post #237
Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Does the paper distinguish between hallucination and approximation? If LLMs could be trained to approximate NP-complete functions rather than making stuff up, that would be good enough in many contexts. I guess it's what humans would do.

You cannot approximate NP-complete functions. If you could approximate them with a practically useful limited error and at most P effort you would have solved P=NP. (disclaimer my computer science classes have been a long time ago)

Re: Hallucination is inevitable: An innate limitation of large language models

#242

Post by Karellen »

DiogenesKynikos wrote: post #207
Karellen wrote: post #148

Earlier quoted context omitted.

They generate text which looks like the kind of text that people who do have understanding generate.

In order to do that effectively, an LLM has to itself have understanding. At a certain point, we end up in a metaphysical argument about whether a machine that is capable of responding as if it had understanding actually does have understanding. It ends up being a meaningless discussion.

I am reminded of Feynman's story about teaching physics in one Brazil university, one semester, a few decades ago:

http://v.cx/2010/04/feynman-brazil-education

The students learned to repeat the text of the books, without "understanding" what the books were describing. I'm sure this says something about one side or the other of this conundrum, but I'm not sure which. :-)

Re: Hallucination is inevitable: An innate limitation of large language models

#243

Post by someplaceguy »

dragonwriter wrote: post #232
Fripplebubby wrote: post #227

Earlier quoted context omitted.

Yes, I think you're right. I think one way to phrase the authors' argument is: * There is a class of problems which are harder than polynomial time complexity to solve, but are not np-complete * LLMs will generate an "answer" in formal language to this class of problems posed to it * LLMs can at most solve problems with polynomial time complexity due to their fundamental design and principles * Therefore, LLMs cannot…

> why is it that the LLM gives an answer when we can prove that it cannot answer this problem correctly? Brcause LLMs are not “problem solving machines” they are text completion models, so (when trained for q-and-a response) their function is to produce text output which forms a plausible seeming response to the question posed, not to execute an algorithm which solves the logical problem it communicatss. Asking “why…

> their function is to produce text output which forms a plausible seeming response to the question posed

Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. And it would not be a hallucination.

Re: Hallucination is inevitable: An innate limitation of large language models

#244

Post by eszed »

solarhexes wrote: post #202
ninetyninenine wrote: post #188

Earlier quoted context omitted.

Explain sora. It must have of course a blurry understanding of reality to even produce those videos. I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality. Even the LLMs that are weaker than sora understand things. What is debatable is whether LLMs are conscious. But whether it can understand something is a pretty clear yes. But does it understand e…

If by “understand” you mean “can model reasonably accurately much of the time” then maybe you’ll find consensus. But that’s not a universal definition of “understand”. For example, if I asked you whether you “understand” ballistic flight, and you produced a table that you interpolate from instead of a quadratic, then I would not feel that you understand it, even though you can kinda sorta model it. And even if you do…

I think there are two axes: reason about and intuit. I "understand" ballistic flight when I can calculate a solution that puts an artillery round on target. I also "understand" ballistic flight when I make a free throw with a basketball.

On writing that, I have an instinct to revise it to move the locus of understanding in the first example to the people who calculated the ballistic tables, based on physics first-principles. That would be more accurate, but my mistake highlights something interesting: an artillery officer / spotter simultaneously uses both. Is theirs a "deeper" / "truer" understanding? I don't think it is. I don't know what I think that means, for humans or AI.

Re: Hallucination is inevitable: An innate limitation of large language models

#245

Post by ta8645 »

caditinpiscinam wrote: post #229
username332211 wrote: post #195

Earlier quoted context omitted.

If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as ea…

> Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velo…

I think the best response is to steal the joke and repeat it without comment.

"Eleven meters per second."

Full stop. It's humorous, and any reasonable interlocutor understands not to take it seriously.

Of course, there are more serious questions that demand more serious answers. LLMs will eventually need to be able to understand the current context and assess the appropriate level of confidence required in any answer.

Re: Hallucination is inevitable: An innate limitation of large language models

#246

Post by tgv »

Jensson wrote: post #97
tgv wrote: post #52

Earlier quoted context omitted.

Search for a piece of code you wrote. If it's more than 15 lines and not boilerplate, chances are you won't find it anywhere on the net.

But there are many pieces of code that I've written that you can find in many places on the net, having a tool that can adapt that to your codebase in seconds is useful. It doesn't have to be smart, just pasting in an function and fitting that to your code is useful.

Sure, but the point is it will have to adapt it to your code, if only in naming. So it has to make up things, i.e. hallucinate. It can't just reproduce the best match in memory.

Re: Hallucination is inevitable: An innate limitation of large language models

#247

Post by MauranKilom »

ninetyninenine wrote: post #182

Fiction and story writing is hallucination. It is the opposite of a stochastic parrot. We've achieved both extremes of AI. Computers can be both logical machines and hallucinators. Our goal is to create a machine that can be both at the same time and can differentiate between both. That's the key. Hallucination is important but the key is for the computer to be self aware about when it's hallucinating. Of course it's…

> even humans hallucinate massively

Simpler example: Dreams.

Re: Hallucination is inevitable: An innate limitation of large language models

#248

Post by c22 »

DiogenesKynikos wrote: post #228
beardedwizard wrote: post #215

Earlier quoted context omitted.

Human brains form new connections dynamically. Llms are trained on connections human brains have already made. They never make new connections that aren't in training data. Nothing was synthesized, all the data was seen before and related to each other by vector similarity. It can just parrot the collective understanding humans already have and teach it.

> It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that re…

You claim that logical and reasonable responses "require understanding" therefore LLMs must understand. But I see LLMs as evidence that understanding is not required to produce logical and reasonable responses.

Thinking back to when I used to help tutor some of my peers in 101-level math classes there were many times someone was able to produce a logical and reasonable response to a problem (by rote use of an algorithm) but upon deeper interrogation it became clear that they lacked true understanding.

Re: Hallucination is inevitable: An innate limitation of large language models

#249

Post by dragonwriter »

someplaceguy wrote: post #243
dragonwriter wrote: post #232

Earlier quoted context omitted.

> why is it that the LLM gives an answer when we can prove that it cannot answer this problem correctly? Brcause LLMs are not “problem solving machines” they are text completion models, so (when trained for q-and-a response) their function is to produce text output which forms a plausible seeming response to the question posed, not to execute an algorithm which solves the logical problem it communicatss. Asking “why…

> their function is to produce text output which forms a plausible seeming response to the question posed Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. And it would not be a hallucination.

> Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question.

Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I couldn't solve”. And that’s why LLMs behave the way they do.

Re: Hallucination is inevitable: An innate limitation of large language models

#250

Post by cpuguy83 »

tsimionescu wrote: post #226
cpuguy83 wrote: post #190

Earlier quoted context omitted.

Are humans not the same way? There's a saying "I don't know what I don't know".

The existence of such a saying means little: it is uncontroversially true that humans often do know what they don't know. For example, I know for sure that I don't know how long I will live. This disproves the saying.

The saying is not "I don't know anything that I don't know", which would be self contradictory. It disproves nothing.

ChatGPT "knows" that it doesn't know anything after a certain date, for whatever it means to "know" something.

Post reply on HN