Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

231–240 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#231

Post by wnkrshm »

Scene_Cast2 wrote: post #76

I don't think anyone has mentioned Bayesian Neural Nets (I forget the exact term). Sure, the paradigm adds an order of magnitude overhead (at least - and that's why I've never seen it used in the industry), but you can bolt it on to existing architectures. The basic idea is that besides the probabilities, the network also spits out confidence (IIRC based on how out-of-distribution the input is). There's been a ton of…

I would imagine that to propagate any confidence value through the system you'd need to have priors for the confidence of correctness for all data in your training set. (and those priors change over time)

Re: Hallucination is inevitable: An innate limitation of large language models

#232

Post by dragonwriter »

Fripplebubby wrote: post #227
someplaceguy wrote: post #219

Earlier quoted context omitted.

> because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I haven't read the paper, but that sounds like it would only be true if the definition of "hallucinating" is giving a wrong answer, but that's not how it's commonly understood. When people refer to LLMs hallucinating, they are indeed referring to an LLM giving a wrong (and confident) answer. However, not all wrong answers are hallucina…

Yes, I think you're right. I think one way to phrase the authors' argument is: * There is a class of problems which are harder than polynomial time complexity to solve, but are not np-complete * LLMs will generate an "answer" in formal language to this class of problems posed to it * LLMs can at most solve problems with polynomial time complexity due to their fundamental design and principles * Therefore, LLMs cannot…

> why is it that the LLM gives an answer when we can prove that it cannot answer this problem correctly?

Brcause LLMs are not “problem solving machines” they are text completion models, so (when trained for q-and-a response) their function is to produce text output which forms a plausible seeming response to the question posed, not to execute an algorithm which solves the logical problem it communicatss. Asking “why do LLMs do exactly what they are designed to do, even when they cannot do the thing that that behavior implies to a human would have been done to produce it” just reveals a poor understanding of what an LLM is. (Also, the fact that they structurally can't solve a class of problems does not mean that they can't produce correct answers, it means they can't infallibly produce correct answers; the absence of a polynomial time solution does not rule out an arbitrarily good polynomial time approximation algorithm, though its unlikely than an LLM is doing that, either.)

Re: Hallucination is inevitable: An innate limitation of large language models

#233

Post by beardedwizard »

DiogenesKynikos wrote: post #228
beardedwizard wrote: post #215

Earlier quoted context omitted.

Human brains form new connections dynamically. Llms are trained on connections human brains have already made. They never make new connections that aren't in training data. Nothing was synthesized, all the data was seen before and related to each other by vector similarity. It can just parrot the collective understanding humans already have and teach it.

> It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that re…

Isn't this describing temperature induced randomness and ascribing some kind of intelligence to it? This assertion has been made and refuted multiple times on this thread and no solid evidence to the contrary presented.

To go back to your first sentence - interacting with an llm is not understanding how it works, building one is. The actual construction of a neural network llm refutes your assertions.

Re: Hallucination is inevitable: An innate limitation of large language models

#234

Post by lazide »

foobarian wrote: post #205
digitalsushi wrote: post #199

Earlier quoted context omitted.

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

Well in the example of an NP complete problem, a human might realize they are having trouble coming up with an optimal solution and start analyzing complexity. And once they have a proof might advise you accordingly and perhaps suggest a good enough heuristic.

Have you managed dev teams before? It's really nice when they do that, but that is far from the common case.

Re: Hallucination is inevitable: An innate limitation of large language models

#235

Post by flextheruler »

foobarian wrote: post #205
digitalsushi wrote: post #199

Earlier quoted context omitted.

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

Well in the example of an NP complete problem, a human might realize they are having trouble coming up with an optimal solution and start analyzing complexity. And once they have a proof might advise you accordingly and perhaps suggest a good enough heuristic.

Is the commenter above you implying humans hallucinate to the level of LLMs? Maybe hungover freshman working on a tight deadline without having read the book do, but not professionals.

Even a mediocre employees will often realize they’re stuck, seek assistance, and then learn something from the assistance instead of making stuff up.

Re: Hallucination is inevitable: An innate limitation of large language models

#236

Post by lazide »

FpUser wrote: post #222
p1esk wrote: post #208

Earlier quoted context omitted.

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

You post amounts to: in order to be smarter I need to increase my smartness. Great insight.

I think it's more subtly misleading - to be smarter, I need more knowledge. But knowledge != smart, knowledge == informed, or educated.

And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers.

Which arguably isn't about being smart, and is only tangentially about less or more (external) knowledge really. It's about self-knowledge.

Going down the first path is about knowing everything (in the form of facts, usually). Which hey, maybe?

Going down the second path is about knowing oneself. Which hey, maybe?

They are not the same.

Re: Hallucination is inevitable: An innate limitation of large language models

#237

Post by fauigerzigerk »

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Does the paper distinguish between hallucination and approximation?

If LLMs could be trained to approximate NP-complete functions rather than making stuff up, that would be good enough in many contexts. I guess it's what humans would do.

Re: Hallucination is inevitable: An innate limitation of large language models

#238

Post by lazide »

foobiekr wrote: post #224
ryanwaggoner wrote: post #165

Earlier quoted context omitted.

I’m really curious how you managed that. I pasted your exact prompt and GPT-3.5 gave me this: === Making 20-year predictions about the future of Michael Jackson is challenging due to his passing in 2009. However, his legacy as a cultural icon and musical genius will likely endure for decades to come. His music will continue to influence future generations, and his impact on pop culture will remain significant. Additi…

GPT-4’s tendencies to write these long winded but mostly empty responses is so frustrating.

Just wait until you spend a bunch of time around a politician.

Re: Hallucination is inevitable: An innate limitation of large language models

#239

Post by Der_Einzige »

Culonavirus wrote: post #111
precompute wrote: post #34

Ah, the AI hype is now entering the "let's be real" phase. Haven't seen a frenzied post on alignment in a while now.

The hype is insane. Listen, I think LLMs still have a lot of room to grow and they're already very useful, but like some excellent researchers say, they're not the holy grail. If we want AGI, LLMs are not it. A lot of people seem to think this is an engineering issue and that LLMs can get us there, but they can't, because it is not an engineering issue.

I'm going to take the exact opposite take and claim that "some excellent researchers" support it.

"AGI" is practically already here, you just don't want to admit it: https://www.noemamag.com/artificial-general-intelligence-is-...

Re: Hallucination is inevitable: An innate limitation of large language models

#240

Post by furyofantares »

sandworm101 wrote: post #163
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

But the can say "I don't know." They can be trained to do so ("as of my knowledge cutoff in September 2020 I don't know who Bob Whatgenflabl is") and they can be given context that makes it more likely they do so (I've had good success with this for RAG applications, and extremely little, but not zero, for general prompts.)
Post reply on HN