Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

441–450 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#441

Post by tonnydourado »

ddalex wrote: post #437
tonnydourado wrote: post #126

Earlier quoted context omitted.

People really need to understand that your single/double digit dataset of interactions with an inherently non-deterministic process is less than irrelevant. It's saying that global warming isn't real because it was really cold this week. I don't even know enough superlatives to express how irrelevant it is that "it seems to you" that an LLM behaves this way or that. And even the "protocol" in question is weak. Self r…

Yes, I get it from a science point of view. But if it makes me happy that I have better results with this technique, and I want to share it with others, who are you to tell me to stop? If you don't like it, don't use it.

I don't have a problem with you doing and even sharing whatever cargo culting prompting technique you want to share.

My problem starts when you make bold claims like "LLMs can self reflect" and your only evidence is "I asked one and it said yes".

Re: Hallucination is inevitable: An innate limitation of large language models

#442

Post by ben_w »

mistermann wrote: post #400
ben_w wrote: post #375

Earlier quoted context omitted.

Sometimes. No idea what you're getting at here, though.

"you know what I mean" ("x is true [for a certain definition of true, other than the correct technical definition]", etc) on both sides causes humans to believe that they adequately understand the meaning trying to be communicated, which is a hallucination. It's true that this is often not a big deal, but which times it is and which times it is not is not known (which itself is typically not known, once again because…

Ah! Yes, indeed. That was a strange, even frustrating, experience — though not as annoying as the times I failed to get deeper explanations from a teacher or a line manager.

Re: Hallucination is inevitable: An innate limitation of large language models

#443

Post by jncfhnb »

pixl97 wrote: post #382
jncfhnb wrote: post #308

Earlier quoted context omitted.

> They cannot say "I dont know" because they dont actually know anything. print(“I don’t know”) You don’t need proper cognition to identify that the answer is not stored in source data. Your conception of the model is incomplete as is easily demonstrable by testing such cases now. Chat gpt does just fine on your simpsons test. You, however, have made up an answer of how something works that you don’t actually know de…

>You don’t need proper cognition to identify that the answer is not stored in source data. Uh, what? So lets imagine you have an LLM that knows everything, except you withhold the data that you can put peanut butter on toast. Toast + Peanut butter = does not exist in data set. So what exactly do you expect the LLM to say when someone asks "Can you put peanut butter on toast?". I would expect an intelligent agent to '…

This behavior you’re describing is trainable either way.

Tuned LLMs are not simple most likely token models. They are most likely token given a general overarching strategy for contextualizing future tokens model.

Which can be conservative or imaginative.

Re: Hallucination is inevitable: An innate limitation of large language models

#444

Post by jijijijij »

jncfhnb wrote: post #350
jijijijij wrote: post #334

Earlier quoted context omitted.

> to identify that the answer is not stored in source data How would an LLM do that?

They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic. How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning. I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the…

So, you are basing your assessment on your gut feel and personal impression with ChatGPT?

Maybe you should tone down the spice a bit, then.

Unless you can explain how an actual understanding emerges within an LLM, you can't explain how it would answer the question definitely - it doesn't know, if it does, or does not know something. Generally speaking.

Re: Hallucination is inevitable: An innate limitation of large language models

#445

Post by ddalex »

tonnydourado wrote: post #441
ddalex wrote: post #437

Earlier quoted context omitted.

Yes, I get it from a science point of view. But if it makes me happy that I have better results with this technique, and I want to share it with others, who are you to tell me to stop? If you don't like it, don't use it.

I don't have a problem with you doing and even sharing whatever cargo culting prompting technique you want to share. My problem starts when you make bold claims like "LLMs can self reflect" and your only evidence is "I asked one and it said yes".

I see no evidence that they can't self reflect. Certainly they can evaluate the confidence of the next predicted token, and that a form of reflection

Re: Hallucination is inevitable: An innate limitation of large language models

#446

Post by tonnydourado »

ddalex wrote: post #445
tonnydourado wrote: post #441

Earlier quoted context omitted.

I don't have a problem with you doing and even sharing whatever cargo culting prompting technique you want to share. My problem starts when you make bold claims like "LLMs can self reflect" and your only evidence is "I asked one and it said yes".

I see no evidence that they can't self reflect. Certainly they can evaluate the confidence of the next predicted token, and that a form of reflection

> Certainly they can (...)

No, that's the problem. You don't have certainty, not in any remotely scientific definition of the word, because you don't have enough data, and the data you do have is crap.

Also:

> I see no evidence that they can't self reflect

I see no evidence that there isn't a magical invisible unicorn in the sky that grants wishes to those who wear unicorn themed underwear, so, it must exist.

Re: Hallucination is inevitable: An innate limitation of large language models

#447

Post by naasking »

godelski wrote: post #432
naasking wrote: post #423

Earlier quoted context omitted.

> It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. Doesn't understand what exactly? That seems like a fairly open ended statement and almost certainly wrong as a result. GPT doesn't understand certain things because it hasn't seen those things or anything like it in its training data. How much do you understand…

> Doesn't understand what exactly? Just about anything. Including it's own claims. It isn't uncommon for it to be inconsistent within a singular output. > Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole? You're misunderstanding the test. Let's try. > = me, >> = GPT > I have a fox, a goose, and a bag of corn that I need to transport…

> So I don't care about it being "born in a box" or perceiving the world "through a pinhole." Because it isn't alive. It's a tool. It isn't sentient. It isn't thinking. It is an incredibly complex statistical system.

You have no idea what "sentient" or "thinking" mean mechanistically, so you literally cannot make this claim, nor can you demonstrate at this time that the human mind is not just an incredibly complex statistical system. This argument of yours is just a basic fallacy of ignorance, and unfortunately very common among people who are very certain that LLMs don't understand anything.

I'm not sure what you think your example proves, but humans can exhibit comparable confusions from comparable prompts (like from priming). In fact, one might say that priming shows some behaviours that are eerily similar to some LLM failure modes. The classic Surgeon's Dilemma riddle test for unconscious bias is a perfect example of human failures comparable to the bias in your own example.

Ultimately, you're guilty of exactly the same leap to conclusions that those hyping GPT and LLMs are doing, just in the opposite direction.

Re: Hallucination is inevitable: An innate limitation of large language models

#448

Post by mannykannot »

ninetyninenine wrote: post #429
mannykannot wrote: post #420

Earlier quoted context omitted.

This is a common misunderstanding, one also seen with regard to definitions. When applied to knowledge acquisition, it suffers from a fairly obvious bootstrapping problem, which goes away when you realize that metrics and definitions are rewritten and refined as our knowledge increases. Just look at what has happened to concepts of matter and energy over the last century or so. You are free to disagree with this, but…

No it's not a misunderstanding. Without a concrete definition on a metric comparisons are impossible because everything is based off of wishy washy conjectures on vague and fuzzy concepts. Hard metrics bring in quantitative data. It shows hard differences. Even if the metric is some side marker where in the future is found to have poor correlation or causation with the the thing being measured the hard metric is stil…

This is rather self-contradictory: you insist we can't make progress with wishy-washy conjectures on vague and fuzzy concepts, and yet your entire argument in this thread for your claim that machine understanding of the real world has been achieved is based on exactly that: your personal subjective assessment of LLM performance!

In your final paragraph, you attempt to suggest that my proposed test is no better than the Turing test (and therefore no better than what you are doing), but as you have not addressed the ways in which my proposal differs from the Turing test, I regard this as merely waffling on the issue. In practice, it is not so easy to come up with tests for whether a human understands an issue (as opposed to having merely committed a bunch of related propositions to memory) and I am trying to capture the ways in which we can make that call.

You entered this debate saying "I think we are way past the point of debate here. LLMs are not stochastic parrots. LLMs do understand an aspect of reality", yet your post here ends with "in the end there's a human in the loop making a judgment call", explicitly acknowledging that your strong initial claims are matters of opinion, rather than established facts supported by hard metrics.

Re: Hallucination is inevitable: An innate limitation of large language models

#449

Post by AlexeyBelov »

Anotheroneagain wrote: post #430
p1esk wrote: post #208

Earlier quoted context omitted.

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

Your understanding is distorted by dealing mostly with psychotic people. Iron poisoning (well within the supposed healthy levels) and lead deficiency cause schizophrenia; normal people look childlike or "not intelligent" to the affected.

Do you have a source for this? It seems like most of your comments mention the same things but nothing is substantiated.

Re: Hallucination is inevitable: An innate limitation of large language models

#450

Post by mannykannot »

ninetyninenine wrote: post #428
mannykannot wrote: post #419

Earlier quoted context omitted.

What the author of the post actually said - and I am quoting, to make it clear that I'm not putting my spin on someone else's opinion - was "There's no difference between doing something that works without understanding and doing the exact same thing with understanding."

I'm the author. To be clear. I referred to myself as "the author." And no I did not say that. Let me be clear I did not say that there is "no difference". I said whether there is or isn't a difference we can't fully know because we can't define or know about what "understanding" is. At best we can only observe external reactions to input.

That was just about guaranteed to cause confusion, as in my reply to solarhexes, I had explicitly picked out "the author of the post to which you are replying", who is cultureswitch, not you, and that post most definitely did make the claim that "there's no difference between doing something that works without understanding and doing the exact same thing with understanding."

It does not seem that cultureswitch is an alias you are using, but even if it is, the above is unambiguously the claim I am referring to here, and no other.

As for the broader issues, we have already continued that discussion elsewhere: https://news.ycombinator.com/item?id=39503027

Post reply on HN