Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

411–420 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#411

Post by godelski »

renegade-otter wrote: post #381
godelski wrote: post #351

Earlier quoted context omitted.

Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV , https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE…

You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world. I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly…

> You kind of proved my point.

Yeah I did say you were more right. But it was difficult to distinguish exaggeration from actual intent. You can check my comment history of me battling the common ML mindset. I love the area of study (I'm a researcher myself) but there's a lot of problems that even in the research community a lot want to ignore. It's odd to me. It's been hilarious to watch big names claim Sora understands physics. Or people think just because it doesn't understand physics that the videos aren't still impressive and even useful.

But with how you updated your language, I think we are in a very high level of agreement. You are perfectly right: no ML model "understands" anything. GPT doesn't understand how to code and image models don't understand how to... art(?) or do physics or whatever. They don't have world models. And I'm deeply frustrated that people think a single example of a accurately acting like a world model is proof and will do gymnastics to say a single counter example isn't. A single counter does disprove a world model and to understand you need to be able to self-correct. Hallucinations are fine but "are you sure?" should be enough to get it to reconsider, not double down or just switch. We can be fooled with setups, but we laugh at ourselves quickly because we self-correct fairly easily (or rather, we can).

Re: Hallucination is inevitable: An innate limitation of large language models

#412

Post by godelski »

mr_toad wrote: post #404
renegade-otter wrote: post #381

Earlier quoted context omitted.

You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world. I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly…

Put a brain in a jar and expose it only to photos and you’d get the same results. It’s hard to learn what holding a book is like if you’ve never held anything.

I'm not sure I see the utility of this "thought experiment." It's not really provable and not even possible to do. Not to mention highly unethical. I mean just make the embodiment argument instead. But we don't know if embodiment is necessary, or even what kind of body is even necessary.

Re: Hallucination is inevitable: An innate limitation of large language models

#413

Post by pests »

paulmd wrote: post #379
HeatrayEnjoyer wrote: post #299

Earlier quoted context omitted.

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

> Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new. it's true that most people do not actually understand the problem/limitation, but it's a discussion that is statistically likely to occur on the internet and therefore people tend to regurgitate the words without understanding the concept. I'm being facetious but honest…

What does that have to do with an LLM doing a couple of matrix multiplies?

Re: Hallucination is inevitable: An innate limitation of large language models

#414

Post by godelski »

paulmd wrote: post #379
HeatrayEnjoyer wrote: post #299

Earlier quoted context omitted.

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

> Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new. it's true that most people do not actually understand the problem/limitation, but it's a discussion that is statistically likely to occur on the internet and therefore people tend to regurgitate the words without understanding the concept. I'm being facetious but honest…

Yes, human brains are machines. Anyone saying anything less is implying magic. I'm not sure people are making this argument anymore despite being wildly popular even just 5 years ago.

No, our brains do not work like neural networks. I'd call Taco Bell Mexican Food sooner than I'd call neural nets the same as brains. Neuromorphic computers are closer but even still not the same. Yeah, we are inspired by the brain, but this is very different from being the same.

Re: Hallucination is inevitable: An innate limitation of large language models

#415

Post by ninetyninenine »

samatman wrote: post #328
ninetyninenine wrote: post #294

Earlier quoted context omitted.

Yeah good point. But dreams are easily distinguishable from reality. Religion is often indistinguishable from truth and reality to those who hallucinate it.

Confusing sincere but incorrect belief with hallucination is categorically wrong.

[flagged]

Re: Hallucination is inevitable: An innate limitation of large language models

#416

Post by godelski »

xetplan wrote: post #398
renegade-otter wrote: post #383

Earlier quoted context omitted.

What's the evidence? OpenAI's claims? They do have an inherent interest is making investors believe this technology is magic.

There is no evidence. This entire line of thought is really stupid when chatGPT itself will tell you it doesn't understand the responses it gives. Try the prompt in chatGPT4 "Can you explain to a human how you don't understand anything in the response you give to this question?" For those that don't have chatGPT4. chatGPT4: "Imagine you're using a very advanced calculator that can help you solve all kinds of math pro…

To save people time, here's the inverse

> Can you explain to a human how you understand things and respond to this question?

GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response based on the most probable language patterns that align with the input received. My process is purely computational and lacks consciousness or genuine comprehension.

It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. I used to use the feathers/bricks question but give it different units (kilograms and pounds), but they patched that. But you only need a single example to show it doesn't understand. People will try to prove it understands by asking follow-up questions, but they don't realize that the questions they act spoil the answer. It is very hard to ask follow-ups and not give away the answer.

Re: Hallucination is inevitable: An innate limitation of large language models

#417

Post by godelski »

sshumaker wrote: post #265
renegade-otter wrote: post #254

Earlier quoted context omitted.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

You’re being downvoted because this is a hot take that isn’t supported by evidence. I just tried exactly that with dalle-3 and it worked well. More to the point, it’s pretty clear LLMs do form a model of the world, that’s exactly how they reason about things. There was some good experiments on this a while back - check out the Othello experiment. https://thegradient.pub/othello/

It's a hot take, but it is supported by a ton of evidence.

Re: Hallucination is inevitable: An innate limitation of large language models

#418

Post by type0 »

mr_toad wrote: post #402
samatman wrote: post #325

Earlier quoted context omitted.

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and…

I suspect it’s just an incomplete memory and no “filter”. LLMs aren’t self-aware enough to judge their own confidence in their responses, so they don’t know when to shut up.

> LLMs aren’t self-aware enough

LLMs aren't self-aware at all, it's an illusion presented to the observer

Re: Hallucination is inevitable: An innate limitation of large language models

#419

Post by mannykannot »

ninetyninenine wrote: post #396
mannykannot wrote: post #391

Earlier quoted context omitted.

The author of the post to which you are replying seems to be defining "understanding" as merely meaning "able to do something."

The author of the post is saying that understanding something can't be defined because we can't even know how the human brain works. It is a black box. The author is saying at best you can only set benchmark comparisons. We just assume all humans have the capability of understanding without even really defining the meaning of understanding. And if a machine can mimic human behavior to it must also understand. That is…

What the author of the post actually said - and I am quoting, to make it clear that I'm not putting my spin on someone else's opinion - was "There's no difference between doing something that works without understanding and doing the exact same thing with understanding."

Re: Hallucination is inevitable: An innate limitation of large language models

#420

Post by mannykannot »

ninetyninenine wrote: post #406
mannykannot wrote: post #397

Earlier quoted context omitted.

It is interesting that you are demanding a metric here, as yours appears to be like duck typing: in effect, if it quacks like a human... Defining "understanding" is difficult (epistemology struggles with the apparently simpler task of defining knowledge), but if I saw a dialogue between two LLMs figuring out something about the external world that they did not initially have much to say about, I would find that prett…

Without a metric no position can be made. All conversation about this topic is just conjecture with no path to a conclusion.

This is a common misunderstanding, one also seen with regard to definitions. When applied to knowledge acquisition, it suffers from a fairly obvious bootstrapping problem, which goes away when you realize that metrics and definitions are rewritten and refined as our knowledge increases. Just look at what has happened to concepts of matter and energy over the last century or so.

You are free to disagree with this, but I feel your metric for understanding resembles the Turing test, while the sort of thing I have proposed here, which involves AIs interacting with each other, is a refinement that makes a step away from defining understanding and intelligence as being just whatever human judges recognize as such (it still depends on human judgement, but I think one could analyze the sort of dialogue I am envisioning more objectively than in a Turing test.)

Post reply on HN