Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs aren't world models

yosefk.com

Post reply on HN
Print view View original

121–130 of 240 posts

Re: LLMs aren't world models

#121

Post by pixl97 »

pron wrote: post #114
Terr_ wrote: post #112

Earlier quoted context omitted.

Why meaningless? Imperfect knowledge can still be useful , and ultimately that's the only kind we can ever have about anything. "We could learn to sail the oceans and discover new lands and transport cargo cheaply... But in a few centuries we'll discover we were wrong and the Earth isn't really a sphere and tides are extra-complex so I guess there's no point."

Because if there's an infinite number of laws, are they laws at all? You can't predict anything because you don't even know if some of the laws you don't know yet (which is pretty much all of them) makes an exception to the 0% of laws you do know. I'm not saying it's not interesting, but it's more history - today the apple fell down rather than up or sideways - than physics.

In the infinite set of all laws is there an infinite set of laws that do not conflict with each other?

.000000000000001% of infinity is still infinite.

Re: LLMs aren't world models

#122

Post by nomel »

semiquaver wrote: post #108
nomel wrote: post #94

Earlier quoted context omitted.

I don’t think it’s a communication problem as much as there is no possible relation between a word and a (literal) physical experiences. They’re, quite literally, on different planes of existence.

Well shit, I better stop reading books then.

I think you've missed the concept here.

You exist in the full experience. That lossy projection to words is still meaningful to you, in your reading, because you know the experience it's referencing. What do I mean by "lossy projection"? It's the experience of seeing the color blue to the word "blue". The word "blue" is meaningless without already having experienced it, because the word is not a description of the experience, it's a label. The experience itself can't be sufficiently described, as you'll find if you try to explain a "blue" to a blind person, because it exists outside of words.

The concept here is that something like an LLM, trained on human text, can't having meaningful comprehension of some concepts, because some words are labels of things that exist entirely outside of text.

You might say "but multimodal models use tokens for color!", or even extending that to "you could replace the tokens used in multimodal models with color names!" and I would agree. But, the understanding wouldn't come from the relation of words in human text, it would come from the positional relation of colors across a space, which is not much different than our experience of the color, on our retina

tldr: to get AI to meaningful understand something, you have to give it a meaningful relation. Meaningful relations sometimes aren't present, in human writing.

Re: LLMs aren't world models

#123

Post by drdeca »

nomel wrote: post #104
drdeca wrote: post #102

Earlier quoted context omitted.

When I have a physical experience, sometimes it results in me saying a word. Now, maybe there are other possible experiences that would result in me behaving identically, such that from my behavior (including what words I say) it is impossible to distinguish between different potential experiences I could have had. But, “caused me to say” is a relation, is it not? Unless you want to say that it wasn’t the experience…

Yes, but it's a unidirectional relation : it was the result of the experience. The word cannot represent the context (the experience), in a meaningful way. It's like trying to describe a color to a blind person: poetic subjective nonsense.

I don’t know what you mean by “unidirectional relation”. I get that you gave an explanation after the colon, but I still don’t quite get what you mean. Do you just mean that what words I use doesn’t pick out a unique possible experience? That’s true of course, but I don’t know why you call that “unidirectional”

I don’t think describing colors to a blind person is nonsense. One can speak of how the different colors relate to one-another. A blind person can understand that a stop sign is typically “red”, and that something can be “borderline between red and orange”, but that things will not be “borderline between green and purple”. A person who has never had any color perception won’t know the experience of seeing something red or blue, but they can still have a mental model of the world that includes facts about the colors of things, and what effects these are likely to have, even though they themselves cannot imagine what it is like to see the colors.

Re: LLMs aren't world models

#124

Post by eru »

slt2021 wrote: post #81
ameliaquining wrote: post #59

One thing I appreciated about this post, unlike a lot of AI-skeptic posts, is that it actually makes a concrete falsifiable prediction; specifically, "LLMs will never manage to deal with large code bases 'autonomously'". So in the future we can look back and see whether it was right. For my part, I'd give 80% confidence that LLMs will be able to do this within two years, without fundamental architectural changes.

>LLMs will never manage to deal time to prove hypothesis: infinity years

Eh, if the hypothesis remains unfalsified for longer and longer, we can have increased confidence.

Similar, Newton's laws say that bodies always stay at rest unless acted upon by a force. Strictly speaking, if a billiard ball jumps up without cause tomorrow that would disprove Newton. So we'd have to wait an infinite amount of time to prove Newton right.

However no one has to wait so long, and we found ways to express how Newton's ideas are _better_ than those of Aristotle without waiting an eternity.

Re: LLMs aren't world models

#125

Post by eru »

typpilol wrote: post #63
WillPostForFood wrote: post #51

Earlier quoted context omitted.

Your LLM output seems abnormally bad, like you are using old models, bad models, or intentionally poor prompting. I just copied and pasted your Krita example into ChatGPT, and reasonable answer, nothing like what you paraphrased in your post. https://imgur.com/a/O9CjiJY

This seems like a common theme with these types of articles

Perhaps the people who get decent answers don't write articles about them?

Re: LLMs aren't world models

#126

Post by eru »

p1esk wrote: post #60
marcellus23 wrote: post #58

Earlier quoted context omitted.

I think it's hard to take any LLM criticism seriously if they don't even specify which model they used. Saying "an LLM model" is totally useless for deriving any kind of conclusion.

Yes, I’d be curious about his experience with GPT-5 Thinking model. So far I haven’t seen any blunders from it.

I've seen plenty of blunders, but in general it's better than their previous models.

Well, it depends a bit on what you mean by blunders. But eg I've seen it confidently assert mathematically wrong statements with nonsense proofs, instead of admitting that it doesn't know.

Re: LLMs aren't world models

#127

Post by eru »

AyyEye wrote: post #8
libraryofbabel wrote: post #4

This essay could probably benefit from some engagement with the literature on “interpretability” in LLMs, including the empirical results about how knowledge (like addition) is represented inside the neural network. To be blunt, I’m not sure being smart and reasoning from first principles after asking the LLM a lot of questions and cherry picking what it gets wrong gets to any novel insights at this point. And it alr…

With LLMs being unable to count how many Bs are in blueberry, they clearly don't have any world model whatsoever. That addition (something which only takes a few gates in digital logic) happens to be overfit into a few nodes on multi-billion node networks is hardly a surprise to anyone except the most religious of AI believers.

> With LLMs being unable to count how many Bs are in blueberry, they clearly don't have any world model whatsoever.

Train your model on characters instead of on tokens, and this problem goes away. But I don't think this teaches us anything about world models more generally.

Re: LLMs aren't world models

#128

Post by eru »

nosioptar wrote: post #29
andyjohnson0 wrote: post #16

Earlier quoted context omitted.

> With LLMs being unable to count how many Bs are in blueberry, they clearly don't have any world model whatsoever. Is this a real defect, or some historical thing? I just asked GPT-5: How many "B"s in "blueberry"? and it replied: There are 2 — the letter b appears twice in "blueberry". I also asked it how many Rs in Carrot, and how many Ps in Pineapple, amd it answered both questions correctly too.

Shouldn't the correct answer be that there is not a "B" in "blueberry"?

No, why?

It depend on context. English is often not very precise and relies on implied context clues. And that's good. It makes communication more efficient in general.

To spell it out: in this case I suspect you are talking about English letter case? Most people don't care about case when they ask these questions, especially in an informal question.

Re: LLMs aren't world models

#129

Post by whoknowsidont »

ameliaquining wrote: post #59

One thing I appreciated about this post, unlike a lot of AI-skeptic posts, is that it actually makes a concrete falsifiable prediction; specifically, "LLMs will never manage to deal with large code bases 'autonomously'". So in the future we can look back and see whether it was right. For my part, I'd give 80% confidence that LLMs will be able to do this within two years, without fundamental architectural changes.

I don't think that statement is falsifiable until you define "deal with" and "large code bases."

Re: LLMs aren't world models

#130

Post by drdeca »

zeroonetwothree wrote: post #111
drdeca wrote: post #92

Earlier quoted context omitted.

simulate ≠ simulate-in-real-time

All simulation is realtime to the brain being simulated.

Sure, but that’s not the clock that’s relevant to the question of the light speed communication limits in a large computer?
Post reply on HN