LLMs aren't world models • Hacker News Board

Re: LLMs aren't world models

#71

Post by auggierose » Tue, Aug 12, 2025, 9:57 PM UTC

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

First: true propositions (that are not provable) can definitely be expressed, if they couldn't, the incompleteness theorem would not be true ;-)

It would be interesting to know what the percentage of people is, who invoke the incompleteness theorem, and have no clue what it actually says.

Most people don't even know what a proof is, so that cannot be a hindrance on the path to AGI ...

Second: ANY world model that can be digitally represented would be subject to the same argument (if stated correctly), not only LLMs.

Re: LLMs aren't world models

#72

Post by shinycode » Tue, Aug 12, 2025, 10:00 PM UTC

ameliaquining wrote: post #59

One thing I appreciated about this post, unlike a lot of AI-skeptic posts, is that it actually makes a concrete falsifiable prediction; specifically, "LLMs will never manage to deal with large code bases 'autonomously'". So in the future we can look back and see whether it was right. For my part, I'd give 80% confidence that LLMs will be able to do this within two years, without fundamental architectural changes.

« autonomously » what happens when subtle updates that are not bugs but change the meaning of some features that might break the workflow on some other external parts of a client’s system ? It happens all the time and, because it’s really hard to have the whole meaning and business rules written and maintained up to date, an LLM might never be able to grasp some meaning. Maybe if instead of developing code and infrastructures, the whole industry shifts toward only writing impossibly precise spec sheets that make meaning and intent crystal clear then, maybe « autonomously » might be possible to pull off

Re: LLMs aren't world models

#73

Post by helloplanets » Tue, Aug 12, 2025, 10:02 PM UTC

skeledrew wrote: post #42

Agree in general with most of the points, except > but because I know you and I get by with less. Actually we got far more data and training than any LLM. We've been gathering and processing sensory data every second at least since birth (more processing than gathering when asleep), and are only really considered fully intelligent in our late teens to mid-20s.

Don't forget the millions of years of pre-training! ;)

Re: LLMs aren't world models

#74

Post by bithive123 » Tue, Aug 12, 2025, 10:03 PM UTC

auggierose wrote: post #71

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

First: true propositions (that are not provable) can definitely be expressed, if they couldn't, the incompleteness theorem would not be true ;-) It would be interesting to know what the percentage of people is, who invoke the incompleteness theorem, and have no clue what it actually says. Most people don't even know what a proof is, so that cannot be a hindrance on the path to AGI ... Second: ANY world model that can…

I knew someone would call me out on that. I used the wrong word; what I meant was "expressed in a way that would satisfy" which implies proof within the symbolic order being used. I don't claim to be a mathematician or philosopher.

Re: LLMs aren't world models

#75

Post by chamomeal » Tue, Aug 12, 2025, 10:03 PM UTC

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

I’m not a math guy but the incompleteness theorem applies to formal systems, right? I’ve never thought about LLMs as formal systems, but I guess they are?

Re: LLMs aren't world models

#76

Post by auggierose » Tue, Aug 12, 2025, 10:09 PM UTC

bithive123 wrote: post #74

auggierose wrote: post #71

Earlier quoted context omitted.

First: true propositions (that are not provable) can definitely be expressed, if they couldn't, the incompleteness theorem would not be true ;-) It would be interesting to know what the percentage of people is, who invoke the incompleteness theorem, and have no clue what it actually says. Most people don't even know what a proof is, so that cannot be a hindrance on the path to AGI ... Second: ANY world model that can…

I knew someone would call me out on that. I used the wrong word; what I meant was "expressed in a way that would satisfy" which implies proof within the symbolic order being used. I don't claim to be a mathematician or philosopher.

Well, you don't get it. The LLM definitely can state propositions "that satisfy", let's just call them true propositions, and that this is not the same as having a proof for it is what the incompleteness theorem says.

Why would you require an LLM to have proof for the things it says? I mean, that would be nice, and I am actually working on that, but it is not anything we would require of humans and/or HN commenters, would we?

Re: LLMs aren't world models

#77

Post by bithive123 » Tue, Aug 12, 2025, 10:09 PM UTC

exe34 wrote: post #68

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

> Language models aren't world models for the same reason languages aren't world models. > Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". There is a lot of negatives in there, but I feel like it boils down to a model of a thing is not the thing. Well duh. It's a model. A map…

Right. It's a dead thing that has no independent meaning. It doesn't even exist as a thing except conceputally. The referent is not even another dead thing, but a reality that appears nowhere in the map itself. It may have certain limited usefulness in the practical realm, but expecting it to lead to new insights ignores the fact that it's fundamentally an abstraction of the real, not in relationship to it.

Re: LLMs aren't world models

#78

Post by bithive123 » Tue, Aug 12, 2025, 10:11 PM UTC

chamomeal wrote: post #75

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

I’m not a math guy but the incompleteness theorem applies to formal systems, right? I’ve never thought about LLMs as formal systems, but I guess they are?

Nor am I. I'm not claiming an LLM is a formal system, but it is mechanical and operates on symbols. It can't deal in anything else. That should temper some of the enthusiasm going around.

Re: LLMs aren't world models

#79

Post by scarmig » Tue, Aug 12, 2025, 10:11 PM UTC

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

> If we accept the incompleteness theorem

And, by various universality theorems, a sufficiently large AGI could approximate any sequence of human neuron firings to an arbitrary precision. So if the incompleteness theorem means that neural nets can never find truth, it also means that the human brain can never find truth.

Human neuron firing patterns, after all, only represent a thing; they are not the same as the thing. Your experience of seeing something isn't recreating the physical universe in your head.

Re: LLMs aren't world models

#80

Post by overgard » Tue, Aug 12, 2025, 10:13 PM UTC

bithive123 wrote: post #62

Language models aren't world models for the same reason languages aren't world models. Symbols, by definition, only represent a thing. They are not the same as the thing. The map is not the territory, the description is not the described, you can't get wet in the word "water". They only have meaning to sentient beings, and that meaning is heavily subjective and contextual. But there appear to be some who think that w…

I don't think you can apply the incompleteness theorem like that, LLMs aren't constrained to formal systems