Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs aren't world models

yosefk.com

Post reply on HN
Print view View original

201–210 of 240 posts

Re: LLMs aren't world models

#201

Post by Terretta »

boxed wrote: post #174
yosefk wrote: post #159

Earlier quoted context omitted.

If you're right, there will soon be a flood of software teams with no programmers on them - either across all domains, or in some domains where this works well. We shall see. Indeed I have no experience with Claude Code, but I use Claude via chat, and it fails all the time on things not remotely as hard as orientation in a large code base. Claude Code is the same thing with the ability to run tools. Of course tools h…

I was very skeptical of Claude Code but was finally convinced to try it and it does feel very different to use. I made three hobby projects in a weekend that I had pushed up for years due to "it's too much hassle to get started" inertia. Two of the projects it did very well with, the third I had to fight with it and it still is subtly wrong (swiftUI animations and claude code seemingly is not a good mix!) That being…

> SwiftUI animations and claude code seemingly is not a good mix

Where is the corpus of SwiftUI animations to train Claude what probable soup you probably want regurgitated?

Hypothesis: iOS devs don't share their work openly for reasons associated with how the App Store ecosystem (mis)behaves.

Relatedly, the models don't know about Swift 6 except from maybe mid-2024 WWDC announcements. It's worth feeding them your own context. If you are 5.10, great. If you want to ship iOS 26 changes, wait till 2026 or again, roll your own context.

Re: LLMs aren't world models

#202

Post by ordu »

Mikhail_Edoshin wrote: post #199
ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

You probably know the Law of Archimedes. Many people do. But do you know it in the same way Archimedes did? No. You were told the law, then taught how to apply it. But Archimedes discovered it without any of that. Can we repeat the feat of Archimedes? Yes, we can, but first we'd have to forget what we were told and taught. The way we actually discover things is very different from amassing lots of hearsay. Indeed, we…

> But to get to the real understanding we actually shut down that part, forget what we "know", start from a clean slate.

Close, but not exactly. To start from a clean slate is not very difficult, the trick is to reject some chosen parts of existing knowledge, or more specifically the difficulty is to choose what to reject. Starting from a clean slate you'll end up spending millennia to get the knowledge you've just rejected.

So the overall process of generating knowledge is to look under the streetlight till finding something new becomes impossible or too hard, and then you start experimenting with rejecting some bits of your knowledge to rethink them. I was taught to read works of Great Masters of the past critically, trying to reproduce their path while looking for forks where you can try to go the other way. It is a little bit like starting from a clean slate, but not exactly.

Re: LLMs aren't world models

#203

Post by ordu »

otabdeveloper4 wrote: post #165
ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

> I believe, that human mind does something like that all the time Absolutely not. Human brains have online one-shot training. LLMs weights are fixed and fine-tuning them is a huge multi-year enterprise. Fundamentally it's two completely different architectures.

I really don't like how you rejecting the idea completely. People have online one-shot training, but have you tried to learn how to play on piano? To learn it you need a lot of repetitions. Really a lot. You need a lot of repetitions to learn how to walk, or how to do arithmetic, or how to read English. This is very similar to LLMs, isn't it? So they are not completely different architectures, aren't they? It is more like human brains have something on top of "LLM" that allows it to do tricks that LLMs couldn't do.

Re: LLMs aren't world models

#204

Post by glenstein »

This is the best and clearest explanation I have yet seen that describe a tricky thing, namely that LLMs, which are synonymous with "AI" for so many people, are just one variation of many possible types of machine intelligence.

Which I find important because, well, hallucinating facts is what you would expect from an LLM, but isn't necessarily inherent issue with machine intelligence writ large if it's trained from the ground up on different principles, or modelling something else. We use LLMs as a stand in for tutors because being really good at language incidentally makes them able to explain math or history as a side effect.

Importantly it doesn't show that hallucinating is a baked in problem for AI writ large. Presumably different models will have different kinds of systemic errors based on their respective designs.

Re: LLMs aren't world models

#205

Post by mft_ »

revskill wrote: post #184
mft_ wrote: post #179

Earlier quoted context omitted.

Lots of people assume, confabulate, misremember, and lie every day.

They are not intelligent.

LOL, you really think that intelligence (however you want to define or measure the concept) is a guarantee that people won't make mistakes, misremember, make stuff up, or lie?

Re: LLMs aren't world models

#206

Post by mft_ »

TheOtherHobbes wrote: post #182
mft_ wrote: post #178

Earlier quoted context omitted.

Sure, your points about the body aren’t wrong, but (as you say) LLMs are only modelling a small subset of a brain’s functions at the moment: applied knowledge, language/communication, and recently interpretation of visual data. There’s no need or opportunity for an LLM (as they currently exist) to do anything further. Further, just because additional inputs exist in the human body (gut-brain axis, for example) it doe…

The point is that knowledge/language work can't work reliably unless it's grounded in something outside of itself. Without it you don't get an oracle, you get a superficially convincing but fundamentally unreliable idiot savant who lacks a stable sense of self, other, or real world. The fundamental foundation of science and engineering is reliability. If you start saying reliability doesn't matter, you're not doing s…

I'm really struggling to understand what you're trying to communicate here; I'm even wondering if you're an LLM set up to troll, due to the weird language and confusing non-sequiturs.

> The point is that knowledge/language can't work reliably unless it's grounded in something outside of itself.

Just, what? Knowledge is facts, somehow held within a system allowing recall and usage of those facts. Knowledge doesn't have a 'self', and I'm totally not understanding how pure knowledge as a concept or medium needs "grounding"?

Being charitable, it sounds more like you're trying to describe "wisdom" - which might be considered as a combination of knowledge, lived experience, and good judgement? Yes, this is valuable in applying knowledge more usefully, but has nothing to do with the other bodily systems which interact with the brain, which is where you started?

> The fundamental foundation of science and engineering is reliability.

> If you start saying reliability doesn't matter, you're not doing science and engineering any more.

No-one mentioned reliability - not you in your original post, or me in my reply. We were discussing whether the various (unconscious) systems which link to the brain in the human body (like the gut:brain axis) might influence its knowledge/language/interpretation abilities.

Re: LLMs aren't world models

#207

Post by codebastard »

ej88 wrote: post #21

This article is interesting but pretty shallow. 0(?): there’s no provided definition of what a ‘world model’ is. Is it playing chess? Is it remembering facts like how computers use math to blend Colors? If so, then ChatGPT: https://chatgpt.com/s/t_6898fe6178b88191a138fba8824c1a2c has a world model right? 1. The author seems to conflate context windows with failing to model the world in the chess example. I challenge…

I my opinion the author refers to a LLMs inability to create a inner world, a world model.

That means it does not build a mirror of a system based on its interactions.

It just outputs fragments of world models it was build one and tries to give you a string of fragments that should match to the fragment of your world model that you provided through some input method.

It can not abstract the code base fragments you share it can not extend them with details using the model of the whole project.

Re: LLMs aren't world models

#208

Post by boxed »

Terretta wrote: post #201
boxed wrote: post #174

Earlier quoted context omitted.

I was very skeptical of Claude Code but was finally convinced to try it and it does feel very different to use. I made three hobby projects in a weekend that I had pushed up for years due to "it's too much hassle to get started" inertia. Two of the projects it did very well with, the third I had to fight with it and it still is subtly wrong (swiftUI animations and claude code seemingly is not a good mix!) That being…

> SwiftUI animations and claude code seemingly is not a good mix Where is the corpus of SwiftUI animations to train Claude what probable soup you probably want regurgitated? Hypothesis: iOS devs don't share their work openly for reasons associated with how the App Store ecosystem (mis)behaves. Relatedly, the models don't know about Swift 6 except from maybe mid-2024 WWDC announcements. It's worth feeding them your ow…

In my case the big issue seems to be that if you hide a component in SwiftUI, it's by default animated with a fade. This not shown in the API surface area at all.

Re: LLMs aren't world models

#209

Post by thecupisblue »

flender wrote: post #195
thecupisblue wrote: post #191

Earlier quoted context omitted.

Ironically, that lesswrong article is more wrong than right. First, chess is perfect for such modeling. The game is basically a tree of legal moves. The "world model" representation is already encoded in the dataset itself and at a certain scale the chance of making an illegal move is minimal, as the dataset itself includes an insane amount of legal moves compared to illegal moves, let alone when you are training it…

And if it knew every possible board configuration and optimal move, it could potentially do as well as it could, but instead if it were to just recognize “this looks like a chess game” and use an optimized tool to determine the next move, that would be a better use of training, it would seem.

Way better use, at this point that engine is more like a world's most expensive monte carlo search.

Re: LLMs aren't world models

#210

Post by adastra22 »

TheOtherHobbes wrote: post #183
adastra22 wrote: post #172

Earlier quoted context omitted.

> Indeed I have no experience with Claude Code, but I use Claude via chat... These are not even remotely similar, despite the name. Things are moving very fast, and the sort of chat-based interface that you describe in your article is already obsolete. Claude is the LLM model. Claude Code is a combination of internal tools for the agent to track its goals, current state, priorities, etc., and a looped mechanism for k…

Claude Code isn't an LLM. It's a hybrid architecture where an LLM provides the interface and some of the reasoning, embedded inside a broader set of more or less deterministic tools. It's obvious LLMs can't do the job without these external tools, so the claim above - that LLMs can't do this job - is on firm ground. But it's also obvious these hybrid systems will become more and more complex and capable over time, an…

If you want to be pedantic about word definitions, it absolutely is AGI: artificial general intelligence.

Whether you draw the system boundary of an LLM to include the tools it calls or not is a rather arbitrary distinction, and not very interesting.

Post reply on HN