Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generative AI's failure to induce robust models of the world

garymarcus.substack.com

Post reply on HN
Print view View original

41–50 of 89 posts

Re: Generative AI's failure to induce robust models of the world

#41

Post by phoe-krk »

sgt101 wrote: post #38
bitmasher9 wrote: post #36

Earlier quoted context omitted.

I think negative feedback loops of AIs trained on AI generated data might lead to a position where AI quality peaks and slides backwards.

Thank goodness we have version control systems then.

"Version control systems", in case of AI, mean that their knowledge will stay frozen in time, and so their usefulness will diminish. You need fresh data to train AI systems on, and since contemporary data is contaminated with generative AI, it will inevitably lead to inbreeding and eventual model collapse.

Re: Generative AI's failure to induce robust models of the world

#42

Post by kookamamie »

SubiculumCode wrote: post #9

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

I hope this will happen, too. I think it might as soon as investors realize the LLMs will not become the AGI they were sold as an idea.

Re: Generative AI's failure to induce robust models of the world

#43

Post by seanhunter »

squirrel wrote: post #14
extr wrote: post #8

I find Gary's arguments increasingly semantic and unconvincing. He lists several examples of how LLMs "fail to build a world model", but his definition of "world model" is an informal hand-wave ("a computational framework that a system (a machine, or a person or other animal) uses to track what is happening in the world"). His examples are lifted from a variety of unclear or obsolete models - what is his opinion of O…

He cites o3 and o4-mini as examples of LLMs that play illegal chess moves.

But really, so what? We already have specialised chess engines (stockfish, leela, alphazero etc) that are far far stronger than humans will ever be, so insofar as that’s an interesting goal, we achieved it with deep blue and have gone way way beyond it since. The fact that a large Language model isn’t able to discern legal chess moves seems to me to be neither here nor there. Most humans can’t do that either. I don’t see it as evidence of lack of a world model either (because most people with a real chess board in front of them and a mental model of the world can’t play legal chess moves).

I find it astonishing that people pay any attention to Gary Marcus and doubly so here. Whether or not you are an “AI optimist”, he clearly is just a bloviator.

Re: Generative AI's failure to induce robust models of the world

#44

Post by [deleted] »

seanhunter wrote: post #43
squirrel wrote: post #14

Earlier quoted context omitted.

He cites o3 and o4-mini as examples of LLMs that play illegal chess moves.

But really, so what? We already have specialised chess engines (stockfish, leela, alphazero etc) that are far far stronger than humans will ever be, so insofar as that’s an interesting goal, we achieved it with deep blue and have gone way way beyond it since. The fact that a large Language model isn’t able to discern legal chess moves seems to me to be neither here nor there. Most humans can’t do that either. I don’t…

[deleted]

Re: Generative AI's failure to induce robust models of the world

#45

Post by antonvs »

voidhorse wrote: post #22
etaioinshrdlu wrote: post #16

Earlier quoted context omitted.

I don't think it's accurate anymore to say LLMs are just really good word predictors. Especially in the last year, they are trained with reinforcement learning to solve specific problems. They are functions that predict next tokens, but the function they are trained to approximate doesn't have to be just plain internet text.

Yeah, that's fair. It's probably more accurate to call them sequence predictors or general data predictors than to limit it to words (unless we mean words in the broad, mathematical sense) they are free monoid emulators

And what are humans?

Re: Generative AI's failure to induce robust models of the world

#46

Post by seanhunter »

Animats wrote: post #32

That LLMs are a black box and that LLMs lack an underlying model are both true, but orthogonal. It's possible to have a black box system which has an underlying model. That's true of many statistical prediction methods. Early attempts at machine learning were a white box with no underlying model. This is true of most curve-fitting. The AI version was where you're trying to divide a high-dimensional space with a cutti…

“LLMs lack an underlying model” is very obviously incorrect. LLMs have an underlying model of semantics as tokens embedded into a high-dimensional vector space.

The question is not whether or not they have any model at all, the question is whether the model they indisputably have (which is a model of language in terms of linear algebra) maps onto a model of the external universe (a “world model”) that emerges during training.

This is pretty much an unfalsifiable question as far as I can see. There has been research that aims to show this one way or another and it doesn’t settle the question of what a “world model” even means if you permit a “world model” to mean anything other than “thinks like we do”.

For example, LLMs have been shown to produce code that can make graphics somewhat in the style of famous modern artists (eg Kandinsky and Mondrian) but fail at object-stacking problems (“take a book, four wine glasses, a tennis ball, a laptop and a bottle and stack them in a stable arrangement”). Depending on the objects you choose the LLM either succeeds or fails (generally in a baffling way). So what does this mean? Clearly the model doesn’t “know” the shape of various 3-D objects (unless the problem is in their training set which it sometimes seems to be) but on the other hand seems to have shown some ability to pastiche certain visual styles. How is any of this conclusive? A baby doesn’t understand the 3-D world either. A toddler will try and fail to stack things in various ways. Are they showing the presence or lack of a world model? How do you tell?

Re: Generative AI's failure to induce robust models of the world

#47

Post by adventured »

bitmasher9 wrote: post #36
SubiculumCode wrote: post #9

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

I think negative feedback loops of AIs trained on AI generated data might lead to a position where AI quality peaks and slides backwards.

AI will radically leap forward in specialized function gain over the next decade. That's what everybody should be focusing on. It'll rapidly splinter and acquire dominance over the vast minutia. The intricacy of the endeavor will be led by the AI itself, as it'll fly-wheel itself on becoming an expert at every little thing far faster than we can. We're just seeding that possibility now. Not only will it not slide backwards, it'll leap a great distance forward from where it's at now.

Mainframes -> desktop computers -> a computer in every hand

Obese LLMs you visit -> agents riding with you whereever you are, integrated into your life and things -> everything everywhere, max specialization and distribution into every crevice, dominance over most tasks whether you're there or active or not

They haven't even really started working together yet. They're still largely living in sandboxes. We're barely out of the first inning. Pick a field and it's likely hardly even at the first pitch for most of them you can name, eg aircraft/flight.

In hindsight people will (jokingly?) wonder whether AI self-selected software development as one of its first conquests, as the ultimate foot in the door so it could pursue dominion over everything else (of course it had to happen in that progression; it'll prompt some chicken or the egg debates 30-50 years out).

Re: Generative AI's failure to induce robust models of the world

#48

Post by imtringued »

Lerc wrote: post #18
squirrel wrote: post #14

Earlier quoted context omitted.

He cites o3 and o4-mini as examples of LLMs that play illegal chess moves.

I don't understand the reasoning behind drawing a conclusion that if something fails a task that requires reasoning implies that thing cannot reason. To use chess as an example. Humans sometimes play illegal moves. That does not mean Humans cannot reason. It is an instance of failing to show proof of reasoning. Not a proof of the inability to reason.

Anthropomorphic fallacy.

Human fails at task due to not knowing the rules in perfect detail.

AI fails at task even though it knows the rules and could easily reproduce them for chess and dozens of chess variants.

"Look! The fallibility of humans rubbed off onto the AI, proving that they are more human and AGI than we give them credit to!"

Re: Generative AI's failure to induce robust models of the world

#49

Post by energy123 »

bitmasher9 wrote: post #36
SubiculumCode wrote: post #9

I definitely would be okay if we hit an AI winter; our culture and world cannot adapt fast enough for the change we are experiencing. In the meantime, the current level of AI is just good enough to make us more productive, but not so good as to make us irrelevant.

I think negative feedback loops of AIs trained on AI generated data might lead to a position where AI quality peaks and slides backwards.

I would not bet against synthetic data. AlphaZero is trained only on synthetic data and it's better than any human, and keeps getting better with more training compute. There is no negative feedback loop in the narrow cases we have tried previously. There may be trade-offs but on net we are going forward.

Re: Generative AI's failure to induce robust models of the world

#50

Post by tim333 »

I usually disagree with Garry Marcus but his basic point seems fair enough if not surprising - Large Language Models model language about the world, not the world itself. For a human like understanding of the world you need some understanding of concepts like space, time, emotion, other creatures thoughts and so on, all things we pick up as kids.

I don't see much reason why future AI couldn't do that rather than just focusing on language though.

Post reply on HN