Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs aren't world models

yosefk.com

Post reply on HN
Print view View original

161–170 of 240 posts

Re: LLMs aren't world models

#161

Post by yosefk »

cavisne wrote: post #134
smnrchrds wrote: post #97

Earlier quoted context omitted.

Indeed. If the LLM calls a chess engine tool behind the scenes, it would be able to play excellent chess as well.

The author would still be wrong in the tool-calling scenario. There is already perfect (or at least superhuman) chess engines. There is no perfect "coding engine". LLM's + tools being able to reliably work on large codebases would be a new thing.

Correct - as long as the tools the LLM uses are non-ML-based algorithms existing today, and it operates on a large code base with no programmers in the loop, I would be wrong. If the LLM uses a chess engine, then it does nothing on top of the engine; similarly if an LLM will use another system adding no value on top, I would not be wrong. If the LLM uses something based on a novel ML approach, I would not be wrong - it would be my "ML breakthrough" scenario. If the LLM uses classical algorithms or an ML algo known today and adds value on top of them and operates autonomously on a large code base - no programmer needed on the team - then I am wrong

Re: LLMs aren't world models

#162

Post by shkkmo »

ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

> If LLM fell to tricks of Elizabet Loftus, we'd say LLM hallucinated.

She's strongly oversold how and when false memories can be created. She testified in defense of Ghislaine Maxwell at her 2021 trial that financial incentives can create false memories and only later admitted that there were no studies to back this up when directly questioned.

She's spent a career over-generalizing data about implanting false minor memories to make money discrediting victims' traumatic memories and defend abusers.

You conflate "hallucination" with "imagination" but the former has much more in common with lieing than it does with imagining.

Re: LLMs aren't world models

#163

Post by mac-mc »

ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

Do you have many memories of that time, around 3 to 5, and remember what your cognitive processes were?

When the child is afraid of the monster in the dark, they are not literally visually hallucinating a beast in the dark; they are worried that there could be a beast in the dark, and they are not sure that there is due to a lack of sensory information confirming a lack of the monster. They are not being hyper precise because they are 3, so they say "there is a monster under my bed"! Children have instincts to be afraid of the dark.

Similarly with imaginary friends and play, it's an instinct to practice through smaller stakes simulations. When they are emotionally attached to their imaginary friends, it's much like they are emotionally attached to their security blanket. They know that the "friend" is not perceptible.

It's much like the projected anxieties of adults or teenagers, who are worried that everyone thinks they are super lame and thus act like people do, because on the balance of no information, they choose the "safer path".

That is pretty different than the hallucinations of LLMs IMO.

Re: LLMs aren't world models

#164

Post by TazeTSchnitzel »

ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

I have recently lived through something called a psychotic break, which was an unimaginably horrible thing, but it did let me see from the inside what insanity does to your thinking. And what's fascinating, coming out the other side of this, is how similar LLMs are to someone in psychosis. Someone in psychosis can have all the ability LLMs have to recognise patterns and sound like they know what they're talking about, but their brain is not working well enough to have proper self-insight, to be able to check their thoughts actually fully make sense. (And “making sense” turns out to be a sliding scale — it is not as if you just wake up one day suddenly fully rational again, there's a sliding scale of irrational thinking and you have to gradually re-process your older thoughts into more and more coherent shapes as your brain starts to work more correctly again.) I believe this isn't actually a novel insight either, many have worried about this for years! Psychosis might be an interesting topic to read about if you want to get another angle to understand the AI models from. I won't claim that it's exactly the same thing, but I will say that most people probably have a very undeveloped idea of what mental illness actually is or how it works, and that leaves them badly prepared for interacting with a machine that has a strong resemblance to a mentally ill person who's learned to pretend to be normal.

Re: LLMs aren't world models

#165

Post by otabdeveloper4 »

ordu wrote: post #158

> LLMs are not by themselves sufficient as a path to general machine intelligence; in some sense they are a distraction because of how far you can take them despite the approach being fundamentally incorrect. I don't believe that it is a fundamentally incorrect approach. I believe, that human mind does something like that all the time, the difference is our minds have some additional processes that can, for example,…

> I believe, that human mind does something like that all the time

Absolutely not. Human brains have online one-shot training. LLMs weights are fixed and fine-tuning them is a huge multi-year enterprise.

Fundamentally it's two completely different architectures.

Re: LLMs aren't world models

#166

Post by otabdeveloper4 »

ameliaquining wrote: post #59

One thing I appreciated about this post, unlike a lot of AI-skeptic posts, is that it actually makes a concrete falsifiable prediction; specifically, "LLMs will never manage to deal with large code bases 'autonomously'". So in the future we can look back and see whether it was right. For my part, I'd give 80% confidence that LLMs will be able to do this within two years, without fundamental architectural changes.

> LLMs will never manage to deal with large code bases 'autonomously'

Absolutely nothing about that statement is concrete or falsifiable.

Hell, you can already deal with large code bases 'autonomously' without LLMs - grep and find and sed goes a long way!

Re: LLMs aren't world models

#167

Post by fxtentacle »

eru wrote: post #135
simiones wrote: post #48

Earlier quoted context omitted.

> where it certainly hadn’t seen the questions before? What are you basing this certainty on? And even if you're right that the specific questions had not come up, it may still be that the questions from the math olympiad were rehashes of similar questions in other texts, or happened to correspond well to a composition of some other problems that were part of the training set, such that the LLM could 'pick up' on the…

> What are you basing this certainty on? People make up new questions for each IMO.

Didn’t OpenAI get caught bribing their way to pre-tournament access of the questions?

Re: LLMs aren't world models

#168

Post by globular-toast »

I agree with the article. I will be very surprised if LLMs end up being "it". I say this as a language geek who has always been amazed how language drives our thinking. However, I think language exists between brains, not inside them. There's something else in us and LLMs aren't it.

Re: LLMs aren't world models

#169

Post by derdi »

AyyEye wrote: post #8
libraryofbabel wrote: post #4

This essay could probably benefit from some engagement with the literature on “interpretability” in LLMs, including the empirical results about how knowledge (like addition) is represented inside the neural network. To be blunt, I’m not sure being smart and reasoning from first principles after asking the LLM a lot of questions and cherry picking what it gets wrong gets to any novel insights at this point. And it alr…

With LLMs being unable to count how many Bs are in blueberry, they clearly don't have any world model whatsoever. That addition (something which only takes a few gates in digital logic) happens to be overfit into a few nodes on multi-billion node networks is hardly a surprise to anyone except the most religious of AI believers.

Ask a kid that doesn't know how to read and write how many Bs there are in blueberry.

Re: LLMs aren't world models

#170

Post by eru »

fxtentacle wrote: post #167
eru wrote: post #135

Earlier quoted context omitted.

> What are you basing this certainty on? People make up new questions for each IMO.

Didn’t OpenAI get caught bribing their way to pre-tournament access of the questions?

This is the first time I hear about this. (It's certainly possible, but I'd need to see some evidence or at least a write-up.)

OpenAI got flamed over announcing their results before the embargo was up:

IMO had asked companies to wait at least a week or so after the human winners were announced to announce the AI results. OpenAI did not wait.

Post reply on HN