Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs aren't world models

yosefk.com

Post reply on HN
Print view View original

141–150 of 240 posts

Re: LLMs aren't world models

#141

Post by ehnto »

marcellus23 wrote: post #58
WillPostForFood wrote: post #51

Earlier quoted context omitted.

Your LLM output seems abnormally bad, like you are using old models, bad models, or intentionally poor prompting. I just copied and pasted your Krita example into ChatGPT, and reasonable answer, nothing like what you paraphrased in your post. https://imgur.com/a/O9CjiJY

I think it's hard to take any LLM criticism seriously if they don't even specify which model they used. Saying "an LLM model" is totally useless for deriving any kind of conclusion.

When talking about the capabilities of a class of tools long term, it makes sense to be general. I think deriving conclusions at all is pretty difficult given how fast everything is moving, but there is some realities we do actually know about how LLMs work and we can talk about that.

Knowing that ChatGPT output good tokens last tuesday but Sonnet didn't does not help us know much about the future of the tools on general.

Re: LLMs aren't world models

#142

Post by interstice »

jononor wrote: post #96
moduspol wrote: post #61

Earlier quoted context omitted.

"Deal with" and "autonomously" are doing a lot of heavy lifting there. Cursor already does a pretty good job indexing all the files in a code base in a way that lets it ask questions and get answers pretty quickly. It's just a matter of where you set the goalposts.

"LLM" as well, because coding agents are already more than just an LLM. There is very useful context management around it, and tool calling, and ability to run tests/programs, etc. Though they are LLM-based systems, they are not LLMs.

This rapidly gets philosophical. If I use tools am I not handling the codebase? Are we classing LLM as tool or user in this scenario?

Re: LLMs aren't world models

#143

Post by nhaehnle »

lordnacho wrote: post #20

Here's what LLMs remind me of. When I went to uni, we had tutorials several times a week. Two students, one professor, going over whatever was being studied that week. The professor would ask insightful questions, and the students would try to answer. Sometimes, I would answer a question correctly without actually understanding what I was saying. I would be spewing out something that I had read somewhere in the huge…

Good on you for having the meta-cognition to recognize it.

I've graded many exams in my university days (and set some myself), and it's exceedingly obvious that that's what many students are doing. I do wonder though how often they manage to fly under the radar. I'm sure it happens, as you described.

(This is also the reason why I strongly believe that in exams where students write free-form answers, points should be subtracted for incorrect statements even if a correct solution is somewhere in the word salad.)

Re: LLMs aren't world models

#144

Post by grey-area »

eru wrote: post #126
p1esk wrote: post #60

Earlier quoted context omitted.

Yes, I’d be curious about his experience with GPT-5 Thinking model. So far I haven’t seen any blunders from it.

I've seen plenty of blunders, but in general it's better than their previous models. Well, it depends a bit on what you mean by blunders. But eg I've seen it confidently assert mathematically wrong statements with nonsense proofs, instead of admitting that it doesn't know.

In a very real sense it doesn’t even know that it doesn’t know.

Re: LLMs aren't world models

#145

Post by intended »

eru wrote: post #138
2muchcoffeeman wrote: post #133

Earlier quoted context omitted.

It’s not just on the job learning though. I’m no AI expert, but the fact that you have “prompt engineers” and AI doesn’t know what it doesn’t know, gives me pause. If you ask an expert, they know the bounds of their knowledge and can understand questions asked to them in multiple ways. If they don’t know the answer, they could point to someone who does or just say “we don’t know”. LLMs just lie to you and we call it…

> I’m no AI expert, but the fact that you have “prompt engineers” [...] gives me pause. Why? A bunch of human workers can get a lot more done with a capable leader who helps prompt them in the right direction and corrects oversights etc. And overall, prompt engineering seems like exactly the kind of skill AI will be able to develop by itself. You already have a bit like this happening: when you ask Gemini to create a…

This is the goalpost flip which happens in AI conversations. If goalpost is even the right term, conversation switch?

Theres 2 AI conversations on HN occurring simultaneously.

Convo A: Is it actually reasoning? does it have a world model? etc..

Convo B: Is it good enough right now? (for X, Y, or Z workflow)

Re: LLMs aren't world models

#146

Post by samrus »

Uehreka wrote: post #110
geraneum wrote: post #57

Earlier quoted context omitted.

> it wrote by itself as part of a larger task I had given it, so it certainly understands transparency Or it’s a common step or a known pattern or combination of steps that is prevalent in its training data for certain input. I’m guessing you don’t know what’s exactly in the training sets. I don’t know either. They don’t tell ;) > but it adapts and combines them well to suit my particular requirements. So they seem t…

> We tend to overestimate the novelty of our own work and our methods and at the same time, underestimate the vastness of the data and information available online for machines to train on. LLMs are very sophisticated pattern recognizers. If LLMs are stochastic parrots, but also we’re just stochastic parrots, then what does it matter? That would mean that LLMs are in fact useful for many things (which is what I care…

We're not just stochastic parrots though, we can parrot things stochastically when that has utility, but we can also be original. The first time that work was done, it was sone by a person, autonomously. Current LLMs couldnt have done it the first time

Re: LLMs aren't world models

#147

Post by eru »

intended wrote: post #145
eru wrote: post #138

Earlier quoted context omitted.

> I’m no AI expert, but the fact that you have “prompt engineers” [...] gives me pause. Why? A bunch of human workers can get a lot more done with a capable leader who helps prompt them in the right direction and corrects oversights etc. And overall, prompt engineering seems like exactly the kind of skill AI will be able to develop by itself. You already have a bit like this happening: when you ask Gemini to create a…

This is the goalpost flip which happens in AI conversations. If goalpost is even the right term, conversation switch? Theres 2 AI conversations on HN occurring simultaneously. Convo A: Is it actually reasoning? does it have a world model? etc.. Convo B: Is it good enough right now? (for X, Y, or Z workflow)

Maybe, yes. It's good to acknowledge that both of these conversations are worthwhile to have.

Re: LLMs aren't world models

#148

Post by eru »

grey-area wrote: post #144
eru wrote: post #126

Earlier quoted context omitted.

I've seen plenty of blunders, but in general it's better than their previous models. Well, it depends a bit on what you mean by blunders. But eg I've seen it confidently assert mathematically wrong statements with nonsense proofs, instead of admitting that it doesn't know.

In a very real sense it doesn’t even know that it doesn’t know.

Maybe. But in math you can either produce the proof (with each step checkable) or you can't.

Re: LLMs aren't world models

#149

Post by libraryofbabel »

ehnto wrote: post #139
eru wrote: post #125

Earlier quoted context omitted.

Perhaps the people who get decent answers don't write articles about them?

I imagine people give up silently more often than they write a well syndicated article about it. The actual adoption and efficiencies we see in enterprises will be the most verifiable data on if LLMs are generally useful in practice. Everything so far is just academic pontificating or anecdata from strangers online.

This. I think we’ve about reached the limit of the usefulness of anecdata “hey I asked an LLM this this and this” blog posts. We really need more systematic large scale data and studies on the latest models and tools - the recent one on cursor (which had mixed results) was a good start but it was carried out before Claude Code was even released, i.e. prehistoric times in terms of AI coding progress.

For my part I don’t really have a lot of doubts that coding agents can be a useful productivity boost on real-world tasks. Setting aside personal experience, I’ve talked to enough developers at my company using them for a range of tickets on a large codebase to know that they are. The question is more, how much: are we talking a 20% boost, or something larger, and also, what are the specific tasks they’re most useful on. I do hope in the next few years we can get some systematic answers to that as an industry, that go beyond people asking LLMs random things and trying to reason about AI capabilities from first principles.

Re: LLMs aren't world models

#150

Post by eru »

ehnto wrote: post #139
eru wrote: post #125

Earlier quoted context omitted.

Perhaps the people who get decent answers don't write articles about them?

I imagine people give up silently more often than they write a well syndicated article about it. The actual adoption and efficiencies we see in enterprises will be the most verifiable data on if LLMs are generally useful in practice. Everything so far is just academic pontificating or anecdata from strangers online.

I am inclined to agree.

However, I'm not completely sure. Eg object oriented programming was basically a useless fad full of empty, never-delivered-on promises, but software companies still lapped it up. (If you happen to like OOP, you can probably substitute your own favourite software or wider management fad.)

Another objection: even an LLM with limited capabilities and glaring flaws can still be useful for some commercial use-cases. Eg the job of first line call centre agents that aren't allowed to deviate from a fixed script can be reasonable automated with even a fairly bad LLM.

Will it suck occasionally? Of course! But so does interacting with the humans placed into these positions without authority to get anything done for you. So if the bad LLM is cheaper, it might be worthwhile.

Post reply on HN