Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#271

Post by ben_w »

sysguest wrote: post #245
ben_w wrote: post #173

Earlier quoted context omitted.

> babies are already born with "the model of the world" > but a lot of experiments on babies/young kids tell otherwise I believe they are born with such a model? It's just that model is one where mummy still has fur for the baby to cling on to? And where aged something like 5 to 8 it's somehow useful for us to build small enclosures to hide in, leading to a display of pillow forts in the modern world?

damn I guess I had to be more specific: "LLM-level world-detail knowledge"

I think I'm even more confused now about what you mean…

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#273

Post by rootusrootus »

asah wrote: post #210
jb1991 wrote: post #58

I would bet all of my assets of my life that AGI will not be seen in the lifetime of anyone reading this message right now. That includes anyone reading this message long after the lives of those reading it on its post date have ended. Which of course raises the interesting question of how I can make good on this bet.

Depends on the definition, I might take that bet because under some definitions were already here. Example: better than average human across many thinking tasks is done.

I think that the definition needs to include something about performance on out-of-training tasks. Otherwise we're just talking about machine learning, not anything like AGI.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#275

Post by imadierich »

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

[dead]

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#276

Post by imadierich »

exe34 wrote: post #181
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

To me, it's a matter of a very big checklist - you can keep adding tasks to the list, but if it keeps marching onwards checking things off your list, some day you will get there. whether it's a linear or asymptotic march, only time will tell.

[dead]

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#277

Post by robotresearcher »

godelski wrote: post #224
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

> that to understand knowledge you have to have a model of the world. You have a small but important mistake. It's to recite (or even apply ) knowledge. To understand does actually require a world model. Think of it this way: can you pass a test without understanding the test material? Certainly we all saw people we thought were idiots do well in class while we've also seen people we thought were geniuses fail. The t…

In my view 'understand' is a folk psychology term that does not have a technical meaning. Like 'intelligent', 'beautiful', and 'interesting'. It usefully labels a basket of behaviors we see in others, and that is all it does.

In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. There's no problem of how to do understanding, only how to do tasks. The 'problem' melts away when you take this stance.

Just my opinion, but my professional opinion from thirty-plus years in AI.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#278

Post by newsclues »

dlcarrier wrote: post #241

Redefinitions aside, fully capable AI is right up there with commercially viable fusion power, cost effective quantum completing, and fully capable self-driving cars, as a technology that is quickly advancing yet always a decade or two away.

What was the last example where humans succeeded at a hard problem like that?

Space flight?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#279

Post by bentt »

ekjhgkejhgk wrote: post #134
Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

I think this a useful challenge to our normal way of thinking.

At the same time, "the world" exists only in our imagination (per our brain). Therefore, if LLMs need a model of a world, and they're trained on the corpus of human knowledge (which passed through our brains), then what's the difference, especially when LLMs are going back into our brains anyway?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#280

Post by hackinthebochs »

SR2Z wrote: post #246
tyre wrote: post #226

Earlier quoted context omitted.

There is some evidence from Anthropic that LLMs do model the world. This paper[0] tracing their "thought" is fascinating. Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages. It does have clusters of parameters that correlate with concepts, not just randomly "after X word tends to have Y word." Otherwise you would expect…

Right, but modeling the structure of language is a question of modeling word order and binding affinities. It's the Chinese Room thought experiment - can you get away with a form of "understanding" which is fundamentally incomplete but still produces reasonable outputs? Language in itself attempts to model the world and the processes by which it changes. Knowing which parts-of-speech about sunrises appear together an…

LLMs aren't just modeling word co-occurrences. They are recovering the underlying structure that generates word sequences. In other words, they are modeling the world. This model is quite low fidelity, but it should be very clear that they go beyond language modeling. We all know of the pelican riding a bicycle test [1]. Here's another example of how various language models view the world [2]. At this point it's just bad faith to claim LLMs aren't modeling the world.

[1] https://simonwillison.net/2025/Aug/7/gpt-5/#and-some-svgs-of...

[2] https://www.lesswrong.com/posts/xwdRzJxyqFqgXTWbH/how-does-a...

Post reply on HN