Andrej Karpathy – It will take a decade to work through the issues with agents • Hacker News Board

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#601

Post by zaphos » Sat, Oct 18, 2025, 4:30 AM UTC

ekjhgkejhgk wrote: post #134

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

"just a matter of adding more 9s" is a wild place to use a "just" ...

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#602

Post by p1esk » Sat, Oct 18, 2025, 4:34 AM UTC

vlovich123 wrote: post #542

tyre wrote: post #226

Earlier quoted context omitted.

There is some evidence from Anthropic that LLMs do model the world. This paper[0] tracing their "thought" is fascinating. Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages. It does have clusters of parameters that correlate with concepts, not just randomly "after X word tends to have Y word." Otherwise you would expect…

If it was modeling the world you’d expect “give me a picture of a glass filled to the brim” to actually do that. It’s inability to correctly and accurately combine concepts indicates it’s probably not building a model of the real world.

I just gave chatgpt this prompt - it produced a picture of a glass filled to the brim with water.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#603

Post by throwaway-0001 » Sat, Oct 18, 2025, 4:35 AM UTC

AnimalMuppet wrote: post #163

throw54465665 wrote: post #150

Earlier quoted context omitted.

[flagged]

Most people cannot comprehend an audiobook? No way. If you have evidence for that claim, show it. Otherwise, no, you're just making stuff up.

Did you ever had a mainstream product and answered customer questions? You should try to see what it truly means an average person.

Examples:

Send email with subject “I need support” (no body).

I answer by email: what you need?

Reply: I need to activate email support

…

Truly agi.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#604

Post by phoenixreader » Sat, Oct 18, 2025, 4:40 AM UTC

A decade is nothing. If issues will be worked through a decade from now, that means the best time to think about opportunities/coonsequences related to that is now.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#605

Post by theptip » Sat, Oct 18, 2025, 4:42 AM UTC

ekjhgkejhgk wrote: post #134

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

> LLMs seem to udnerstand language therefore they've trained a model of the world.

This isn’t the claim, obviously. LLMs seem to understand a lot more than just language. If you’ve worked with one for hundreds of hours actually exercising frontier capabilities I don’t see how you could think otherwise.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#606

Post by golergka » Sat, Oct 18, 2025, 4:42 AM UTC

hashiyakshmi wrote: post #578

klipt wrote: post #563

Earlier quoted context omitted.

What about a blind human? Are they just like an LLM? What about a multimodal model trained on video? Is that like a human?

This is actually a great point but for the opposite reason - if you ask a blind person if the night sky is beautiful, they would say they don't know because they've never seen it (they might add that they've heard other people describe it as such). Meanwhile, I just asked ChatGPT "Do you think the night sky is beautiful?" And it responded "Yes, I do..." and went on to explain why while describing senses its incapable…

Wha if you asked the blind man to play the role of helpful assistant

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#607

Post by palmotea » Sat, Oct 18, 2025, 4:47 AM UTC

galangalalgol wrote: post #25

CaptainOfCoit wrote: post #15

Earlier quoted context omitted.

And a program that can write, sound and paint like a human was 20 years away perpetually as well, until it wasn't.

This is the key insight I believe. It is inherently unpredictable. There are species that pass the mirror test with a far fewer equivalent number of parameters than large models are using already. Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi, but that it might take decades to stumble on that way, or someone might find it…

> Carmack has said something to the effect that about 10ksloc would glue the right existing achictectures together in the right way to make agi

What does he know about that?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#608

Post by adrianN » Sat, Oct 18, 2025, 4:49 AM UTC

dreambuffer wrote: post #509

LarsDu88 wrote: post #443

Earlier quoted context omitted.

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic co…

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to comp…

The human experience is also several degrees removed from the „real“ world. I don’t think sensory chauvinism is a useful tool in assessing intelligence potential.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#609

Post by seydor » Sat, Oct 18, 2025, 4:51 AM UTC

Zacharias030 wrote: post #560

With all due respect, what does it say about us that „famous researcher voices his speculative opinion“ is an instant top 1 on hackernews?

Is it worse than "Rich CEO expressing certainly over his hunch"

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#610

Post by jdiff » Sat, Oct 18, 2025, 4:52 AM UTC

p1esk wrote: post #602

vlovich123 wrote: post #542

Earlier quoted context omitted.

If it was modeling the world you’d expect “give me a picture of a glass filled to the brim” to actually do that. It’s inability to correctly and accurately combine concepts indicates it’s probably not building a model of the real world.

I just gave chatgpt this prompt - it produced a picture of a glass filled to the brim with water.

Like most quirks that spread widely, a bandaid is swiftly applied. This is also why they now know how many r's are in "strawberry." But we don't get any closer to useful general intelligence by cobbling together thousands of hasty patches.