Andrej Karpathy – It will take a decade to work through the issues with agents

Post by scrollop » Sat, Oct 18, 2025, 5:14 AM UTC

visarga wrote: post #612

dreambuffer wrote: post #509

Earlier quoted context omitted.

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to comp…

> then the human came up with language to describe that and then encoded the language into the LLM No individual human invented language, we learn it from other people just like AI. I go as far as to say language was the first AGI, we've been riding the coats tails of language for a long time.

Post by dang » Sat, Oct 18, 2025, 5:17 AM UTC

ctoth wrote: post #201

ActorNightly wrote: post #135

Not a decade. More like a century, and that is if society figures itself out enough to do some engineering on a planetary scale, and quantum computing is viable. Fundamentally, AGI requires 2 things. First it needs to be able to operate without information, learning as it goes. The core kernel should be such that it doesn't have any sort of training on real world concepts, only general language parsing that it can us…

You want a "core kernel" with "general language parsing" but no training on real-world concepts. Read that sentence again. Slowly. What do you think "general language parsing" IS if not learned patterns from real-world data? You're literally describing a transformer and then saying we need to invent it. And your TLS example is deranged. You want an agent to discover the TLS protocol by randomly sending ethernet packe…

Post by vasco » Sat, Oct 18, 2025, 5:17 AM UTC

danielmarkbruce wrote: post #577

DanHulton wrote: post #570

Earlier quoted context omitted.

The thing about this, though - cars have been built before. We understand what's necessary to get those 9s. I'm sure there were some new problems that had to be solved along the way, but fundamentally, "build good car" is known to be achievable, so the process of "adding 9s" there makes sense. But this method of AI is still pretty new, and we don't know it's upper limits. It may be that there are no more 9s to add, o…

While you are right about the broader (and sort of ill defined) chase toward 'AGI' - another way to look at it is the self driving car - they got there eventually.And, if you work on applications using LLMs you can pretty easily see that Karpathy's sentiment is likely correct. You see it because you do it. Even simple applications are shaped like this, albeit each 9 takes less time than self driving cars for a simple…

Post by dang » Sat, Oct 18, 2025, 5:20 AM UTC

lofaszvanitt wrote: post #516

[flagged]

Post by simonh » Sat, Oct 18, 2025, 5:21 AM UTC

FloorEgg wrote: post #590

del82 wrote: post #569

Earlier quoted context omitted.

I mean, I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. So I’d agree that the night sky is “beautiful”, but not because I somehow have access to a deeper meaning of the word or the sky than an LLM does. As someone who (long ago) studied philosophy of mind and (Chomskian) li…

The more I learn about AI, biology and the brain, the more it seems to me that the difference between life and machines is just complexity. People are just really really complex machines. However there are clearly qualitative differences between the human mind and any machines we know of yet, and those qualitative differences are emergent properties, in the same way that a rabbit is qualitatively different than a sto…