Earlier quoted context omitted.
you can do that by shorting Oracle here
“Markets can remain irrational longer than you can remain solvent.” - John Maynard Keynes
yeah, of course. just framing the OP's bravado
Andrej Karpathy – It will take a decade to work through the issues with agents
321–330 of 1001 posts
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#322Redefinitions aside, fully capable AI is right up there with commercially viable fusion power, cost effective quantum completing, and fully capable self-driving cars, as a technology that is quickly advancing yet always a decade or two away.
Fusion power seems closer than ever. And plenty of experts just five years ago thought AGI would still be decades away. A credible expert suggesting AGI is ten years away is a sign of real progress.
A credible expert suggesting AGI is ten years away is a sign of good marketing.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#323Earlier quoted context omitted.
It's an excellent time-frame that sounds imminent enough to draw interest (and funding), but is distant enough that you can delay the promised arrival a few times in the span of a career before retiring. Fusion research lives and dies on this premise, ignoring the hard problems that require fundamental breakthroughs in areas such as materials science, in favor of touting arbitrary benchmarks that don't indicate real…
> ”Full self driving" is another example; your car won't be doing this, but companies will brag about limited roll-outs of niche cases in dry, flat, places that are easy to navigate. Not expecting my car to be self-driving anytime soon, but I have understood there is actual working robotaxi service in San Francisco which is not easy or flat? I think we can’t keep saying self driving cars will never happen when this k…
It's true that SF isn't flat, but it's incredibly well mapped, it never snows and you don't have to worry about roads ravaged by frost-heaves. There's a reason that the new Doordash automated delivery service is starting off in Phoenix and not Boston for example.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#324Agency. If one studied the humanities they’d know how incredible a proposal “agentic” AI is. In the natural world, agency is a consequence of death: by dying, the feedback loop closes in a powerful way. The notion of casual agency (I’m thinking of Jensen Huang’s generative > agentic > robotic insistence) is bonkers. Some things are not easily speedrunned. (I did listen to a sizable portion of this podcast while makin…
Models die too - the less agentic ones are out-competed by the more agentic ones.
Every AI lab brags how "more agentic" their latest model is compared to the previous one and the competition, and everybody switches to the new model.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#325Earlier quoted context omitted.
Why not?
Most likely because you'll be filthy reach from selling AGI and won't need to go after secondary revenue sources.
>Most likely because you'll be filthy reach from selling AGI
Why? If AGI costs more than a human or operates slower than one, it may not be economical for people to buy it. By the time it becomes economical, competitors may have also cracked it reducing your ability to charge high margins on it.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#326Earlier quoted context omitted.
There is some evidence from Anthropic that LLMs do model the world. This paper[0] tracing their "thought" is fascinating. Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages. It does have clusters of parameters that correlate with concepts, not just randomly "after X word tends to have Y word." Otherwise you would expect…
> Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages I thought that’s the basic premise of how transformers work - they encode concepts into high dimensional space, and similar concepts will be clustered together. I don’t think it models the world, but just the texts it ingested. It’s observation and regurgitation, not u…
That's a model. Not a higher-order model like most humans use, but it's still a model.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#327Just like fusion. It will revolutionize the world, but it's always just ... one more decade away.
Also, I like like how almost nobody takes issue with a decade time interval. If he means that current LLMs, slowly plateauing in performance, would somehow take a decade to create AI (which he calls AGI)? Where would this fantastical gain in performance come from? Or he thinks it will be a different mechanism as a basis? But then what mechanism, it should be at least real in theory by now if it were to realize in a decade time.
Basically what I mean, is that if LLMs are future real AI basis, it would take less than a decade because they are in diminishing returns today. And if it is something completely new, then what exactly? And if it is something abstract, fuzzy and hypothetical, whence did a decade number come from?
This is basically Sam Altman's "5 to 10 years in the future"(1) all over again. Not less than 5 so as not to be verified in the near future, and no need to show at least something as a prototype or at least scientific theory. And no more than 10 year so as not to scare Softbank and other investors.
(1) https://fortune.com/2025/09/26/sam-altman-openai-ceo-superin...
https://www.forbes.com/sites/jodiecook/2024/07/16/openais-5-...
https://www.tomsguide.com/ai/chatgpt/sam-altman-claims-agi-i...
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#328Earlier quoted context omitted.
In my view 'understand' is a folk psychology term that does not have a technical meaning. Like 'intelligent', 'beautiful', and 'interesting'. It usefully labels a basket of behaviors we see in others, and that is all it does. In this view, if a machine performs a task as well as a human, it understands it exactly as much as a human. There's no problem of how to do understanding, only how to do tasks. The 'problem' me…
Nonsense. A QC operator may be able to carry out a test with as much accuracy (or perhaps better accuracy, with enough practice) than the PhD quality chemist who developed it. They could plausibly do so with a high school education and not be able to explain the test in any detail. They do not understand the test in the same way as the chemist. If 'understand' is a meaningless term to someone who's spent 30 years in…
> They do not understand the test in the same way as the chemist.
Can you explain precisely what 'understand' means here, without using the word 'understand'? I don't think anyone can.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#329Earlier quoted context omitted.
Right, but modeling the structure of language is a question of modeling word order and binding affinities. It's the Chinese Room thought experiment - can you get away with a form of "understanding" which is fundamentally incomplete but still produces reasonable outputs? Language in itself attempts to model the world and the processes by which it changes. Knowing which parts-of-speech about sunrises appear together an…
LLMs aren't just modeling word co-occurrences. They are recovering the underlying structure that generates word sequences. In other words, they are modeling the world. This model is quite low fidelity, but it should be very clear that they go beyond language modeling. We all know of the pelican riding a bicycle test [1]. Here's another example of how various language models view the world [2]. At this point it's just…
and we can say that a bastardized version of the Sapir-Worf hypothesis applies: what's in the training set shapes or limits LLM's view of the world
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#330Earlier quoted context omitted.
The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…
> that to understand knowledge you have to have a model of the world. You have a small but important mistake. It's to recite (or even apply ) knowledge. To understand does actually require a world model. Think of it this way: can you pass a test without understanding the test material? Certainly we all saw people we thought were idiots do well in class while we've also seen people we thought were geniuses fail. The t…
> It's to recite (or even apply) knowledge. To understand does actually require a world model.
This is a shell game, or a god of the gaps. All you're saying is that the models "understand" how to recite or apply knowledge or language, but somehow don't understand knowledge or language. Well what else is there really?