Andrej Karpathy – It will take a decade to work through the issues with agents • Hacker News Board

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#671

Post by j16sdiz » Sat, Oct 18, 2025, 7:05 AM UTC

ziofill wrote: post #545

dreambuffer wrote: post #509

Earlier quoted context omitted.

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to comp…

I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.

Beauty standard changes over time, see how people perceive body fat in the past few hundred years. We learns what is beautiful from our peers.

Taste can be acquired and can be cultural. See how people used to had their coffee.

Comparing human to LLM is like comparing something constantly changing to something random -- we can't compare them directly, we need a good model for each of them before comparing.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#672

Post by rohit89 » Sat, Oct 18, 2025, 7:10 AM UTC

CSSer wrote: post #27

Topgamer7 wrote: post #5

Whenever someone brings up "AI", I tell them AI is not real AI. Machine learning is a more apt buzzword. And real AI is probably like fusion. Its always 10 years away.

The best part of this is I watched Sam Altman say he really thinks fusion is a short period of time away in response to a question about energy consumption a couple years ago. That was the moment I knew he's a quack.

Sam is an investor in a fusion startup. In any case, how long it takes us to get to working fusion is proportional to the amount of funding it recieves. I'm hopeful that increased energy needs will spur more investment into it.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#673

Post by moozilla » Sat, Oct 18, 2025, 7:17 AM UTC

dang wrote: post #518

hax0ron3 wrote: post #355

If the transcript is accurate, Karpathy does not actually ever, in this interview, say that AGI is a decade away, or make any concrete claims about how far away AGI is. Patel's title is misleading.

Hmm good point. I skimmed the transcript looking for an accurate, representative quote that we could use in the title above. I couldn't exactly find one (within HN's 80 char limit), so I cobbled together "It will take a decade to get agents to work", which is at least closer to what Karpathy actually said. If anyone can suggest a more accurate and representative title, we can change it again. Edit: I thought of using…

You could go with the title from the associated YouTube video (https://www.youtube.com/watch?v=lXUZvyajciY)?

Andrej Karpathy — “We’re summoning ghosts, not building animals”

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#674

Post by Bengalilol » Sat, Oct 18, 2025, 7:19 AM UTC

consumer451 wrote: post #642

sarchertech wrote: post #557

Earlier quoted context omitted.

He’s not just talking about agents good enough to replace workers. He’s talking about whether agents are currently useful at all. >Overall, the models are not there. I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it’s not. It’s slop. They’re not coming to terms with it, and maybe they’re trying to fundraise or something like that. I’m not sure what’s going on,…

I am just some shmoe, but I agree with that assessment. My biggest take-away is that we got super lucky. At least now we have a slight chance to prepare for the potential economic and social impacts.

I am thinking the same.

And we should start considering on what makes us humans and how we can valorize our common ground.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#675

Post by jsiepkes » Sat, Oct 18, 2025, 7:20 AM UTC

Unless someone can show me some sort of "Moore's law" for LLM's, saying it will "take a decade" sounds more to me like it could "take 10 years for the next 20 years".

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#676

Post by misnome » Sat, Oct 18, 2025, 7:24 AM UTC

yoyohello13 wrote: post #282

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

I think a ton of people see a line going up and they think exponential. When in Reality, the vast majority of the time it’s actually logistic.

I mean the cost line does look somewhat exponential…

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#677

Post by pjmlp » Sat, Oct 18, 2025, 7:29 AM UTC

Regardless of the time, I am already seeing programming as we know it slowly moving into prompts, in what concerns low coding environments for SaaS products integrations.

Dealing with Rust's borrow checker issues, how complex C++ might be, Go's approach to language design, Java vs C#, and whatever else in the same vein, will slowly be matter of discussion to a selected few, while everyone else is promoting or doing voice dictation, creating kaban tickets for agents.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#678

Post by zwnow » Sat, Oct 18, 2025, 7:38 AM UTC

ekjhgkejhgk wrote: post #134

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

A world model can not exist, the context windows aren't even near big enough for that. Weird that every serious scientist agrees on AGI not being a thing in the next decades. LLMs are good if you train them for a specific thing. Not so much if you expect them to explain the whole world to you. This is not possible yet.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#679

Post by tim333 » Sat, Oct 18, 2025, 7:39 AM UTC

jb1991 wrote: post #630

tim333 wrote: post #504

Earlier quoted context omitted.

I'd bet the other way because I think Moore's law like advances in compute will make things much easier for researchers. Like I was watching Hinton explain LLMs to Jon Stewart and they were saying they came up with the algorithm in 1986 but then it didn't really work for the decades until now because the hardware wasn't up to it ( https://youtu.be/jrK3PsD3APk?t=1899 ) If things were 1000x faster you could semi random…

You’re making the common assumption that “the algorithm“ is everything we need to get to AGI and it’s just a question of scaling.

I guess so. Is there reason to think an appropriate algorithm and scale can't do that?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#680

Post by _fizz_buzz_ » Sat, Oct 18, 2025, 7:41 AM UTC

tyre wrote: post #226

ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

There is some evidence from Anthropic that LLMs do model the world. This paper[0] tracing their "thought" is fascinating. Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages. It does have clusters of parameters that correlate with concepts, not just randomly "after X word tends to have Y word." Otherwise you would expect…

> Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages.

That doesn't seem surprising at all. My understanding is that transformers where invented exactly for the application of translations. So, concepts must be grouped together in different languages. That was originally the whole point and then turned out to be very useful for broader AI applications.