Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

841–850 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#841

Post by theptip »

steveBK123 wrote: post #806
getnormality wrote: post #804

One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today. No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward. Also, speculation…

It's nice seeing commentary from someone who is both knowledgable in AI and NOT trying to pump the AI bag. Right now the median actor in the space loudly proclaims AGI is right around the corner, while rolling out pornbots/ads/in-chat-shopping, which generally seems at odds with a real belief that AGI is close (TAM of AGI must be exponentially larger than the former).

Zvi made this point the other day, and then this counterpoint which I agree with more - if you think AGI is soon but you need to keep up the exponential datacenter growth for 2-3 years (or whatever “around the corner” means for the company in question) then a land-grab on consumer ARR is a faster way to short-term revenue (and therefore higher valuations at your next round).

OAI is also doing F100 and USG work; it takes longer to book the revenue though.

By selling porn and shopping you are in some sense weakening your position with regulators which you'll need when AGI starts displacing jobs - but you can also imagine thinking that this is a second order problem and winning the race is way more urgent.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#842

Post by maccard »

sarchertech wrote: post #228
fair_enough wrote: post #110

Earlier quoted context omitted.

Reminds me of a time-honored aphorism in running: A marathon consists of two halves: the first 20 miles, and then the last 10k (6.2mi) when you're more sore and tired than you've ever been in your life.

Why just run 20 miles then?

Because it would be 16 miles of bliss and 4 miles of torture then. The point is the last section of the run is always significantly harder - it’s even the same for 5k

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#844

Post by sarchertech »

weatherlite wrote: post #821
sarchertech wrote: post #761

Earlier quoted context omitted.

I’m not saying he’s saying agents aren’t useful at all. It’s literally in the quotes I provided that he says they are useful for some subset of tasks. I’m saying that he is answering the question “are agents useful at all”. not “can agents replace humans”. His answer is mostly not. He generally prefers autocomplete. But they are useful for some limited tasks.

> I’m not saying he’s saying agents aren’t useful at all I'm not saying you're saying he's saying agents aren't useful at all

You’re not the person I’m replying to.

The person I’m replying to said

>I don't think he is saying agents are not useful at all, just that they are not anywhere near the capability of human software developers.

Implying I was supporting the first clause.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#845

Post by more_corn »

I can’t believe we’re talking about agents. 1) agents are autonomous actors 2) llms are terrible at achieving consistent outcomes based on rules 3) with current technology AI agents will produce inconsistent and unreliable results.

Therefore turning autonomous actors based on LLMs loose is a recipe for disaster.

It won’t take a decade. That’s an arbitrary statement based on a big unknown. It will take an entirely new technology. One we haven’t invented yet, one I can’t even imagine. One that is consistently accurate and reliable in ways NO EXISTING AI PRODUCT HAS EVER BEEN.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#846

Post by theptip »

zmmmmm wrote: post #757

It's good to see experts with similar scepticism about agents that I have. I don't doubt they will be useful in some settings, but they lean into all the current weak points of large language models and make them worse. Security, reproducibility, hallucinations, bias, etc etc. With all these issues already being hard to manage, I just don't believe businesses are going to delegate processes to autonomous agents in a…

Fully autonomous agents are marketing fluff right now, but there is like $10T of TAM from promoting most knowledge workers to a manager and automating the boring 80% of their work, and this doesn’t require this full autonomy.

Karpathy’s definition of “agent” here is really AGI (probably somewhere between expert and virtuoso AGI https://arxiv.org/html/2311.02462v2). In my taxonomy you can have non-AGI short-task-timeframe agents. Eg in the METR evals, I think it’s meaningful to talk about agent tasks if you set the thing loose for 4-8h human-time tasks.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#847

Post by cmrdporcupine »

leptons wrote: post #154
ciconia wrote: post #36

It's funny how there's such a pervasive cynicism about AI in the developer community, yet everyone is still excited about vibe coding. Strange times...

What developer is excited about "vibe coding"? The only people excited about "vibe coding" are people who can't code.

This of course depend completely on how you define "vibe" coding.

Assisted coding has been incredibly useful. I have been using Claude Code daily.

But if you let it take over completely without review and let it write whole features... which I take to be the meaning of "vibe" in some people's definitions... you're in for a world of long-term pain.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#848

Post by marshfram »

LarsDu88 wrote: post #443
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic co…

There are no world models in biology. Idea Johnson-Laird is being promoted in AI as a solution is sado-masochistic. The brain doesn't compress info about our world, it ecologically relates to it. It doesn't compress, it never has to. How these folk science ideas of the brain entered engineering from cog-sci mistaken complexes and how they remain in power is pretty suspect.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#849

Post by ambicapter »

simonw wrote: post #522

It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: When you’re talking about an agent, or what the labs have in mind and maybe what I have in mind as well, you should think of it almost like an employee or an intern that you would hire to work with you. For example, you work with some employees here. When would yo…

Do you have a comment? Most of what you've said here is a quote.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#850

Post by CamperBob2 »

qlm wrote: post #708
CamperBob2 wrote: post #356

Earlier quoted context omitted.

If it were that simple, LLMs wouldn't work at all.

I think it explains quite well why LLMs are useful in some ways but stupid in many other ways.

LLMs clearly think. They don't have a sense of object permanence, at least not yet, but they absolutely, indisputably use pretrained information to learn and reason about the transient context they're working with at the moment.

Otherwise they couldn't solve math problems that aren't simple rephrasings of problems they were trained on, and they obviously can do that. If you give a multi-step undergraduate level math problem to the human operator of a Chinese room, he won't get very far, while an LLM can.

So that leads to the question: given that they were trained on nothing but language, and given that they can reason to some extent, where did that ability come from if it didn't emerge from latent structure in the training material itself? Language plus processing is sufficient to produce genuine intelligence, or at least something indistinguishable from it. I don't know about you, but I didn't see that coming.

Post reply on HN