Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

531–540 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#531

Post by bamboozled »

breuleux wrote: post #198
omidsa1 wrote: post #146

Earlier quoted context omitted.

I also quite like the way he puts it. However, from a certain point onward, the AI itself will contribute to the development—adding nines—and that’s the key difference between this analogy of nines in other systems (including earlier domain‑specific ML ones) and the path to AGI. That's why we can expect fast acceleration to take off within two years.

I don't think we can be confident that this is how it works. It may very well be that our level of intelligence has a hard limit to how many nines we can add, and AGI just pushes the limit further, but doesn't make it faster per se. It may also be that we're looking at this the wrong way altogether. If you compare the natural world with what humans have achieved, for instance, both things are qualitatively different,…

It's also assuming that all advances in AI just lead to cold hard gains, people have suggested this before but would a sentient AI get caught up in philosophical, silly or religious ideas? Silicone investor types seem to hope it's all just curing diseases they can profit from, but it might also be, "let's compose some music instead"?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#532

Post by eddiewithzato »

simonw wrote: post #522

It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: When you’re talking about an agent, or what the labs have in mind and maybe what I have in mind as well, you should think of it almost like an employee or an intern that you would hire to work with you. For example, you work with some employees here. When would yo…

Because that's the definition that is leading to all these investments, the promise that very soon they will reach it. If Altman said plainly that LLMs will never reach that stage, there would be a lot less investment into the industry.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#533

Post by bamboozled »

tim333 wrote: post #508
hax0ron3 wrote: post #355

If the transcript is accurate, Karpathy does not actually ever, in this interview, say that AGI is a decade away, or make any concrete claims about how far away AGI is. Patel's title is misleading.

He says re agents: >They don't have enough intelligence, they're not multimodal enough, they can't do computer use and all this stuff. They don't do a lot of the things you've alluded to earlier. They don't have continual learning. You can't just tell them something and they'll remember it. They're cognitively lacking and it's just not working. >It will take about a decade to work through all of those issues. (2:20)

Couldn't have even been bothered watching ~ 2 minutes of an interview before commenting.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#534

Post by bamboozled »

qingcharles wrote: post #416
stack_framer wrote: post #259

Just like fusion. It will revolutionize the world, but it's always just ... one more decade away.

I can't use fusion power yet. Several hundred million people are using LLMs every day. There has to be at least two orders of magnitude more investment in "AI" technologies than there are in fusion techs right now.

We're driving LLMs to get results though, which is different to what's being discussed.

Everytime I've used an LLM to achieve something, while useful, it's taken considerable effort on my part.

In fact I don't think I've ever receive anything for free when using any "AI", except maybe saved time by typing.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#535

Post by keeda »

dingnuts wrote: post #208
nadermx wrote: post #190

I bet you we are all wrong and some random person is going to vibe code himself into something none of us expected. I half kid, if none of you have see it, highly suggest https://karpathy.ai/zero-to-hero.html

Then why didn't Karpathy vibe code this? https://x.com/GaryMarcus/status/1978500888521068818

This is actually discussed in the interview: https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits

It seems to be more nuanced than what people have assumed. The best I can summarize it as is that he was doing rather non-standard things that confused the LLMs which have been trained on vast amounts of very standard code and hence kept defaulting to those assumptions.

Maybe a rough analogy is that he was trying to "code golf" this repo while LLMs kept trying to write "enterprise" code because that is overwhelmingly what they have been trained on.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#536

Post by moron4hire »

homarp wrote: post #329
hackinthebochs wrote: post #280

Earlier quoted context omitted.

LLMs aren't just modeling word co-occurrences. They are recovering the underlying structure that generates word sequences. In other words, they are modeling the world. This model is quite low fidelity, but it should be very clear that they go beyond language modeling. We all know of the pelican riding a bicycle test [1]. Here's another example of how various language models view the world [2]. At this point it's just…

and we can say that a bastardized version of the Sapir-Worf hypothesis applies: what's in the training set shapes or limits LLM's view of the world

Neither Sapir nor Whorf presented Linguistic Relativism as their own hypothesis and they never published together. The concept, if it exists at all, is a very weak effect, considering it doesn't reliably replicate.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#537

Post by danielvaughn »

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

I have a very surface level understanding of AI, and yet this always seemed obvious to me. It's almost a fundamental law of the universe that complexity of any kind has a long tail. So you can get AI to faithfully replicate 90% of a particular domain skill. That's phenomenal, and by itself can yield value for companies. But the journey from 90%-100% is going to be a very difficult march.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#539

Post by spjt »

stack_framer wrote: post #259

Just like fusion. It will revolutionize the world, but it's always just ... one more decade away.

The difference with fusion is that we have a very good understanding of how fusion works, and exactly what we need to figure out how to do, to make it a viable energy source. It's basically just an engineering problem, albeit a very difficult one due to the extreme conditions. AGI is more like developing warp drive. With AGI, we really have no idea how the brain works or any clue of what problems need to be solved. It's basically just like the underpants gnomes.

Phase 1: Buying more GPU to increase the number of parameters in a LLM Phase 2: ??? Phase 3: AGI

AGI may come anywhere between next week, 1000 years in the future, or never. Anyone who claims to have any idea is full of shit, because we don't even know what problems we need to solve to get there. If we develop a good model of how human cognition works at a biological level, there is at least a direction, but that isn't going to be coming out of some AI hype factory with a datacenter full of H100's making videos of anthropomorphic cats working as pastry chefs.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#540

Post by aik »

eddiewithzato wrote: post #532
simonw wrote: post #522

It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: When you’re talking about an agent, or what the labs have in mind and maybe what I have in mind as well, you should think of it almost like an employee or an intern that you would hire to work with you. For example, you work with some employees here. When would yo…

Because that's the definition that is leading to all these investments, the promise that very soon they will reach it. If Altman said plainly that LLMs will never reach that stage, there would be a lot less investment into the industry.

Hard disagree. You don’t need AGI to transform countless workflows within companies, current LLMs can do it. A lot of the current investments are to help with the demand with current generation LLMs (and use cases we know will keep opening up with incremental improvements). Are you aware of how intensely all the main companies that host leading models (azure, aws, etc) are throttling usage due to not enough data center capacity? (Eg. At my company we have 100x more demand than we can get capacity for, and we’re barely getting started. We have a roadmap with 1000x+ the current demand and we’re a relatively small company.)

AGI would be more impactful of course, and some use cases aren’t possible until we have it, but that doesn’t diminish the value of current AI.

Post reply on HN