Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

201–210 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#201

Post by ctoth »

ActorNightly wrote: post #135

Not a decade. More like a century, and that is if society figures itself out enough to do some engineering on a planetary scale, and quantum computing is viable. Fundamentally, AGI requires 2 things. First it needs to be able to operate without information, learning as it goes. The core kernel should be such that it doesn't have any sort of training on real world concepts, only general language parsing that it can us…

You want a "core kernel" with "general language parsing" but no training on real-world concepts.

Read that sentence again. Slowly.

What do you think "general language parsing" IS if not learned patterns from real-world data? You're literally describing a transformer and then saying we need to invent it.

And your TLS example is deranged. You want an agent to discover the TLS protocol by randomly sending ethernet packets? The combinatorial search space is so large this wouldn't happen before the sun explodes. This isn't intelligence! This is bruteforce with extra steps!

Transformers already ARE general algorithms with zero hardcoded linguistic knowledge. The architecture doesn't know what a noun is. It doesn't know what English is. It learns everything from data through gradient descent. That's the entire damn point.

You're saying we need to solve a problem that was already solved in 2017 while claiming it needs a century of quantum computing.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#202

Post by ekjhgkejhgk »

exe34 wrote: post #181
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

To me, it's a matter of a very big checklist - you can keep adding tasks to the list, but if it keeps marching onwards checking things off your list, some day you will get there. whether it's a linear or asymptotic march, only time will tell.

I don't know if you will get there, that's far from clear at this stage.

Did you see the recent video by Nick Beato [1] where he asks various models about a specific number? The models that get it right are the models that consume youtube videos, because there was a youtube video about that specific number. It's like, these models are capable of telling you about very similar things that they've seen, but they don't seem like they understand it. It's totally unclear whether this is a quantitative or qualitative gap.

[1] https://www.youtube.com/watch?v=TiwADS600Jc

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#203

Post by rpcope1 »

omidsa1 wrote: post #146
Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

I also quite like the way he puts it. However, from a certain point onward, the AI itself will contribute to the development—adding nines—and that’s the key difference between this analogy of nines in other systems (including earlier domain‑specific ML ones) and the path to AGI. That's why we can expect fast acceleration to take off within two years.

> However, from a certain point onward, the AI itself will contribute to the development—adding nines—and that’s the key difference between this analogy of nines in other systems (including earlier domain‑specific ML ones) and the path to AGI.

There's a massive planet-sized CITATION NEEDED here, otherwise that's weapons grade copium.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#204

Post by throwaway-0001 »

kachapopopow wrote: post #6

AGI is already here if you shift some goal posts :) From skimming the conversation it seems to mostly revolve around LLMs (transformer models) which is probably not going to be the way we obtain AGI to begin with, frankly it is too simple to be AGI, but the reason why there's so much hype is because it is simple to begin with so really I don't know.

A transistor is very simple too, and here we are. Don’t dismiss something because it’s simple.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#206

Post by red75prime »

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The question is how many nines are humans.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#207

Post by chrisweekly »

nopinsight wrote: post #124

A definition of AGI: https://www.agidefinition.ai/ A new contribution by quite a few prominent authors. One of the better efforts at defining AGI *objectively*, rather than through indirect measures like economic impact. I believe it is incomplete because the psychological theory it is based on is incomplete. It is definitely worth discussing though. —- In particular, creative problem solving in the strong sense, ie…

I agree it seems like a better-structured effort than many others. But its shortcomings go beyond a shallow and incomplete foundation in psychology. It also has basic errors in its execution, eg a "Geography" question about centripetal and centrifugal forces. Color me extremely skeptical.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#208

Post by dingnuts »

nadermx wrote: post #190

I bet you we are all wrong and some random person is going to vibe code himself into something none of us expected. I half kid, if none of you have see it, highly suggest https://karpathy.ai/zero-to-hero.html

Then why didn't Karpathy vibe code this?

https://x.com/GaryMarcus/status/1978500888521068818

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#209

Post by password54321 »

ecocentrik wrote: post #127
kachapopopow wrote: post #6

AGI is already here if you shift some goal posts :) From skimming the conversation it seems to mostly revolve around LLMs (transformer models) which is probably not going to be the way we obtain AGI to begin with, frankly it is too simple to be AGI, but the reason why there's so much hype is because it is simple to begin with so really I don't know.

LLMs are close enough to pass the Turing Test. That was a huge milestone. They are capable of abstract reasoning and can perform many tasks very well but they aren't AGI. They can't teach themselves to play chess at the level of a dedicated chess engine or fly an airplane using the same model they use to copypasta a React UI. They can only fool non-proficient humans into believing that they might be capable of doing…

Turing Test was a thought experiment not a real benchmark for intelligence. If you read the paper the idea originated from it is largely philosophical.

As for abstract reasoning, if you look at ARC-2 it is barely capable though at least some progress has been made with the ARC-1 benchmark.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#210

Post by asah »

jb1991 wrote: post #58

I would bet all of my assets of my life that AGI will not be seen in the lifetime of anyone reading this message right now. That includes anyone reading this message long after the lives of those reading it on its post date have ended. Which of course raises the interesting question of how I can make good on this bet.

Depends on the definition, I might take that bet because under some definitions were already here.

Example: better than average human across many thinking tasks is done.

Post reply on HN