Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

921–930 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#921

Post by joe_the_user »

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The thing is, the example of the "march of nines" is self-driving cars. These deal with roads and roads are interface between the chaos of the overall world and a system that has quite well-defined rules.

I can imagine other task on a human/rules-based "frontier" would have a similar quality. But I think there are others that are going to be inaccessible entirely "until AGI" (or something). Humanoid robots moving freely in human society would an example I think.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#922

Post by K0balt »

alganet wrote: post #816
K0balt wrote: post #815

Earlier quoted context omitted.

Yeah, it’s definitely some kind of new chapter. It’s reducing hiring, and will drive unemployment, no matter what people are saying. It’s a poison pill in a way, since no one will hire junior staff anymore. The reliance on AI will skyrocket as experienced staff ages out and there are no replacements coming up through the ranks.

https://en.wikipedia.org/wiki/Chewbacca_defense

I may have missed the target of this reference, but I enjoyed it nonetheless. The CD definitely seems to be gaining traction in the last decade.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#923

Post by giardini »

throwaway-0001 wrote: post #199
mkbelieve wrote: post #114

I don't understand how anyone can believe that we're near even a whiff of AGI when we barely understand what dreaming is, or how the human brain interacts with the quantum world. There are so many elements of human creativity that are still utterly hidden behind a wall that it makes me feel insane when an entire industry is convinced we're just magically going to have the answer soon. The people heralding the emergen…

We Dont know how a horse works, but we got cars. Analogy doesn’t work.

What's your point here? Be careful: language w/o analogy might be impossible. But both are asides to the ongoing discussion.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#924

Post by giardini »

kazinator wrote: post #455

AGI is forever away if you stay blinkered to the LLM/difussion stochastic parrot program, which might have already passed its peak.

Unless you can figure out the math and provide an explanatory context for it's behavior.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#925

Post by JumpCrisscross »

ben_w wrote: post #920
JumpCrisscross wrote: post #303

Earlier quoted context omitted.

> Your brain is doing computation with neurotransmitters instead of transistors If it is, sure. But this isn't a given. We don't actually understand how the brain computes, as evidenced by our inability to simulate it. > Evolution didn't discover some mystical process that imbues meat with special properties Sure. But the complexity remains beyond our comprehension. Against the (nearly) binary action potential of a t…

> Straw man. Who said this? If anything, the symbolic linguists have been overpromising on this front since the 1980s. I'm sure I've seen people say this about language translation and playing go. Ditto chess, way back before Kasparov lost. I don't think I've seen anyone so specific as to say that about medical licensing exams, nor as vague as "write code", but on the latter point I do even now see people saying that…

Fair enough. I’m not going to argue nobody said anything. What I’ll contest is that anyone of consequence said it with consequence. These beliefs didn’t slow down the field. They didn’t stop it from raising capital or attracting engineers.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#926

Post by giardini »

superconduct123 wrote: post #233

I always get a weird feeling when AI researchers and CS people start talking about comparisons between human brains and AI/computers Why is there a presumption that we (as people who have only studied CS) know enough about biology/neuroscience/evolution to make these comparisons/parallels/analogies? I enjoy the discussions but I always get the thought in the back of my head "...remember you're listening to 2 CS major…

There are plenty of mathematicians, psychologists, philosophers, physicists et al that are listening in. Perhaps one day, one or more of these will drop the (probably math) that will achieve critical mass (AGI).

There are two periods in history that "feel" like this time to me: - prior to Einstein's theory of relativity and - the uncovering of quantum mechanics.

In both cases bits and pieces of math and science were floating in the air but no one could connect them. It took teams of people/individuals and years of arduous effort to pull it all together.

Today there are a lot more participants. Main difference seems that a lot of them seem to be capitalists!8-))

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#927

Post by giardini »

matusp wrote: post #309
superconduct123 wrote: post #233

I always get a weird feeling when AI researchers and CS people start talking about comparisons between human brains and AI/computers Why is there a presumption that we (as people who have only studied CS) know enough about biology/neuroscience/evolution to make these comparisons/parallels/analogies? I enjoy the discussions but I always get the thought in the back of my head "...remember you're listening to 2 CS major…

We should completely strip all this talk from AI as a field (and get rid of that name as well). It just causes endless confusion, especially for general audience. In the end, the whole shtick with LLMs is that we train matrices to predict next tokens. You can explain this entire concept without invoking AGI, Roko's basilisk, the nature of human consciousness, and all the other mumbo jumbo that tries so hard to make t…

I eagerly await your publications. I will buy your book.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#928

Post by HarHarVeryFunny »

Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

I think the point Andrej was making here is that in some areas, such as self driving, the cost of failure is extremely high (maybe death), so 99.9% reliable doesn't cut it, and therefore doesn't mean you are almost done, or have done 99.9% of the work. It's "The last 10% is 90% of the work" recursively applied.

He was also pointing out that the same high cost of failure consideration applies to many software systems (depending on what they are doing/controlling). We may already be at the level where AI coding agents are adequate for some less critical applications, but yet far away from them being a general developer replacement. I see software development as something that uses closer to 100% of your brain than 10% - we may well not see AI coding agents approach human reliability levels until we have human level AGI.

The AI snake oil salesmen/CEOs like to throw out competitive coding or math olympiad benchmarks as if they are somehow indicative of the readiness of AI for other tasks, but reliability matters. Nobody dies or loses millions of dollars if you get a math problem wrong.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#929

Post by baobun »

theptip wrote: post #913
ekjhgkejhgk wrote: post #901

Earlier quoted context omitted.

> This isn’t the claim, obviously. This is precisely the claim that leads a of lot people to believe that all you need to reach AGI is more compute.

What I mean here is that this is certainly not what Dwarkesh would claim. It’s a ludicrous strawman position. Dwarkesh is AGI-pilled and would base his assumption of a world model on much more impressive feats than mere language understanding.

Watching the video it seems that Dwarkesh doesn't really have a clue what he's confidently talking about yet running fast with his personal half-baked ideas, to the points where it gets both confusing and cringe when Karpathy apparently manages to make sense of it and yes-anding the word salad AK. Karpathy is supposedly there to clear up misunderstanding yet lets all the nonsense Dwarkesh is putting before him slide.

"ludicrous" sure but I wouldn't be so certain about "strawman" or that Dwarkesh has a consistent view.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#930

Post by danans »

LarsDu88 wrote: post #880
danans wrote: post #835

Earlier quoted context omitted.

> It's incredibly difficult to compress information without have at least some internal model of that information. Whether that model is a "world model" that fits the definition of folks like Sutton and LeCunn is semantic. Sutton's emphasizes his point by saying is that LLMs trying to reach AGI is futile because their world models are less capable that a squirrel's, in part because the squirrel has direct experiences…

This is actually a pretty good point, but quite honestly isn't this just an implementation detail? We can wire up a squirrel robot, give it a wifi connection to a Cerebras inference engine with a big context window, then let it run about during the day collecting a video feed while directing it to do "squirrel stuff". Then during the night, we make it go to sleep and use the data collected during the day to continue…

> then let it run about during the day collecting a video feed while directing it to do "squirrel stuff".

Your phrase "squirrel stuff" is doing a lot of work.

What are the robo-squirrels "goals" and how does it relate to the physical robot?

Is it going around trying to find spare electronic parts to repair itself and reproduce? How does the video feed data relate to its goals?

Where do these goals come from?

Despite all their expensive training, LLMs do not emerge goals. Why would they emerge for your robot squirrel, especially when the survival of its brain is not dependent on the survival of its mechanical body.

Post reply on HN