Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

761–770 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#761

Post by sarchertech »

sothatsit wrote: post #699
sarchertech wrote: post #557

Earlier quoted context omitted.

He’s not just talking about agents good enough to replace workers. He’s talking about whether agents are currently useful at all. >Overall, the models are not there. I feel like the industry is making too big of a jump and is trying to pretend like this is amazing, and it’s not. It’s slop. They’re not coming to terms with it, and maybe they’re trying to fundraise or something like that. I’m not sure what’s going on,…

I don't think he is saying agents are not useful at all, just that they are not anywhere near the capability of human software developers. Karpathy later says he used agents to write the Rust translation of algorithms he wrote in Python. He also explicitly says that agents can be useful for writing boilerplate or for code that can be very commonly found online. So I don't think he is saying they are not useful at all…

I’m not saying he’s saying agents aren’t useful at all. It’s literally in the quotes I provided that he says they are useful for some subset of tasks.

I’m saying that he is answering the question “are agents useful at all”. not “can agents replace humans”.

His answer is mostly not. He generally prefers autocomplete. But they are useful for some limited tasks.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#762

Post by TeMPOraL »

chronci739 wrote: post #691
modeless wrote: post #648

Earlier quoted context omitted.

Yeah, I see people pooh-poohing the idea of humanoid robots being useful this decade, saying it will take at least 20 years. Oh yeah? Instead of 5 years to render all human labor obsolete, it will take 20? The magnitude of that change is so large that the implications of it happening anytime in our lifetimes are too big to ignore. The important thing is that this is not going to be perpetually 20 years in the future…

> This is something that will happen. Not in our lifetime. The iPhone came out less than 20 years ago. And what, you scan QR codes at restaurants with iphones?

You do. And so does your non-technical mother (or a friend of your mother).

The impact of the iPhone and its competitors is felt everywhere, it diffused into every domain of people's lives. Think: the whole of social media was pretty much enabled by smartphones.

Or a more pedestrian, random example: every day I go to the office, I see endless store managers, restaurant managers, etc. walking around their store, making photos to upload to HQ. But this is merely a symptom - the actual consequence is the change in busines structure. It's because smartphones make this easy, that it makes franchise and subcontracted businesses more viable, because it's easier for the HQ to micromanage more semi-independent subordinates.

There are many, many more examples like this everywhere you look. Which is why I'm inclined to agree with Karpathy: computers, iPhones, LLMs, are all the same thing - it's just the more notable manifestations of how we've been staying on 2% growth exponential curve for many hundreds of years now, and why we'll continue to stay on this curve.

But the caveat is: that curve is getting steep enough that the world is starting to transform faster than we can handle.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#763

Post by fmbb »

LarsDu88 wrote: post #443
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic co…

Sure but everything is semantics.

LLMs have no internal secret model, they are the model. And the model is of how different lexemes relate to each other in the source material the model was built from.

Some might choose to call that the world.

If you believe your internal model of the world is no different from a statistical model of the words you have seen, then by all means do that. But I believe a lot of humans see their view of the world differently.

I very much believe my cat’s model of the world has barely anything at all to do with language.

This path to AGI through LLM is nothing but religious dogma some Silicon Valley rich types believe.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#764

Post by llbbdd »

jdiff wrote: post #610
p1esk wrote: post #602

Earlier quoted context omitted.

I just gave chatgpt this prompt - it produced a picture of a glass filled to the brim with water.

Like most quirks that spread widely, a bandaid is swiftly applied. This is also why they now know how many r's are in "strawberry." But we don't get any closer to useful general intelligence by cobbling together thousands of hasty patches.

Seems to have worked fine for humans so far.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#765

Post by ben_w »

modeless wrote: post #648
phoenixreader wrote: post #604

A decade is nothing. If issues will be worked through a decade from now, that means the best time to think about opportunities/coonsequences related to that is now.

Yeah, I see people pooh-poohing the idea of humanoid robots being useful this decade, saying it will take at least 20 years. Oh yeah? Instead of 5 years to render all human labor obsolete, it will take 20? The magnitude of that change is so large that the implications of it happening anytime in our lifetimes are too big to ignore. The important thing is that this is not going to be perpetually 20 years in the future…

> Oh yeah? Instead of 5 years to render all human labor obsolete, it will take 20? The magnitude of that change is so large that the implications of it happening anytime in our lifetimes are too big to ignore.

While true, I would suggest two things:

First, that nobody actually knows how long it will take to make fully-general AI to drive robots, humanoid or otherwise. Look how long self-driving cars have taken, and that they're still geo-fenced.

Second, that it doesn't take AI for the robots themselves to have 90% of this impact. All those jokes about AI meaning "Actually Indians"? Well, the same robots controlled not by artificial intelligence but by remote control from cheap 3rd world labourers who charge $5/day, will make current arguments about the effect of immigration on unemployment look laughably naïve. Likewise, unfortunately, crime, because one thing we can guarantee is that someone's going to share their password or access token and some rich person's cheap robot servant will become Mr. Stabby the unknown assassin.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#766

Post by hackerdood »

tablatom wrote: post #700
Bengalilol wrote: post #674

Earlier quoted context omitted.

I am thinking the same. And we should start considering on what makes us humans and how we can valorize our common ground.

This. I believe it’s the most important question in the world right now. I’ve been thinking long and hard about this from an entirely practical perspective and have surprised myself that the answer seems to be our capacity to love. The idea is easily dismissed as romantic but when I say I’m being practical I really mean it. I’m writing about it here https://giftcommunity.substack.com/

You’ve probably already listened to it but in the event you haven’t: https://podcasts.apple.com/us/podcast/freakonomics-radio/id3...

He seems to share your sentiment

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#767

Post by ben_w »

chronci739 wrote: post #691
modeless wrote: post #648

Earlier quoted context omitted.

Yeah, I see people pooh-poohing the idea of humanoid robots being useful this decade, saying it will take at least 20 years. Oh yeah? Instead of 5 years to render all human labor obsolete, it will take 20? The magnitude of that change is so large that the implications of it happening anytime in our lifetimes are too big to ignore. The important thing is that this is not going to be perpetually 20 years in the future…

> This is something that will happen. Not in our lifetime. The iPhone came out less than 20 years ago. And what, you scan QR codes at restaurants with iphones?

The iPhone came out less than 20 years ago, and now I:

• Don't get out my debit card while shopping.

• Don't get lost exploring a new city.

• Have zero-cost video calls with anyone I want.

• Use most spare moments of my time — walking to the shops, or on public transport, or while hiking in the countryside — learning something new. When I'm not too damp for the capacitive touch screen, that can be interactive lessons, not just passive; but even for the passive consumption, mobile internet beats pre-loaded content on an MP3 player.

• Have a real-time augmented-reality translator, for the German I've not yet learned while living in Berlin, and all the other languages I don't (or barely) know while travelling outside the country.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#768

Post by mexicocitinluez »

zmmmmm wrote: post #757

It's good to see experts with similar scepticism about agents that I have. I don't doubt they will be useful in some settings, but they lean into all the current weak points of large language models and make them worse. Security, reproducibility, hallucinations, bias, etc etc. With all these issues already being hard to manage, I just don't believe businesses are going to delegate processes to autonomous agents in a…

> Literally anything that matters is going to get implemented in a crontrolled workflow that strips out all the autonomy with human checkpoint at every step.

Yea, there aren't a ton of problems (that I can see) in my current domain that could be solved by having unattended agents generating something.

I work in healthcare and there are a billion use cases right now, but none that don't require strict supervision. For instance, having an LLM processing history and physicals from potential referrals looking for patient problems/extracting historical information is cool, but it's nowhere near reliable enough to do anything but present that info back to the clinician to have them verify it.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#769

Post by zebrawaffles »

dist-epoch wrote: post #342
ActorNightly wrote: post #135

Not a decade. More like a century, and that is if society figures itself out enough to do some engineering on a planetary scale, and quantum computing is viable. Fundamentally, AGI requires 2 things. First it needs to be able to operate without information, learning as it goes. The core kernel should be such that it doesn't have any sort of training on real world concepts, only general language parsing that it can us…

AIs already fake-simulate the weather (chaotic system) using 1% of the resources used by the real-simulating supercomputers.

Source?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#770

Post by delusional »

spuz wrote: post #747
amelius wrote: post #743

Earlier quoted context omitted.

Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.

Interestingly this is a question I've had for a while. Night brings potentially deadly cold, predators, a drastic limit in vision so why do we find the sunset and night sky beautiful. Why do we stop and watch the sun set - something that happens every day - rather than prepare for the food and warmth we need to survive the night?

You're entering the domain of philosophy. There's a concept of "the sublime" that's been richly explored in literature. If you find the subject interesting, I'd recommend you starting with Immanuel Kant.
Post reply on HN