Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

801–810 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#801

Post by K0balt »

alganet wrote: post #649
K0balt wrote: post #457

We will never achieve AGI, because we keep moving the goalposts. SOTA models are already capable of outperforming any human on earth in a dizzying array of ways, especially when you consider scale. Humans also produce nonsensical, useless output. Lots of it. Yes, LLMs have many limitations that humans easily transcend. But few if any humans on earth can demonstrate the breadth and depth of competence that a SOTA mode…

> We will never achieve AGI, because we keep moving the goalposts. I think it's fair to do it to the idea of AGI. Moving the goalpost is often seen as a bad thing (like, shifting arguments around). However, in a more general sense, it's our special human sauce. We get better at stuff, then raise the bar. I don't see a reason why we should give LLMs a break if we can be more demanding of them. > SOTA models are alread…

I’m with you on the energy and limitations, and even on the moving of goalposts.

I’d like to add that I think limit definition of AGI has jumped the shark though and is already at ASI, since we expect our machine to exhibit professional level acumen across such a wide range of knowledge that it would be similar to the 0.01 percent top career scholars and engineers, or even above any known human capacity just due to breadth of knowledge. And we also expect it to provide that level of focused interaction to a small city of people all at the same time / provide that knowledge 10,000 times faster than any human can.

I think definitionally that is ASÍ.

But I also think AGI that “we are still chasing” focus-groups a lot better than ASI which is legitimately scary as shit to the average Joe, and which seasoned engineers recognize as a significant threat if controlled by people with misaligned intentions.

PR needs us to be “approaching AGI”, not “closing in on ASI”, or we would be pinned down with prohibitive regulatory straitjackets in no time.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#802

Post by aik »

kllrnohj wrote: post #575
aik wrote: post #540

Earlier quoted context omitted.

Hard disagree. You don’t need AGI to transform countless workflows within companies, current LLMs can do it. A lot of the current investments are to help with the demand with current generation LLMs (and use cases we know will keep opening up with incremental improvements). Are you aware of how intensely all the main companies that host leading models (azure, aws, etc) are throttling usage due to not enough data cent…

> Eg. At my company we have 100x more demand than we can get capacity for, and we’re barely getting started. We have a roadmap with 1000x+ the current demand and we’re a relatively small company. OpenAI's revenue is $13bn with 70% of that coming from people just spending $20/mo to talk to ChatGPT. Anthropic is projecting $9bn in revenue in 2025. For nice cold splash of reality, fucking Arizona Iced Tea has $3bn in re…

I have no idea if OpenAI’s valuation is reasonable. All I’m saying is I’m convinced the demand is there, even without AGI around the corner. You do not need AGI to transform countless industries.

And we are profitable on our AI efforts while adding massive value to our clients.

I know less about OpenAI’s economics, I know there are questions on whether their model is sustainable/for how long. I am guessing they are thinking about it and have a plan?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#803

Post by aik »

Culonavirus wrote: post #554
aik wrote: post #540

Earlier quoted context omitted.

Hard disagree. You don’t need AGI to transform countless workflows within companies, current LLMs can do it. A lot of the current investments are to help with the demand with current generation LLMs (and use cases we know will keep opening up with incremental improvements). Are you aware of how intensely all the main companies that host leading models (azure, aws, etc) are throttling usage due to not enough data cent…

Oh look, people with skin in the AI game insist AI is not a massive bubble. More news at 11.

We’re a regular old SaaS company that has figured out how to add massive value using AI. I am making no statements about valuations and bubbles. I’m actually guessing there is some bubble / overhype. That doesn’t mean it isn’t still incredibly valuable.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#804

Post by getnormality »

One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today.

No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward.

Also, speculation as to why AI coders are "mortally terrified of exceptions": it's the same thing OpenAI recently wrote about, trying to get an answer at all costs to boost some accuracy metric. An exception is a signal of uncertainty indicating that you need to learn more about your problem. But that doesn't get you points. Only a "correct answer" gets you points.

Frontier AI research seems to have yet to operationalize a concept of progress without a final correct answer or victory condition. That's why AI is still so bad at Pokemon. To complete open-ended long-running tasks like Pokemon, you need to be motivated to get interesting things to happen, have some minimal sense of what kind of thing is interesting, and have the ability to adjust your sense of what is interesting as you learn more.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#805

Post by ben_w »

shdh wrote: post #433
themafia wrote: post #430

5 decades. You have one decade to clean up your power use problem. If you don't you will find yourself in the next AI winter.

Power use is less important than model capability AGI is either more scale or differing systems, or both They can always optimize for power consumption after AGI has been reached

Disagree. AI that displaces workers is worth spending anything up to that worker's salary on, and this can have a devastating impact on energy prices for everyone.

Worked example, but this is a massive oversimplification in several different ways all at once:

Global electricity supply was around 31,153 TWh in 2024. The world's economy is about $117e12/year. Any AI* that is economically useful enough to handle 33% that, $38.6e12/year, is economically worthwhile to spend anything up to $38.6e12/year to keep that AI running.

If you spend $38.6e12 (per year) to buy all of those 31,153 TWh of electricity (per year), the global average electricity market price is now $1.239/kWh, and a lot of people start to wonder what the point of automating everything was if nobody can afford to keep their heating/AC (delete as appropriate) switched on. Or even the fridge/freezer, for a lot of people.

* I don't care what definition you're using for AGI, this is just about "economically useful"

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#806

Post by steveBK123 »

getnormality wrote: post #804

One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today. No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward. Also, speculation…

It's nice seeing commentary from someone who is both knowledgable in AI and NOT trying to pump the AI bag.

Right now the median actor in the space loudly proclaims AGI is right around the corner, while rolling out pornbots/ads/in-chat-shopping, which generally seems at odds with a real belief that AGI is close (TAM of AGI must be exponentially larger than the former).

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#807

Post by subjectivationx »

ajross wrote: post #293
SR2Z wrote: post #246

Earlier quoted context omitted.

Right, but modeling the structure of language is a question of modeling word order and binding affinities. It's the Chinese Room thought experiment - can you get away with a form of "understanding" which is fundamentally incomplete but still produces reasonable outputs? Language in itself attempts to model the world and the processes by which it changes. Knowing which parts-of-speech about sunrises appear together an…

> Knowing which parts-of-speech about sunrises appear together and where is not the same as understanding a sunrise What does "understanding a sunrise" mean though? Arguments like this end up resting on semantics or tautology, 100% of the time. Arguments of the form "what AI is really doing" likewise fail because we don't know what real brains are "really" doing either . I mean, if we knew how to model human language…

Everyone reading this understands the meaning of a sunrise. It is a wonderful example of the use theory of meaning.

If you raised a baby inside a windowless solitary confinement cell for 20 years and then one day show them the sunrise on a video monitor, they still don't understand the meaning of a sunrise.

Trying to extract the meaning of a sunrise by a machine from the syntax of a sunrise data corpus is just totally absurd.

You could extract some statistical regularity from the pixel data of the sunrise video monitor or sunrise data corpus. That model may provide some useful results that can then be used in the lived world.

Pretending the model understands a sunrise though is just nonsense.

Showing the sunrise statistical model has some use in the lived world as proof the model understands a sunrise I would say borders on intellectual fraud considering a human doing the same thing wouldn't understand a sunrise either.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#808

Post by hyperadvanced »

vharish wrote: post #742
hyperadvanced wrote: post #579

Earlier quoted context omitted.

This is correct, it should burn the retinas of anyone thinking that OAI or Anthropic are in any way worth their multi-billion dollar valuations. I liked AK’s analysis of AI for coding here (it’s overly defensive, lacks style and functionality awareness, is a cargo cultist, and/or just does it wrong a lot) but autocomplete itself is super valuable, as is the ability to generate simple frontend code and let you solve t…

There are many more use cases that aren't fully realised yet. With regards to coding, LLMs have shortcomings. However, there's a lot of work that can be automated. Any work that requires interaction with a computer can eventually be automated to some extent. To what extent is something only time can tell.

Sure, but you don’t need AI to automate computer work. You can make a career out of formalizing the kinds of excel-jockeying that people do for reports or data entry

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#809

Post by alganet »

K0balt wrote: post #801
alganet wrote: post #649

Earlier quoted context omitted.

> We will never achieve AGI, because we keep moving the goalposts. I think it's fair to do it to the idea of AGI. Moving the goalpost is often seen as a bad thing (like, shifting arguments around). However, in a more general sense, it's our special human sauce. We get better at stuff, then raise the bar. I don't see a reason why we should give LLMs a break if we can be more demanding of them. > SOTA models are alread…

I’m with you on the energy and limitations, and even on the moving of goalposts. I’d like to add that I think limit definition of AGI has jumped the shark though and is already at ASI, since we expect our machine to exhibit professional level acumen across such a wide range of knowledge that it would be similar to the 0.01 percent top career scholars and engineers, or even above any known human capacity just due to b…

As much regulatory measures as possible seems good. This things are not toys.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#810

Post by ben_w »

themafia wrote: post #465
shdh wrote: post #433

Earlier quoted context omitted.

Power use is less important than model capability AGI is either more scale or differing systems, or both They can always optimize for power consumption after AGI has been reached

> AGI is either more scale So you plan to scale without increasing power usage. How's that? > They can always optimize for power consumption after AGI has been reached If you don't optimize power consumption you're going to increase surface area required to build it. There are hard physical limits having to do with signal propagation times. You're ignoring the engineering entirely. The software is not hardly interest…

> If you don't optimize power consumption you're going to increase surface area required to build it. There are hard physical limits having to do with signal propagation times.

While true, that probably stopped being an important constraint around the time we switched from thermionic valves to transistors as the fundamental unit of computation.

To be deliberately extreme: if we built cubic-kilometre scale compute hardware where each such structure only modelled a single cortical column from a human's brain, and then spread multiple of these out evenly around the full volume within Earth's geosynchronous orbital altitude until we had enough to represent a full human brain, that would still be on par with human synapses.

Synapses just aren't very fast.

Post reply on HN