Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

881–890 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#881

Earlier quoted context omitted.

I don't think we can ever know that we are generally intelligent. We can be unsure, or we can meet something else which possesses a type of intelligence that we don't, and then we'll know that our intelligence is specific and not general. So to make predictions about general intelligence is just crazy. And yeah yeah I know that OpenAI defines it as the ability to do all economically relevant tasks, but that's an awfu…

All intelligence is specific, as evidenced by the fact that a universal definition regarding the specifics of "common sense" doesn't exist.

Common is not the same as general. A general key would open every lock. Common keys... well they're quite familiar.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#882

Earlier quoted context omitted.

> if a machine performs a task as well as a human, it understands it exactly as much as a human. I think you're right, except that the ones judging "as well as a human" are in fact humans, and humans have expectations that expand beyond the specs. From the narrow perspective of engineering specifications or profit generated, a robot/AI may very well be exactly as understanding as a human. For the people which interac…

This is all nonsense. It is like saying the airplane understands how to fly. "You disagree? Well lets see you fly! You are saying the airplane doesn't understand how to fly and you can't even fly yourself?" This would be confusing the fact humans built the flying machine and the flying machine doesn't understand anything.

Right. A flying machine doesn’t need to understand anything to fly. It’s not even clear what it would mean for it to do so, or how it would fly any differently if it did.

Same with the AI machines.

Understanding is not something that any machine or person does. Understanding is a compact label applied to people’s behavior by an observer that allows the observer to predict future behavior. It’s not a process in itself.

And yes, we apply this label to ourselves. Much of what we do is only available to consciousness post-hoc, and is available to be described just the same as the behavior of someone else.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#883
post #609

With all due respect, what does it say about us that „famous researcher voices his speculative opinion“ is an instant top 1 on hackernews?

Is it worse than "Rich CEO expressing certainly over his hunch"

fair point, I appreciate AI researcher intuition about AI much more than CEO hunch boldly stated but not even firm on the terminology.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#884

Earlier quoted context omitted.

When I think of the term "agency" I think of a feedback loop whereby an actor is aware of their effect and adjusts behavior to achieve desired effects. To be a useful agent, one must operate in a closed feedback loop; an open loop does not yield results. Consider the distinction between probabilistic and deterministic reasoning. When you are dealing with a probabilistic method (eg, LLMs, most of the human experience)…

I see your perspective about the inevitability of death causing a forcing-function directedness for agents, but that's a much much weaker claim than (emphasis mine): > In the natural world, agency is a consequence of death: by dying, the feedback loop closes in a powerful way. My original question was why could agency not exist without death, not why it was hampered without it. For clarity, I'm coming at from an anal…

I love the discussion — thank you.

Your comment makes me more bullish on death. Death isn’t arbitrary as you claim: it is a direct expression of an entity in its environment, it epitomizes contextualization. (I argue that honoring context is the opposite of arbitrariness.)

Further, death encapsulates multiple layers of abstraction. When an entity dies, it dies on every level (eg both instincts and socially learned heuristics). The death reaches deep down inside the hierarchy of its own form to eliminate possibilities. That is some seriously strong directionality; it’s not like “taking your second left” or some other mono-dimensional vector. Layers and layers of genes and learning are discarded. It is truly an incredibly powerful feedback-loop closure.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#885

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic co…

> In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic counterfactual scenarios.

Depends what you mean by "basic". Have you seen Simple Bench? https://simple-bench.com/

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#886

Earlier quoted context omitted.

Infinitely big little numbers Academia has rediscovered itself Signal attenuation, a byproduct of entropy, due to generational churn means there's little guarantee. Occam's Razor; Karpathy knows the future or he is self selecting biology trying to avoid manual labor? His statements have more in common with Nostradamus. It's the toxic positivity form of "the end is nigh". It's "Heaven exists you just have to do this w…

>> Heaven exists you just have to do this work to get there. Or perhaps Karpathy has a higher level understanding and can see a bigger picture? You've said something about heaven. Are you able to understand this statement, for example: "Heaven is a memeplex, it exists." ?

"Higher" than an EE with an MSc in elastic structures, ~30 years industry experience, now working with PhDs across the spectrum on energy models to embed in chips? Energy models in part, inferred from categorization of LLM contents and compression of those contents into geometric functions like I described?

"Higher level" implies acceptance of geometric structure. You place tokens like a Chomsky diagrams at each step up and down, where you should see parameters to transform geometry of the structure.

My team works "above" the contrived state management of software workers to more efficiently sync memory matrix to display matrix. LLMs are a form of compression [1]. My team is working on compressing them further into sets of points that make up each glyph and functions to recreate them.

Electromagnetic geometry transforms hardcoded[2] into hardware so reduce energy use of all the outdated string mangling of software dev as most know it.

What's higher level, relative to our machines, than design and implementation of the machine?

DnD dungeon master versus WOTC game designer.

Notice outside how there are no words and philosophy? Just color gradient and geometry?

Notice inside the human body no philosophy or words?

Language is not intelligence it's an emergent phenomena of geometry created by fundamental forces of physics organizing matter at various speeds relative to light.

You've read too much into an ultimately arbitrary statement meant to invoked a subtext, a subtle emotion context. You think of language as Legos, when it is music to feel.

[1] https://arxiv.org/abs/2309.10668 [2] https://iopscience.iop.org/article/10.1088/1742-6596/2987/1/...

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#887
post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

something that replaces humans doesn’t need to be 99.9999% reliable, it just has to be better than the humans it replaces.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#888

One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today. No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward. Also, speculation…

>An exception is a signal of uncertainty indicating that you need to learn more about your problem No, that would be a warning. Ab exceprion is a signal something failed and it was impossible to continue

Many exceptions are recoverable. This sometimes depends on the context, and on how well polished the software is

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#889

Earlier quoted context omitted.

This. I believe it’s the most important question in the world right now. I’ve been thinking long and hard about this from an entirely practical perspective and have surprised myself that the answer seems to be our capacity to love. The idea is easily dismissed as romantic but when I say I’m being practical I really mean it. I’m writing about it here https://giftcommunity.substack.com/

You’ve probably already listened to it but in the event you haven’t: https://podcasts.apple.com/us/podcast/freakonomics-radio/id3 ... He seems to share your sentiment

The link is broken - could you repost please?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#890
I think he is bearish about agentic workflows because he works at the very highest level of coding. An agentic Karpathy is a few doubling cycles beyond an agentic junior engineer. Agents (or just LLMs on a loop that correct their errors) are very reliable for less complex tasks now, and theyre still getting better at an exponential rate.

We are still on trend by projections to reach human parity in many domains by 2027-2028, the only thing that would prevent this is a major unexpected slowdown in AI progress.

Post reply on HN