Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

111–120 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#111
post #57

Earlier quoted context omitted.

No, it’s because of money and the hype cycle.

I mean you say this, but I havent touched a line of code as a programmer in months, having been totally replaced by AI. I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.

I think AI is great and extremely helpful but if you’ve been replaced already maybe you have more time now to make better code and decisions? If you think the AI output is good by default I think maybe that’s a problem. I think general intelligence is something other than what we have now, these systems are extremely bad at updating their knowledge and hopelessly at applying understanding from one area to another. For example self driving cars are still extremely brittle to the point of every city needing new and specific training - you can just take a car with controls on the opposite side to you and safely drive in another country.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#112

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra The debate about AGI is interesting from a philosophical perspective, but from a practical perspective AI doesn't need to get anywhere close to AGI to turn the world upside down.

I don’t even know what AGI is, and neither does anyone else as far as I can tell. In the parts of the video I watched, he cites several things missing which all have to do with autonomy: continual automated updates of internal state, fully autonomous agentic behavior, etc. I feel like GPT 3 was AGI, personally. It crossed some threshold that was both real and magical, and future improvements are relying on that basic…

AGI is when a computer can accomplish every cognitive task a typical human can. Given tools to speak, hear, and manipulate a computer, an AGI could be dropped in as a remote employee and be successful.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#113
post #77

I think it's a shame that a 146 minute podcast released ~55 minutes ago has so much discussion. Everybody here is clearly just reacting to the title with their own biases. I know it's against the guidelines to discuss the state of a thread, but I really wish we could have thoughtful conversations about the content of links instead of title reactions.

gotta listen at 2x speed!

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#114
I don't understand how anyone can believe that we're near even a whiff of AGI when we barely understand what dreaming is, or how the human brain interacts with the quantum world. There are so many elements of human creativity that are still utterly hidden behind a wall that it makes me feel insane when an entire industry is convinced we're just magically going to have the answer soon.

The people heralding the emergence of AGI are doing little more than pushing Ponzi schemes along while simultaneously fueling vitriolic waves of hate and neo-luddism for a ground-breaking technology boom that could enhance everything about how we live our lives... if it doesn't get regulated into the ground due to the fear they're recklessly cooking up.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#115

Are "agents" just programs that call into an LLM and based on the response, it will do something?

Kinda. It's just an LLM that performs function calling (i.e. the LLM "decides" when a function needs to be called for a task and passes the appropriate function name and arguments for that function based on its context). So yea an "agent" is that LLM doing all of that and then your program that actually executes the function accordingly.

That's an "agent" at its simplest -- a LLM able to derive from natural language when it is contextually appropriate to call out to external "tools" (i.e. functions).

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#116

Earlier quoted context omitted.

Machine learning as a descriptive phrase has stopped being relevant. It implies the discovery of information in a training set. The pre-training of an LLM is most definitely machine learning. But what people are excited and interested in is the use of this learned data in generative AI. “Machine learning” doesn’t capture that aspect.

It's a valid term that is worth introducing to the layperson IMO. Let them know how the magic works, and how it doesn't.

How does "it's called machine learning not AI" help anyone know how it works? It's just a fancier sounding name.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#117

Earlier quoted context omitted.

I don’t even know what AGI is, and neither does anyone else as far as I can tell. In the parts of the video I watched, he cites several things missing which all have to do with autonomy: continual automated updates of internal state, fully autonomous agentic behavior, etc. I feel like GPT 3 was AGI, personally. It crossed some threshold that was both real and magical, and future improvements are relying on that basic…

Most humans do not even have a general intelligence! Many students are practically illiterate, and can not even read and understand book or manual! We are approaching situation, where AI will make most decisions, and people will wear it as a skin suit, to fake competency!

I wouldn’t say that any specific skill (like literacy) is required to have intelligence. It’s more the capability to learn skills and build a model of the world and the people in it using abstract reasoning.

Otherwise we would have to say that pre-literacy societies lacked intelligence, which would be silly since they are the ones that invented writing in the first place!

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#118
Kurzweil has been eerily right so far, and his timeline has AGI at 2029. When software can perform any unattended, self directed task (in principle) at least as well as any human over the sum total of all tasks that humans are capable of doing, we will have reached AGI.

Software can already write more text on any given subject better than a majority of humanity. It can arguably drive better across more contexts than all of humanity - any human driver over a billion miles of normal traffic will have more accidents than self driving AI over the same distance. Short stories, haikus, simple images, utility scripts, simple software, web design, music generation - all of these tasks are already superhuman.

Longer time horizons, realtime and continuous memory, a suite of metacognitive tasks, planning, synthesis of large bodies of disparate facts into novel theory, and a few other categories of tasks are currently out of reach, but some are nearly solved, and the list of things that humans can do better than AI gets shorter by the day. We're a few breakthroughs away, maybe even one big architectural leap, from having software that is capable (in principle) of doing anything humans can do.

I think AGI is going to be here faster than Kurzweil predicted, because he probably didn't take into consideration the enormous amount of money being spent on these efforts.

There has never been anything like this in history - in the last decade, over 5 trillion dollars has been spent on AI research and on technologies that support AI, like crypto mining datacenters that pivoted to AI, new power, water, data support, providing the infrastructure and foundation for the concerted efforts in research and development. There are tens of thousands of AI researchers, some of them working in private finance, some for academia, some doing military resarch, some doing open source, and a ton doing private sector research, of which an astonishing amount is getting published and shared.

In contrast, the entire world spent around 16 trillion dollars on world war II - all of the R&D and emergency projects and military logistics, humanitarian aid, and so on.

We have AI getting more resources and attention and humans involved in a singular development effort, pushing toward a radical transformation of the very concept of "labor" - while I think it might be a good thing if it is a decade away, even perpetually so until we have some reasonable plan for coping with it, I very much think we're going to see AGI within the very near future.

*When I say "in principle" I mean that given the appropriate form factor, access, or controls, the AI can do all the thinking, planning, and execution that a human could do, at least as well as any human. We will have places that we don't want robots or AI going, tasks reserved for humans, traditions, taboos, economics, and norms that dictate AI capabilities in practice, but there will be no legitimacy to the idea that an AI couldn't do a thing.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#119
post #80

Earlier quoted context omitted.

I don’t even know what AGI is, and neither does anyone else as far as I can tell. In the parts of the video I watched, he cites several things missing which all have to do with autonomy: continual automated updates of internal state, fully autonomous agentic behavior, etc. I feel like GPT 3 was AGI, personally. It crossed some threshold that was both real and magical, and future improvements are relying on that basic…

It crossed some threshold that was both real and magical Only compared to our experience at the time. and future improvements are relying on that basic set of features at their core Language models are inherently limited, and it's possible - likely, IMO - that the next set of qualitative leaps in machine intelligence will come from a different set of ideas entirely.

Learning != Training.

Thats not a period, it's a full stop. There is no debate to be had here.

IF an LLM makes some sort of breakthrough (and massive data collation allows for that to happen) it needs to be "re trained" to absorb its own new invention.

But we also have a large problem in our industry, where hardware evolved to make software more efficient. Not only is that not happening any more but we're making our software more complex and to some degree less efficient with every generation.

This is particularly problematic in the LLM space: every generation of "ML" on the llm side seems to be getting less efficient with compute. (Note: this isnt quite the case in all areas of ML, yolo models working on embedded compute is kind of amazing).

Compactness, efficiency and reproducibility are directions the industry needs to evolve in, if it ever hopes to be sustainable.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#120
post #77

I think it's a shame that a 146 minute podcast released ~55 minutes ago has so much discussion. Everybody here is clearly just reacting to the title with their own biases. I know it's against the guidelines to discuss the state of a thread, but I really wish we could have thoughtful conversations about the content of links instead of title reactions.

a very human reaction ;)
Post reply on HN