Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

681–690 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#681

Post by Hendrikto »

tyre wrote: post #226
ekjhgkejhgk wrote: post #134

Earlier quoted context omitted.

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

There is some evidence from Anthropic that LLMs do model the world. This paper[0] tracing their "thought" is fascinating. Basically an LLM translating across languages will "light up" (to use a rough fMRI equivalent) for the same concepts (e.g. bigness) across languages. It does have clusters of parameters that correlate with concepts, not just randomly "after X word tends to have Y word." Otherwise you would expect…

That is just how embeddings work. It does not confirm nor deny whether LLMs have a world model.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#682

Post by awesome_dude »

scrubs wrote: post #598
awesome_dude wrote: post #527

Earlier quoted context omitted.

I know the attributes of an Apple, i know the attributes of a Pear. As does a computer. But only i can bite into one and know without any doubt what it is and how it feels emotionally.

You have half a point. "Without any doubt" is merely the apex of a huge undefined iceberg. I write half .. eating is multi modal and consequential. The llm can read the menu, but it didn't eat the meal. Even humans are bounded. Feeling, licking, smelling, or eating the menu still is not eating the meal. There is an insuperable gap in the analogy ... a gap in the concept and of sensory data doing it. Back to first poi…

I think more, also, how i feel about the taste.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#683

Post by Animats »

ekjhgkejhgk wrote: post #134
Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

The interview which I've watched recently with Rich Sutton left me with the impression that AGI is not just a matter of adding more 9s. The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to udnerstand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here…

> The interviewer had an idea that he took for granted: that to understand language you have to have a model of the world. LLMs seem to understand language therefore they've trained a model of the world. Sutton rejected the premise immediately. He might be right in being skeptical here.

That's the basic success of LLMs. They don't have much of a model of the world, and they still work. "Attention is all you need". Good Old Fashioned AI was all about developing models, yet that was a dead end.

There's been some progress on representation in an unexpected area. Try Perchance's AI character chat. It seems to be an ordinary chatbot. But at any point in the conversation, you can ask it to generate a picture, which it does using a Stable Diffusion type system. You can generate several pictures, and pick the one you like best. Then let the LLM continue the conversation continue from there.

It works from a character sheet, which it will create if asked. It's possible to start from an image and get to a character sheet and a story. The back and forth between the visual and textural domains seems to help.

For storytelling, such system may need to generate the collateral materials needed for a stage or screen production - storyboards, scripts with stage directions, character summaries, artwork of sets, blocking (where everybody is positioned on stage), character sheets (poses and costumes) etc. Those are the modeling tools real productions use to keep a work created by many people on track. Those are a form of world model for storytelling.

I've been amazed at how good the results I can get from this thing are. You have to coax it a bit. It tends to stay stuck in a scene unless you push the plot forward. But give it a hint of what happens next and it will run with it.

[1]https://perchance.org/ai-character-chat

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#684

Post by ionwake »

an0malous wrote: post #189
ionwake wrote: post #57

Earlier quoted context omitted.

I mean you say this, but I havent touched a line of code as a programmer in months, having been totally replaced by AI. I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.

Let’s see the code

pls no

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#685

Post by ionwake »

rootusrootus wrote: post #308
ionwake wrote: post #57

Earlier quoted context omitted.

I mean you say this, but I havent touched a line of code as a programmer in months, having been totally replaced by AI. I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.

I don't want to sound mean, but c'mon, the reality is that if you haven't touched a line of code in months, you are/were not a programmer. I love Claude Code, it really has its moments. But even for the stuff it is exceptionally good at, I have to regularly fix mistakes it has made. And I only give it the fairly easy stuff I don't feel like doing myself.

It is ok brother I can handle the accusation. Perhaps after 2 decades... I really am no longer a programmer?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#686

Post by ionwake »

andy_ppp wrote: post #111
ionwake wrote: post #57

Earlier quoted context omitted.

I mean you say this, but I havent touched a line of code as a programmer in months, having been totally replaced by AI. I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.

I think AI is great and extremely helpful but if you’ve been replaced already maybe you have more time now to make better code and decisions? If you think the AI output is good by default I think maybe that’s a problem. I think general intelligence is something other than what we have now, these systems are extremely bad at updating their knowledge and hopelessly at applying understanding from one area to another. Fo…

Yeah i agree. However like ... I don't understand why you think I am making bad decisions? I'm a self made ( to an extent ) millionaire, I am doing ok

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#687

Post by awesome_dude »

throwaway-0001 wrote: post #626
awesome_dude wrote: post #437

Earlier quoted context omitted.

That's not the creativity aspect, my comment is an observation, which, by definition, is a regurgitation of events. Edit: This also demonstrates that people think (erroneously) that AI pumping out code, or content, or even essays, is inventive, but it's not. This is merely a description and reduction, both of which AI can do, but neither of which are an invention.

Actually I think the line between creative and regurgitate is so blurred you can’t tell me a single creative thing you did. So if 99% of people are not creative, and just regurgitate then why we keep AI standards so high? Can you show me one single thing you did in your life that was truly creative and not regurgitated?

I think that was my point, I generally regurgitate. A person can do that a lot in life.

That's why people are conflating LLMs for AGI.

For now, I think that the key difference between me, and an LLM is that an LLM still needs a prompt.

It's not surveying the world around it determining what it needs to do.

I do a lot of something that I think an LLM cannot get do, look at things and try to find what attributes they have and how I can harness those to solve problems. Most of the attributes are unknown by the human race when I start.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#688

Post by nurettin »

Continual learning would mean that the data somehow has to be part of the model and the model needs to incrementally adapt to novel inputs. Not just tacked on and backpropagated, but within the network affecting decisions. Current architectures are pretty much dead ends in that aspect.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#689

Post by adrianN »

jacquesm wrote: post #667
stouset wrote: post #652

Earlier quoted context omitted.

To play devil’s advocate, you have never seen the night sky. Photoreceptors in your eye have been excited in the presence of photons. Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons. Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly…

That sounds very profound but it isn't: it the sum of your states interaction that is your consciousness, there is no 'consciousness' unit in your brain, you can't point at it, just like you can't really point at the running state of a computer. At that level it's just electrons that temporarily find themselves in one spot or another. Those cells aren't living organisms, they are components of a multi-cellular organi…

At least some of your cells are fine living without the others as long as they’re provided with an environment with the right kind of nutrients.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#690

Post by Hendrikto »

danielmarkbruce wrote: post #577
DanHulton wrote: post #570

Earlier quoted context omitted.

The thing about this, though - cars have been built before. We understand what's necessary to get those 9s. I'm sure there were some new problems that had to be solved along the way, but fundamentally, "build good car" is known to be achievable, so the process of "adding 9s" there makes sense. But this method of AI is still pretty new, and we don't know it's upper limits. It may be that there are no more 9s to add, o…

While you are right about the broader (and sort of ill defined) chase toward 'AGI' - another way to look at it is the self driving car - they got there eventually.And, if you work on applications using LLMs you can pretty easily see that Karpathy's sentiment is likely correct. You see it because you do it. Even simple applications are shaped like this, albeit each 9 takes less time than self driving cars for a simple…

> another way to look at it is the self driving car - they got there eventually.

No they did not. Elon has been saying Tesla will get there “next year” since 2015. He is still saying that, and despite changing definitions, we still are not there.

Post reply on HN