And I'm looking for a problem to spend my next decade on ...
Andrej Karpathy – It will take a decade to work through the issues with agents
751–760 of 1001 posts
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#752Earlier quoted context omitted.
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.
Compare with news stories from last decade, about people in Pakistan developing a deep fear of clear skies over several years of US drone strikes in the area. They became trained to associate good weather with not beauty, but impending death.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#753Earlier quoted context omitted.
Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.
Interestingly this is a question I've had for a while. Night brings potentially deadly cold, predators, a drastic limit in vision so why do we find the sunset and night sky beautiful. Why do we stop and watch the sun set - something that happens every day - rather than prepare for the food and warmth we need to survive the night?
Maybe it's that we only pause to observe them and realize they're beautiful, when we're feeling safe enough?
"Beautiful sunset" evokes being on a calm sea shore with a loved one, feeling safe. It does not evoke being on a farm and looking up while doing chores and wishing they'd be over already. It does not evoke being stranded on an island, half-starved to death.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#754Earlier quoted context omitted.
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
Beauty standard changes over time, see how people perceive body fat in the past few hundred years. We learns what is beautiful from our peers. Taste can be acquired and can be cultural. See how people used to had their coffee. Comparing human to LLM is like comparing something constantly changing to something random -- we can't compare them directly, we need a good model for each of them before comparing.
Has there been a point in human history where mainstream society denied the beauty in nature?
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#755Earlier quoted context omitted.
I mean, I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. So I’d agree that the night sky is “beautiful”, but not because I somehow have access to a deeper meaning of the word or the sky than an LLM does. As someone who (long ago) studied philosophy of mind and (Chomskian) li…
> I think the reason I would say the night sky is “beautiful” is because the meaning of the word for me is constructed from the experiences I’ve had in which I’ve heard other people use the word. Ok but you don’t look at every night sky or every sunset and say “wow that’s beautiful” There’s a quality to it - not because you heard someone say it but because you experience it
> Ok but you don’t look at every night sky or every sunset and say “wow that’s beautiful
Exactly - because it's a semantic shorthand. Sunsets are fucking boring, ugly, transient phenomena. Watching a sunset while feeling safe and relaxed, maybe in a company of your love interest who's just as high on endorphins as you are right now - this is what feels beautiful. This is a sunset that's beautiful. But the sunset is just a pointer to the experience, something others can relate to, not actually the source of it.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#756>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…
FWIW, Karpathy literally says, multiple times, that he thinks we never left the exponential - that all human progress over last 4+ centuries averages out to that smooth ~2% growth rate exponential curve, that electricity and computing and AI are just ways we keep it going, and we'll continue on that curve for the time being.
It's the major point of contention between him and the host (who thinks growth rate will increase).
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#757It's good to see experts with similar scepticism about agents that I have. I don't doubt they will be useful in some settings, but they lean into all the current weak points of large language models and make them worse. Security, reproducibility, hallucinations, bias, etc etc.
With all these issues already being hard to manage, I just don't believe businesses are going to delegate processes to autonomous agents in a widespread manner. Literally anything that matters is going to get implemented in a crontrolled workflow that strips out all the autonomy with human checkpoint at every step. They may call them agents just to sound cool but it will be completely controlled.
Software people are all fooled by what is really a special case around software development : outcomes are highly verifiable and mistakes (in development) are almost free. This is just not the case out there in the real world.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#758Earlier quoted context omitted.
I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.
Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.
Being struck by lighting may affect your desire to go outside, but it has zero correlation with the sky’s beauty.
Outer space is beautiful, poison dart frogs are beautiful, lava is beautiful. All of them can kill or maim you if you don’t wear protection, but that doesn’t take away from their beauty.
Conversely, boring safe things aren’t automatically beautiful. I see no reasonable reason to believe that finding beauty in the night sky is any sort of “training”.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#759People keep talking about AGI as if it's some mystical leap beyond human capability. But let's be honest; software development at a modern startup is already the upper bound of applied intelligence. You're juggling shifting product specs, ambiguous user feedback, legacy code written by interns, and five competing JS frameworks, all while shipping on a Friday. Models can now do that. They can reason about asynchronous…
Lol, software development at a modern startup isn't even in the upper half of applied intelligence in software engineering much less global human activity/achievement. The "problems" most startups are solving are simple to the point of banality.
Re: Andrej Karpathy – It will take a decade to work through the issues with agents
#760Earlier quoted context omitted.
Humans evolved to think the night sky is beautiful. That's also training. If humans were zapped by lightning every time they went outside at night, they would not think that a night sky is beautiful.
Compare with news stories from last decade, about people in Pakistan developing a deep fear of clear skies over several years of US drone strikes in the area. They became trained to associate good weather with not beauty, but impending death.
Fear and a sense of beauty aren’t mutually exclusive. It is perfectly congruent to fear a snake, or bear, or tiger in your presence, yet you can still find them beautiful.