Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrej Karpathy – It will take a decade to work through the issues with agents

dwarkesh.com

Post reply on HN
Print view View original

891–900 of 1001 posts

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#891

Post by exasperaited »

rhetocj23 wrote: post #833
spjt wrote: post #553

The thing about AGI is that if it's even possible, it's not coming before the money runs out of the current AI hype cycle. At least we'll all be able to pick up a rack of secondhand H100's for a tenner and a pack of smokes to run uncensored diffusion models on in a couple years. The real devastation will be in the porn industry.

"The real devastation will be in the porn industry." The UK govt has started to crack down on this. AI generated porn will lead to a war from govts on nailing this economic activity shut.

The UK government is not cracking down on AI porn generally but has started to crack down on the distribution of certain things, like:

- AI generated CSAM (out of a concern that it might cause people to seek to produce actual CSAM)

- AI generated rape and abuse images of real adults, again out of concern it will cause violence and its distribution is actually degrading and is experienced as and combined with threatening behaviour

- some extreme AI generated rape/abuse images of non-real people.

Despite what internet libertarians say, there is evidence to suggest that porn is changing people's sexual behaviours, particularly young people, both for good and ill.

At the moment there is no good reason to believe that AI-generated alternatives to harmful content are meaningfully less harmful to society.

There's more than enough evidence in articles posted on HN alone that people are beginning to experience psychosis brought on by spending too much time with AI content.

I don't really care if governments ban it; I'd like to see governments being much braver about criminalising AI generated misrepresentation, AI generated hoax content etc.

Sane governments should IMO absolutely ignore the ultra-libertarian angles; there is at least no reason that AI-generated content should be treated any differently under existing obscenity laws just because there are no real people in it.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#892

Post by LarsDu88 »

danans wrote: post #870
ninetyninenine wrote: post #861

Earlier quoted context omitted.

Except Sutton has no idea or even a clue about the internal model of a squirrel. He just uses it as a symbol for utterly stupid but still smarter than an LLM. It’s semantic manipulation in attempt to prove his point but he proves nothing. We have no idea how much of the world a squirrel understands. We understand LLMs more than squirrels. Arguably we don’t know if LLMs are more intelligent than squirrels. > Finally h…

> We have no idea how much of the world I squirrel understands. We understand LLMs more than squirrels Based on our understanding of biology and evolution we know that a squirrel brain works more similarly to the way we humans do vs an LLM. To the extent we understand LLMs, it's because they are strictly less complex than both ours and squirrels' brains, not because they are better model for our intelligence. They ar…

I don't think a modern LLM is necessarily less complicated than a squirrel brain. If anything it's more engineered (well structured and dissectable), but loaded with tons of erroneous circuitry that is completely irrelevant for intelligence.

The squirrel brain is an analogue mostly hardcoded circuit. It can take about one synapse to represent each "weight". A synapse is just a bit of fat membrane with some ion channels stuck on the surface.

A flip flop to represent a bit takes about 6 transistors, but in a typical modern GPU is going to need way more transitors to wire that bit - at least 20-30. multiply that by the minimum amount of bits to represent a single NN weight and you're looking at about 200-300 transitors just to represent one NN param for computing

And that's for actual compute. The actual weights in a GPU are stored most of the time in DRAM which needs to be constantly shuttled back and forth between the GPU's SRAM and HBM DRAM.

300 transistors with memory shuttling overhead versus a bit of fat membrane, and it's obvious general purpose GPU compute has a huge energy and compute overhead.

In the future, all 300 could conceivably replaced with a single crossbar latch in the form of a memristor.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#893

Post by asadotzler »

ben_w wrote: post #767
chronci739 wrote: post #691

Earlier quoted context omitted.

> This is something that will happen. Not in our lifetime. The iPhone came out less than 20 years ago. And what, you scan QR codes at restaurants with iphones?

The iPhone came out less than 20 years ago, and now I: • Don't get out my debit card while shopping. • Don't get lost exploring a new city. • Have zero-cost video calls with anyone I want. • Use most spare moments of my time — walking to the shops, or on public transport, or while hiking in the countryside — learning something new. When I'm not too damp for the capacitive touch screen, that can be interactive lessons…

• Don't get out my debit card while shopping.

You take out your phone though. How is taking your phone out of your pocket, logging in, and tapping it on a terminal significantly different from pulling a credit card or cash from your pocket and tapping the terminal or handing it to the checker?

• Don't get lost exploring a new city. You're young, I guess. We had GPS in cars well before iPhone. GPS navigation in cars was taking off mid-90s to mid-2000s. I had a Garmin in 2002.

• Have zero-cost video calls with anyone I want. I was doing that on my laptop and desktop before iPhone. Heck, I was doing free video conferencing with European friends in 1995.

• Use most spare moments of my time I did much of this filling in empty times on my laptops years before iPhone but you are right, not as much of it as with smartphones. Cramming my day full of even more noise, however, rather than having more breaks from it, feels like devolution to me.

• Have a real-time augmented-reality translator This is an improvement over pocket electronic translators I was using in Japan in the early 2000s, but really the improvements are mostly in fidelity and usability, not in function.

Don't get me wrong, smartphones changed a lot, but it seems like you're eliding at least a decade of pre-iphone advancements here and focusing on when these tasks became easy and in everyone's hands, rather than when the tasks actually became possible and were in reasonably widespread use. You're not a youngster like many here, so I can't attribute that to naivete and that leaves me thinking haste was at work here. Happy to hear back why I'm wrong and willing to change my mind on any of these.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#894

Post by manoDev »

dreambuffer wrote: post #509
LarsDu88 wrote: post #443

Earlier quoted context omitted.

This world model talk is interesting, and Yann Lecunn has broached on the same topic, but the fact is there are video diffusion models that are quite good at representing the "video world" and even counterfactually and temporally coherently generating a representation of that "world" under different perturbations. In fact you can go to a SOTA LLM today, and it will do quite well at predicting the outcomes of basic co…

Photons hit a human eye and then the human came up with language to describe that and then encoded the language into the LLM. The LLM can capture some of this relationship, but the LLM is not sensing actual photons, nor experiencing actual light cone stimulation, nor generating thoughts. Its "world model" is several degrees removed from the real world. So whatever fragment of a model it gains through learning to comp…

That’s a good definition: it’s a model of a model.

It seems the debate seems to center around whether language models are meta-models (in the category sense) or mere encodings (information theory)?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#895

Post by dang »

moozilla wrote: post #673
dang wrote: post #518

Earlier quoted context omitted.

Hmm good point. I skimmed the transcript looking for an accurate, representative quote that we could use in the title above. I couldn't exactly find one (within HN's 80 char limit), so I cobbled together "It will take a decade to get agents to work", which is at least closer to what Karpathy actually said. If anyone can suggest a more accurate and representative title, we can change it again. Edit: I thought of using…

You could go with the title from the associated YouTube video ( https://www.youtube.com/watch?v=lXUZvyajciY )? Andrej Karpathy — “We’re summoning ghosts, not building animals”

It's a good suggestion, but where the 'autocomplete' quote is scoped too narrowly, this one is maybe scoped too broadly. Neither really represent what the article is about.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#896

Post by darkwater »

stouset wrote: post #652
ziofill wrote: post #545

Earlier quoted context omitted.

I agree with this. A metaphor I like is that the reason why humans say the night sky is beautiful is because they see that it is, whereas an LLM says it because it’s been said enough times in its training data.

To play devil’s advocate, you have never seen the night sky. Photoreceptors in your eye have been excited in the presence of photons. Those photoreceptors have relayed this information across a nerve to neurons in your brain which receive this encoded information and splay it out to an array of other neurons. Each cell in this chain can rightfully claim to be a living organism in and of itself. “You” haven’t directly…

> As AI gains more and more capacity, we keep retreating into smaller and smaller realms of what it means to be a live, thinking being.

Maybe it's just because we never really thought about this deeply enough. And this applies even if some philosophers thought about it before the current age of LLMs.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#897

Post by rrrrrrrrrrrryan »

atleastoptimal wrote: post #887
Imnimo wrote: post #68

>What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three…

something that replaces humans doesn’t need to be 99.9999% reliable, it just has to be better than the humans it replaces.

But to be accepted by people, it has to be better than humans in the specific ways that humans are good at things. And less bad than humans in the ways that they're bad at things.

When automated solutions fail in strange alien ways, it understandably freaks people out. Nobody wants to worry about if a car will suddenly serve into oncoming traffic because of a sensor malfunction. Comparing incidents-per-miles-driven might make sense from a utilitarian perspective, just isn't good enough for humans to accept replacement tech psychologically, so we do have to chase those 9s until they can handle all the edge cases at least as well as humans.

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#899

Post by throwaway314155 »

steveBK123 wrote: post #806
getnormality wrote: post #804

One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today. No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward. Also, speculation…

It's nice seeing commentary from someone who is both knowledgable in AI and NOT trying to pump the AI bag. Right now the median actor in the space loudly proclaims AGI is right around the corner, while rolling out pornbots/ads/in-chat-shopping, which generally seems at odds with a real belief that AGI is close (TAM of AGI must be exponentially larger than the former).

What exactly is a "pornbot"?

Re: Andrej Karpathy – It will take a decade to work through the issues with agents

#900

Post by bondarchuk »

dlcarrier wrote: post #241

Redefinitions aside, fully capable AI is right up there with commercially viable fusion power, cost effective quantum completing, and fully capable self-driving cars, as a technology that is quickly advancing yet always a decade or two away.

re: fusion: https://commons.wikimedia.org/wiki/File:U.S._historical_fusi...
Post reply on HN