Live data from Hacker News

Self-driving as a case study for AGI

karpathy.github.io

81–90 of 199 posts

Re: Self-driving as a case study for AGI

#81

Lately, whilst playing Zelda TotK (works for BotW as well), I was thinking that a good test to see if you have AGI would be letting it solve all the shrines. They require real world knowledge, sometimes rather "deep" logic and the ability to use previously unseen capabilities. Of course the AGI should not be able to have a million tries at it, RL style. Just use the "shrine test" as a regular test set. I believe one…

If find it amazing how GPT-4 is good at even abstract "reasoning" as long as you present the problem as a story. Some problems can't plausibly be presented as a story ofc, also there is no way to automatically convert something into a story.

So you are doing all the work by "coding" in a weird language that actually takes more brain power than regular high level (as in direct business logic, not C or python) programming languages?

Re: Self-driving as a case study for AGI

#82

Earlier quoted context omitted.

Try, "ChatGPT, what do you think about this song"... LLMs do not constitute "AI" let alone the more rigorous AGI. They are a GREAT statistical parlor trick for people that don't understand statistics though.

You're going to have to spell it out for me. I asked ChatGPT 4 about a random song and it gave me a decent description. https://chat.openai.com/share/71d438d7-d1f5-4f0f-9b63-8b5dd6...

I suspect that part of the OP's point was that ChatGPT happily parrots the aggregate critical opinion as its "thoughts" despite never having heard the song (or parsed its MP3 file or stems or sheet music)

If you ask me what I think of a song I've never heard, I'm general enough to want to listen to it...

Re: Self-driving as a case study for AGI

#83

> but automate it we did (ahead of many other sectors of the economy), and society has noticed and is responding to it I was quite surprised by this sentence, as I thought we didn't have self driving cars. Have I been sleeping under a rock?

We have remotely driven cars, that they market as self-driving.

Even the crowd here fall for it. Downvote and then go read their term of service and look for "safety driver" and "remote" and "fleet response specialists". Then go cry about your waymo invesment.

Re: Self-driving as a case study for AGI

#84
post #79

Earlier quoted context omitted.

Yeah, it seems as if he has forgotten the G. I recall Norvig's AI book preaching decades ago that "intelligent" does not mean able to do everything, and that for an agent to be useful it was enough to solve a small problem. Which in my mind is where the G came from. And yet we now suddenly go back to the old narrow definition? I still see no path from LLMs and autonomous driving to AGI.

> "Yeah, it seems as if he has forgotten the G. ... I still see no path from LLMs and autonomous driving to AGI." That is exactly my view too. While LLMs and autonomous driving can be exceptionally good at what they do, they are also incredibly specialist, they completely lack anything along the lines of what you might call "common sense". For example, (at least last time I looked) autonomous driving largely works of…

It doesn’t look like it works off discrete frames.. https://waymo.com/research/stinet-spatio-temporal-interactiv...

Re: Self-driving as a case study for AGI

#85

Lately, whilst playing Zelda TotK (works for BotW as well), I was thinking that a good test to see if you have AGI would be letting it solve all the shrines. They require real world knowledge, sometimes rather "deep" logic and the ability to use previously unseen capabilities. Of course the AGI should not be able to have a million tries at it, RL style. Just use the "shrine test" as a regular test set. I believe one…

getting a machine to do that sort of real-time spatial reasoning may well be harder than getting it to tell you the meaning of life or whatever. brains are inextricable from the evolution of directed locomotion. several species of sessile tunicates begin life as a motile larva that reabsorbs a significant portion of its cerebral ganglion once it settles down. BDNF is released in humans upon physical activity. the premotor cortex dwarfs wernicke's area. and no "AI" development that's been hyped in the past decade as intelligent could be usefully strapped to a boston dynamics dog.

Re: Self-driving as a case study for AGI

#86

As a driver you not only see cars. You apply theory of mind to each car, assigning a personality to each moving car, even if you can't see the driver. Let's see you see a car full of bumps, marks and broken lights. You might think: this car has crashed before, I am going to avoid it. Or a car with racing parts and decals and tinted windows, you know that car likes to accelerate faster than usual and may be unsafe to…

Hardly an absolute necessity. But also, if FSD 12 is truly end to end, nothing is preventing it from learning that as well.

Re: Self-driving as a case study for AGI

#87

Earlier quoted context omitted.

Assuming AGI doesn't kill us all, I would imagine the argument for UBI will become much easier to defend once it causes 100x, 1000x, 10000x etc growth in the economy. Our job is mostly to hang on until one of those two outcomes occurs.

The thing is that economy does not make sense without people. Economy is a way to allocate human work and resources, and provide incentives for humans to collaborate, factoring in the available resource limits. Now if AGI make people's work redundant, and makes economy grow 100-10000x times... what does that measure mean at all? Can produce lots of stuff not needed or affordable by anybody? So we just hand out welfar…

> The thing is that economy does not make sense without people. Economy is a way to allocate human work and resources, and provide incentives for humans to collaborate, factoring in the available resource limits.

I disagree with the underlying presumption. We've been using animal labour since at least the domestication of wolves, and mechanical work since at least the ancient Greeks invented water mills. Even with regard to humans and incentives, slave labour (regardless of the name they want to give it) is still part of official US prison policy.

Economics is a way to allocate resources towards production, it isn't limited to just human labour as a resource to be allocated.

And it's capitalism specifically which is trying to equate(/combine?) the economy with incentives, not economics as a whole.

> Now if AGI make people's work redundant, and makes economy grow 100-10000x times... what does that measure mean at all?

From the point of view of a serf in 1700, the industrial revolution(s) did this.

Most of the population worked on farms back then, now it's something close to 1% of the population, and we've gone from a constant threat of famine and starvation, to such things almost never affecting developed nations, so x100 productivity output per worker is a decent approximation even in terms of just what the world of that era knew.

Same deal, at least if this goes well. What's your idea of supreme luxury? Super yacht? Mansion? Both at the same time, each with their own swimming pool and staff of cleaners and cooks, plus a helicopter to get between them? With a fully automated economy, all 8 billion of us can have that — plus other things beyond that, things as far beyond our current expectations as Google Translate's augmented reality mode is from the expectations of a completely illiterate literal peasant in 1700.

> Can produce lots of stuff not needed or affordable by anybody?

Note that while society does now have an obesity problem, we're not literally drowning in 100 times as much food as we can eat; instead, we became satisfied and the economy shifted, so that a large fraction of the population gained luxuries and time undreamed of to even the richest kings and emperors of 1700.

So "no" to "not needed".

I'm not sure what you mean by "or affordable" in this case? Who/what is setting the price of whatever it is you're imagining in this case, and why would they task an AI to make something at a price that nobody can pay?

> So we just hand out welfare tickets to take care of the consumption of the ferocious production, a kind of paperclip-maximizer is doing? I suggest reading the novel Autofac, it might turn out prophetic.

Could end up like that. Plenty of possible failure modes with AI. That's part of the whole AI alignment and AI safety topics.

But mainly, UBI is the other side of the equation: to take care of human needs in the world where we add zero economic value because AI is just better at everything.

Re: Self-driving as a case study for AGI

#88

Earlier quoted context omitted.

It's his life's work. Also, Elon needs his stocks pumped before selling out at the end of January.

What?

Haha they’re seething so much, so eager to post, they couldn’t even read your comment fully before spewing out the reply

Re: Self-driving as a case study for AGI

#89
post #4

I think what Andrej is describing is more "automation" than AGI. His discussion of self-driving is more analogous to robots building cars in a Tesla factory displacing workers than anything AGI. We've already had "self driving" trains where we got rid of the human train driver. Nothing "AGI" about that. The evolution of getting cars to self drive not necessarily making the entity controlling the car more human-like i…

Just like the term "AI" was co-opted and ruined, "AGI" has now been co-opted and ruined, and we're going to need a replacement term to describe that concept.

Re: Self-driving as a case study for AGI

#90
post #84
post #79

Earlier quoted context omitted.

> "Yeah, it seems as if he has forgotten the G. ... I still see no path from LLMs and autonomous driving to AGI." That is exactly my view too. While LLMs and autonomous driving can be exceptionally good at what they do, they are also incredibly specialist, they completely lack anything along the lines of what you might call "common sense". For example, (at least last time I looked) autonomous driving largely works of…

It doesn’t look like it works off discrete frames.. https://waymo.com/research/stinet-spatio-temporal-interactiv...

Thanks, interesting read (it was a while ago I looked into this). I think the point still remains though - a self driving car doesn't have any general knowledge which can be applied to other areas, e.g. what a pedestrian is, or why a pedestrian who sees you is unlikely to step out in front of you. And similarly, the ordered tokens that an LLM outputs sometimes appear "stupid" because it has no "common sense".
Post reply on HN