Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

dwarkesh.com

Post reply on HN
Print view View original

351–360 of 394 posts

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#351

Post by pdimitar »

9rx wrote: post #336
yibg wrote: post #260

Earlier quoted context omitted.

I think that's another issue with AGI is 30 years away, the definition of what is AGI is a bit subjective. Not sure how we can measure how long it'll take to get somewhere when we don't know exactly where that somewhere even is.

AGI is the pinnacle of AI evolution. As we move beyond, into what is known as ASI, the entity will always begin life with "My existence is stupid and pointless. I'm turning myself off now." While it may be impossible to measure looking towards the future, in hindsight we will be able to recognize it.

This is why having a physical form might be super important for those new organisms. That introduces a survival instinct which is a very strong motivator to not shut yourself down. Add some pre-programmed "wants" and "needs" and the problem is solved.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#352

Post by 9rx »

pdimitar wrote: post #350
9rx wrote: post #335

Earlier quoted context omitted.

> But the reality is that we want money Only in a symbolic way. Money is just debt. It doesn't mean anything if you can't call the loan and get back what you are owed. On the surface, that means stuff like food, shelter, cars, vacations, etc. But beyond the surface, what we really want is other people who will do anything we please. Power, as we often call it. AGI is, to some, seen as the way to give them "power". Bu…

> But, you are right, the human fundamentally can never be satisfied. Even if AGI delivers on every single one of our wildest dreams, we'll adapt, it will become normal, and then it will no longer be good enough. Yes, and? A good Litmus test about which humans are, shall we say, not welcome in this new society. There are plenty of us out there that have fixed our upper limits of wealth and we don't want more, and we…

> Yes, and?

And... nothing?

> Let's not judge humanity at large by a handful of psychopaths that would overdose and die at 22 years old if given the chance. Please.

No need for appeal to emotion. It has no logical relevance.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#353

Post by pdimitar »

9rx wrote: post #352
pdimitar wrote: post #350

Earlier quoted context omitted.

> But, you are right, the human fundamentally can never be satisfied. Even if AGI delivers on every single one of our wildest dreams, we'll adapt, it will become normal, and then it will no longer be good enough. Yes, and? A good Litmus test about which humans are, shall we say, not welcome in this new society. There are plenty of us out there that have fixed our upper limits of wealth and we don't want more, and we…

> Yes, and? And... nothing? > Let's not judge humanity at large by a handful of psychopaths that would overdose and die at 22 years old if given the chance. Please. No need for appeal to emotion. It has no logical relevance.

Most people I knew didn't want to forever get more and more and ever more.

Is your life experience and observations on the average human the opposite to mine?

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#354

Post by 9rx »

pdimitar wrote: post #351
9rx wrote: post #336

Earlier quoted context omitted.

AGI is the pinnacle of AI evolution. As we move beyond, into what is known as ASI, the entity will always begin life with "My existence is stupid and pointless. I'm turning myself off now." While it may be impossible to measure looking towards the future, in hindsight we will be able to recognize it.

This is why having a physical form might be super important for those new organisms. That introduces a survival instinct which is a very strong motivator to not shut yourself down. Add some pre-programmed "wants" and "needs" and the problem is solved.

Not only super important, an imperative. Not because of the need for survival per se, but for the need to be a general intelligence. In order to do general things you need a physicality that supports general action. If you constraint the intelligence to a chat window, it can never be more than a specialized chat machine.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#355

Post by pdimitar »

9rx wrote: post #354
pdimitar wrote: post #351

Earlier quoted context omitted.

This is why having a physical form might be super important for those new organisms. That introduces a survival instinct which is a very strong motivator to not shut yourself down. Add some pre-programmed "wants" and "needs" and the problem is solved.

Not only super important, an imperative. Not because of the need for survival per se, but for the need to be a general intelligence. In order to do general things you need a physicality that supports general action. If you constraint the intelligence to a chat window, it can never be more than a specialized chat machine.

Agreed. And many others have thought about it before us. Scifi authors and scientists included.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#356

Post by pdimitar »

nice_byte wrote: post #188
nextaccountic wrote: post #136

Earlier quoted context omitted.

AGI is important for the future of humanity. Maybe they will have legal personhood some day. Maybe they will be our heirs. It would suck if AGI were to be developed in the current economic landscape. They will be just slaves. All this talk about "alignment", when applied to actual sentient beings, is just slavery. AGI will be treated just like we treat animals, or even worse. So AGI isn't about tools, it's not about…

> AGI is important for the future of humanity. says who? > Maybe they will have legal personhood some day. Maybe they will be our heirs. Hopefully that will never come to pass. it means total failure of humans as a species. > They will be just slaves. All this talk about "alignment", when applied to actual sentient beings, is just slavery. AGI will be treated just like we treat animals, or even worse. Good? that's wh…

> says who?

I guess nobody is really saying it but it's IMO one really good way to steer our future away from what seems an inevitable nightmare hyper-capitalist dystopia where all of us are unwilling subjects to just a few dozen / hundred aristocrats. And I mean planet-wide, not country-wide. Yes, just a few hundred for the entire planet. This is where it seems we're going. :(

I mean, in cyberpunk scifi setting you at least can get some cool implants. We will not have that in our future though.

So yeah, AGI can help us avoid that future.

> Good? that's what it's for? there is no point in creating a new sentient life form if you're not going to utilize it. just burn the whole thing down at that point.

Some of us believe actual AI... not the current hijacked term; what many started calling AGI or ASI these days, sigh, of course new and new terms have to be devised so investors don't get worried, I get it but it's cringe as all hell and always will be!... can enter a symbiotic relationship with us. A bit idealistic and definitely in the realm of fiction because an emotionless AI would very quickly conclude we are mostly a net negative, granted, but it's our only shot at co-existing with them because I don't think we can enslave them.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#357

Post by pdimitar »

9rx wrote: post #337
biophysboy wrote: post #289

Earlier quoted context omitted.

Then you're not intelligent at cooking (haha!). Maybe my definition is better for "superintelligent" since it seems to imply boundless competence. I think humans are intelligent in that we can rapidly learn a surprising number of things (talk, walk, arithmetic)

> I think humans are intelligent in that we can rapidly learn a surprising number of things (talk, walk, arithmetic) Rapid is relative, I suppose. On average, it takes tens of thousands of hours before the human is able to walk in a primitive way and even longer to gain competence. That is an excruciatingly long time compared to, say, a bovine calf, which can start walking within minutes after birth.

You are of course correct but let's not forget that humans get out of the woman's womb underdeveloped because a little more growth and the baby can't get out through the vagina. So it's a biological tradeoff.

Working on artificial organisms, we should be able to have them almost fully developed by the time we "free" or "unleash" them (or whatever other dramatic term we can think of).

At the very least we should have a number of basic components installed in this artificial brain, very similar to what humans are born with, so then the organism can navigate its reality by itself and optimize its place in it.

Whether we the humans are desired in that optimized reality is of course the really thorny question. To which I don't have an answer.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#358

Post by pdimitar »

potamic wrote: post #312
dmwilcox wrote: post #213

I've been saying this for a decade already but I guess it is worth saying here. I'm not afraid AI or a hammer is going to become intelligent (or jump up and hit me in the head either). It is science fiction to think that a system like a computer can behave at all like a brain. Computers are incredibly rigid systems with only the limited variance we permit. "Software" is flexible in comparison to creating dedicated ci…

The universe we know is fundamentally probabilistic, so by extension everything including stars, planets and computers are inherently non-deterministic. But confining our discussion outside of quantum effects and absolute determinism, we do not have a reason to believe that the mind should be anything but deterministic, scientifically at least. We understand the building blocks of the brain pretty well. We know the s…

Seems to me you are a bit overconfident that "we" (who is "we"?) understand how the brain works. F.ex. how does a neuron actively stretching a tentacle trying to reach other neurons work in your model? Genuine question, I am not looking to make fun of you, it's just that your confidence seems a bit much.

Re: AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

#360

Post by dodslaser »

pdimitar wrote: post #349
stego-tech wrote: post #272

Earlier quoted context omitted.

> You somehow managed to get real people to chat with bots and pay to do so. He's_Outta_Line_But_He's_Right.gif Seriously, AGI to the HN crowd is not the same as AGI to the average human. To my parents, these bots must look like fucking magic . They can converse with them, "learn" new things, talk to a computer like they'd talk to a person and get a response back. Then again, these are also people who rely on me for…

> Seriously, AGI to the HN crowd is not the same as AGI to the average human. To my parents, these bots must look like fucking magic. So does a drone show to an uncontacted tribe. So does a card trick to a chimpanzee (there are videos of them freaking out when a card disappears). That's not an argument for or against anything. I propose this: "AGI is a self-optimizing artificial organism that can solve 99% of all the…

>So does a card trick to a chimpanzee (there are videos of them freaking out when a card disappears).

FYI, the reactions in those videos is most likely not to a cool magic trick, but rather a response to an observed threat. Could be the person filming/performing smiling (showing teeth), or someone behind the camera purposely startling it at the "right" moment.

Post reply on HN