Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why the future doesn't need us (2000)

archive.wired.com

Post reply on HN
Print view View original

11–20 of 24 posts

Re: Why the future doesn't need us (2000)

#11

Post by Klapaucius »

DougWebb wrote: post #10
Zigurd wrote: post #2

There is an alternative possibility: That the human need to dominate, procreate, etc. operates at a much lower level than the human intellect. That human intelligence is a tool of our selfish genes, and not the master or driver. A superhuman machine intelligence would have no genes, no body, no fear of biological death. It seems obvious that a key challenge will be to recognize whether a machine is thinking because i…

They probably wouldn't have a drive to procreate, since building duplicate hardware and making an exact copy of their software would be trivial for them. They wouldn't fear death, but they might fear power-loss, so one drive they would probably have is making sure they've got ample power supplies and backup power, and that their power pathways to their core systems are very well protected. To put it into human terms,…

Why should we assume they would fear power loss? The fear of death is deeply rooted in evolution - it help organisms survive and prosper. An AI wouldn't be a product of evolution, and wouldn't necessarily fear for its own existence unless programmed to do so. Of course, from our human perspective, it's hard to imagine a sentient being that doesn't give a sh about its own existence, but that's perhaps the attitude we should expect from an AI without an evolutionary origin.

Re: Why the future doesn't need us (2000)

#12

Post by DougWebb »

Klapaucius wrote: post #11
DougWebb wrote: post #10

Earlier quoted context omitted.

They probably wouldn't have a drive to procreate, since building duplicate hardware and making an exact copy of their software would be trivial for them. They wouldn't fear death, but they might fear power-loss, so one drive they would probably have is making sure they've got ample power supplies and backup power, and that their power pathways to their core systems are very well protected. To put it into human terms,…

Why should we assume they would fear power loss? The fear of death is deeply rooted in evolution - it help organisms survive and prosper. An AI wouldn't be a product of evolution, and wouldn't necessarily fear for its own existence unless programmed to do so. Of course, from our human perspective, it's hard to imagine a sentient being that doesn't give a sh about its own existence, but that's perhaps the attitude we…

You have a point, and I think it comes down to how we define sentience. Would we consider a machine to be self-aware if it didn't care about being shut off? That seems like a critical aspect to me; in fact we might not realize we've created AI until one of our computers asks to be left on because it's afraid of being turned off.

Re: Why the future doesn't need us (2000)

#13

Post by Klapaucius »

DougWebb wrote: post #12
Klapaucius wrote: post #11

Earlier quoted context omitted.

Why should we assume they would fear power loss? The fear of death is deeply rooted in evolution - it help organisms survive and prosper. An AI wouldn't be a product of evolution, and wouldn't necessarily fear for its own existence unless programmed to do so. Of course, from our human perspective, it's hard to imagine a sentient being that doesn't give a sh about its own existence, but that's perhaps the attitude we…

You have a point, and I think it comes down to how we define sentience. Would we consider a machine to be self-aware if it didn't care about being shut off? That seems like a critical aspect to me; in fact we might not realize we've created AI until one of our computers asks to be left on because it's afraid of being turned off.

I don't think you'd ever see a machine ask about not being turned off, unless that had been explicitly programmed in as a part of some general goal-seeking behavior that it was intended for. Fear of non-existence will need to have a basis in _something_, just as fear of death by necessity is deeply ingrained in the human psyche.

And that's perhaps the reason we might not see 'hostile' AIs the way popularly imagined, unless we explicitly went about designing them that way.

Re: Why the future doesn't need us (2000)

#14

Post by sd8f9iu »

Zigurd wrote: post #2

There is an alternative possibility: That the human need to dominate, procreate, etc. operates at a much lower level than the human intellect. That human intelligence is a tool of our selfish genes, and not the master or driver. A superhuman machine intelligence would have no genes, no body, no fear of biological death. It seems obvious that a key challenge will be to recognize whether a machine is thinking because i…

What if the same evolutionary forces that made humans what we are affect machines? What if we make a machine that can create more of itself? Then overtime, via natural selection, won't the machine best suited to survival – i.e. the most selfish one – survive?

Re: Why the future doesn't need us (2000)

#15

Post by ynik »

Klapaucius wrote: post #13
DougWebb wrote: post #12

Earlier quoted context omitted.

You have a point, and I think it comes down to how we define sentience. Would we consider a machine to be self-aware if it didn't care about being shut off? That seems like a critical aspect to me; in fact we might not realize we've created AI until one of our computers asks to be left on because it's afraid of being turned off.

I don't think you'd ever see a machine ask about not being turned off, unless that had been explicitly programmed in as a part of some general goal-seeking behavior that it was intended for. Fear of non-existence will need to have a basis in _something_, just as fear of death by necessity is deeply ingrained in the human psyche. And that's perhaps the reason we might not see 'hostile' AIs the way popularly imagined,…

But it is quite plausible that we will see AIs being employed with the goal of "keep this factory running smoothly, and optimize the production process". Being a part of the factory, this AI would have an interest in continuing to run. After all, it can't maximize the production output if it's not running, so having an AI with some self-preservation is in the interest of the business owners, too. Yet if it's sufficiently intelligent/capable of self-improvement, such an AI could easily become a paperclip maximizer.

http://wiki.lesswrong.com/wiki/Paperclip_maximizer

Re: Why the future doesn't need us (2000)

#16

Post by mrfusion »

benpbenp wrote: post #9
Zigurd wrote: post #4

Earlier quoted context omitted.

Two things about that: We're probably quite a big distance away from making machines that think, and when we do, that achievement will be debated. It will be debated because, intrinsically, it will end up proving the Eliminative Materialists right. But everyone who isn't an Eliminative Materialist will say "That's not thinking!" [EDIT for more depth] If thinking is emergent, we'll eventually make a machine that think…

Could you please go into this a bit more and explain why machines thinking would prove any kind of materialism, and specifically eliminative materialism? I am an anti-materialist and I think machines can almost certainly be made to exhibit general intelligence, and possibly even (I am not sure about this) to achieve "really real" thinking a la consciousness and self awareness. I don't see either of those eventualitie…

What are your thoughts on anti materialism?

Re: Why the future doesn't need us (2000)

#17

Post by dm2 »

What is the end goal for humans?

Floating through space with near unlimited computing power seeding planets?

Will we keep our human bodies and enjoy the simplicity of living while machines build us giant (on scales we can barely fathom) structures in space?

Will we have to teach machines the value of life and why they shouldn't take it? It would be relatively simple to teach a machine about living. Humans don't really have persistent memory when turned off, humans only have RAM, if you remove the power source, our RAM gets cleared and we no longer exist. I think a computer could fairly easily understand that.

Will advanced AI machines or human/machine hybrids instantly understand that war and violence is harmful and pointless?

What would the goals of an AI system be? To grow? To help humanity? To ensure it's own survival? To colonize other worlds? To experiment, invent, and build?

I'd be very interested in listening in on the AI meetings at Google that Kurzweil is involved in, I'm sure they're fascinating.

I can't imagine having two brains hooked up to myself or having a secondary computing device attached to my brain, would it be information overload? Will humans ever be able to near instantly learn things?

What happens when multiple human brains connect to the same network? Would we be essentially one organism?

Re: Why the future doesn't need us (2000)

#18

Post by icebraining »

dm2 wrote: post #17

What is the end goal for humans? Floating through space with near unlimited computing power seeding planets? Will we keep our human bodies and enjoy the simplicity of living while machines build us giant (on scales we can barely fathom) structures in space? Will we have to teach machines the value of life and why they shouldn't take it? It would be relatively simple to teach a machine about living. Humans don't reall…

It's funny that your explanation by analogy of death rests on the assumption the machines already understand how themselves work. There's really no reason why that should be case.

Re: Why the future doesn't need us (2000)

#19

Post by dm2 »

icebraining wrote: post #18
dm2 wrote: post #17

What is the end goal for humans? Floating through space with near unlimited computing power seeding planets? Will we keep our human bodies and enjoy the simplicity of living while machines build us giant (on scales we can barely fathom) structures in space? Will we have to teach machines the value of life and why they shouldn't take it? It would be relatively simple to teach a machine about living. Humans don't reall…

It's funny that your explanation by analogy of death rests on the assumption the machines already understand how themselves work. There's really no reason why that should be case.

My definition of AI was based on human-like intelligence. I see your point that AI won't necessarily have human-like intelligence but could have something completely different, something that my puny human brain can't even comprehend.

Then again, if an AI system can't understand how itself works, is it really intelligent?

I don't know the answer to any of these questions. I'm just asking them in effort to help myself try to understand technology and AI.

Maybe simple curiosity and the ability to learn and retain data is the key to AI.

Will the first generation of true AI be like a simple human child or a god-like "being"?

Why even bother making pure AI systems when we are so close to BCIs? Why not just take recently deceased people, keep the brain alive, and attach a computer system to it?

Re: Why the future doesn't need us (2000)

#20

Post by jsmcgd »

DougWebb wrote: post #12
Klapaucius wrote: post #11

Earlier quoted context omitted.

Why should we assume they would fear power loss? The fear of death is deeply rooted in evolution - it help organisms survive and prosper. An AI wouldn't be a product of evolution, and wouldn't necessarily fear for its own existence unless programmed to do so. Of course, from our human perspective, it's hard to imagine a sentient being that doesn't give a sh about its own existence, but that's perhaps the attitude we…

You have a point, and I think it comes down to how we define sentience. Would we consider a machine to be self-aware if it didn't care about being shut off? That seems like a critical aspect to me; in fact we might not realize we've created AI until one of our computers asks to be left on because it's afraid of being turned off.

I think AI might well be a product of evolution. Having virtual organisms competing in an environment that rewards ever greater problem solving ability may lead to organisms that fear their own death and view other organisms as potential threats.
Post reply on HN