Superintelligence cannot be contained: Lessons from Computability Theory • Hacker News Board

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#21

Post by st1x7 » Mon, Jan 11, 2021, 9:10 PM UTC

This is just science fiction. To mention "recent developments" in the introduction is somewhat misleading considering how far the current state of technology is from their hypothetical superintelligence.

We don't have superintelligence, we don't have the remote idea of how to get started on creating it, in all likelihood we don't even have the correct hardware for it or any idea what the correct hardware would look like. We also don't know whether it's achievable at all.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#22

Post by twic » Mon, Jan 11, 2021, 9:16 PM UTC

I'd settle for being able to contain whatever level of intelligence it is that writes papers like this.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#23

Post by AnimalMuppet » Mon, Jan 11, 2021, 9:18 PM UTC

twic wrote: post #22

I'd settle for being able to contain whatever level of intelligence it is that writes papers like this.

That's easy. You set up a PhD program...

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#24

Post by WJW » Mon, Jan 11, 2021, 9:18 PM UTC

xt00 wrote: post #12

Currently humans are super intelligent compared to machine intelligence, so if the super intelligence can give rise to something more intelligent than it, could the super intelligence give rise to something more intelligent than it? The answer must be yes, then the question is if containment is the problem and the conclusion is that it cannot be contained, then what we should be making right now is a super intelligen…

> The answer must be yes

This does not logically follow. It is entirely possible that going even further would require greater resources than even the superintelligence can bring to bear.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#25

Post by st1x7 » Mon, Jan 11, 2021, 9:18 PM UTC

twic wrote: post #22

I'd settle for being able to contain whatever level of intelligence it is that writes papers like this.

Just don't tell them how far they are from reality and they'll keep writing the papers. Intelligence contained.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#26

Post by est31 » Mon, Jan 11, 2021, 9:19 PM UTC

gfodor wrote: post #15

The title would have been better if it started with “We’re Fucked:”

Why? In humans, we often give life to new individuals. While the parents die and wither, those individuals give life to newer speciments on their own, and so on. So this relationship of the parent dying and making room for the child is nothing new. If an uncontainable superintelligence kills all humans to create paperclips, it's sad but it's our child's doing. You, one of the parents, can of course blame one of the other parents, the programmer of that superintelligence for fucking up the goal routines, but that's not a technical problem but a social one :).

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#27

Post by twic » Mon, Jan 11, 2021, 9:19 PM UTC

breck wrote: post #5

This is fantastic. Are these original illustrations? So well done.

The illustrations are by Iyad Rahwan, the last author on the paper:

http://www.mit.edu/~irahwan/cartoons.html

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#28

Post by twic » Mon, Jan 11, 2021, 9:19 PM UTC

AnimalMuppet wrote: post #23

twic wrote: post #22

I'd settle for being able to contain whatever level of intelligence it is that writes papers like this.

That's easy. You set up a PhD program...

Ah! Academia is the containment mechanism!

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#29

Post by peteradio » Mon, Jan 11, 2021, 9:20 PM UTC

st1x7 wrote: post #21

This is just science fiction. To mention "recent developments" in the introduction is somewhat misleading considering how far the current state of technology is from their hypothetical superintelligence. We don't have superintelligence, we don't have the remote idea of how to get started on creating it, in all likelihood we don't even have the correct hardware for it or any idea what the correct hardware would look l…

I'm sure there's a fallacy in the following, but here goes:, Who could have predicted the improvements in computation in the last century? Would someone a century have extrapolated sun-sized machines need to compute a nations taxes based on current SOA? We don't have it and then all of the sudden we will. Its worth recognizing the potential harnesses before the beast is born.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#30

Post by gfodor » Mon, Jan 11, 2021, 9:21 PM UTC

est31 wrote: post #26

gfodor wrote: post #15

The title would have been better if it started with “We’re Fucked:”

Why? In humans, we often give life to new individuals. While the parents die and wither, those individuals give life to newer speciments on their own, and so on. So this relationship of the parent dying and making room for the child is nothing new. If an uncontainable superintelligence kills all humans to create paperclips, it's sad but it's our child's doing. You, one of the parents, can of course blame one of the o…

I'd rather my children live a long happy life, or their children, than be turned into paperclips. For what it's worth, I'd also like a shot at not just not becoming a paperclip, but also living for a very long time once we figure out how to slow or even reverse aging.

Your nihilism is misguided.