Superintelligence cannot be contained: Lessons from Computability Theory • Hacker News Board

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#51

Post by jandrese » Mon, Jan 11, 2021, 9:56 PM UTC

ben_w wrote: post #47

jandrese wrote: post #13

Earlier quoted context omitted.

It's the classic case of seeing a curve pointing upwards and thinking it will continue doing that forever, even though the universe is like 0 for nearly-uncountable in cases where that has held true indefinitely. Every growth curve is an S curve. The AI Singularity is like the Grey Goo scenario. An oversimplified model projected out so far in the future that its flaws become apparent.

Depends which version of “the singularity” is being discussed. IIRC, the original (almost certainly false) idea was a sufficiently powerful AI can start a sequence of ever power powerful AI with decreasing time between each step — reaching infinite power in finite time. I don’t need that version of the singularity to be worried. I think in terms of “the event horizon” rather than “the singularity”: all new tech chang…

The Grey Goo scenario is that it starts eating the planet until the only thing left is a sea of nanomachines.

However, thermodynamics says it becomes increasingly more difficult to distribute power (food) to a growing population and it hits a natural limit to growth, just like bacteria.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#52

Post by stephc_int13 » Mon, Jan 11, 2021, 9:57 PM UTC

In my opinion, all the talks about the potential danger of advanced AI is highly speculative and revolves around a very simple thing: fear of the unknown, that's all.

We simply don't know.

And some people are also afraid of creation by accident, because intelligence is seen as an emergent property of complex networks, but again, this is because we don't understand much about it.

Tldr; Nothing to see here, move along.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#53

Post by cambalache » Mon, Jan 11, 2021, 9:59 PM UTC

I confess I am a frustrated academic, but from time to time I read a paper like this one and and convince myself that my frustration may perhaps be unwarranted.

RECIPE TO PROVE ANYTHING IS INCOMPUTABLE (According to this paper).

Example, beauty is incomputable

Step 1. Assume there is an algorithm Beauty(R,D) that given the program R and the input D, will return True if it is beautiful and False otherwise.

Step 2. Create a high-order function that is a simple wrapper of the halting problem over the algorithm created in step 1

Theorem. Beauty is undecidable.

Proof. Assume by contradiction that the halting problem is decidable, then, feed it the function created in step 2, thus you can always have an answer for step 1, but since the halting problem is undecidable this is not possible.

For more variety please substitute beauty for "awesomeness", "arachnid power", "goodness","fear" and so on.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#54

Post by f430 » Mon, Jan 11, 2021, 10:03 PM UTC

To me the question as to whether we will witness artificially created superintelligence hinges on whether we will find concrete proof of non-locality of the consciousness.

When the latter happens, we will not only be on the final stretch to become a Type I civilization but flirting with the threshold of Type II civilization very quickly.

I believe that an advanced interstellar civilization simply cannot rely on Newtonian physics and petrodollar economy. It needs to transcend beyond this dimension because we are limited by Speed of Light in the 3d world. trans-dimensional projection and materialization much like what we see with "UFOS" is the only way to cover massive distances. Our science shows the possibility of wormhole travel. The maths show time travel is possible. We need to breakout of the waterfall model of time, space, our existence.

Speed of Light is far too slow and inefficient for us to be a multi-planetary and eventually a multi-galaxy species. We also cannot accomplish this through wars or ape-like behaviors.

A shift in global consciousness is in order and I think we are slowly heading towards that. We have never seen so much awareness for the well being of the planet which we need to stop treating like a piece of rock that we feel entitled to.

Like I mentioned before, there is something pulling us to expand, become better, become one with the universe. The maths show the universe is teeming with life, we have evidence of life on Venus FFS now, we can infer that life isn't rare it is our narrow view of life and intelligence that limits our search.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#55

Post by twhitmore » Mon, Jan 11, 2021, 10:16 PM UTC

boxmonster wrote: post #43

These arguments about hypothetical super intelligences are interesting, but my concern is not very great because we can just pull the power plug if necessary

I think this assumption is worth at least casually testing.

Turning off sounds easy -- Except that it's almost certainly connected to, or adjacent to, the network. Where else are the large volumes of training data coming from?

Presumably being super-intelligent the AI would be:

1) likely to anticipate being turned off,

2) super-normally capable to identify & traverse any network security measures;

3) potentially also highly capable to identify & traverse weaknesses in other (eg. social or physical) security measures).

4) possibly able to be covert/ stealthy actions via the network.

5) possibly able, if able to reform on the Internet, to escape to there.

I'm fairly much on the super-AI fence. But in a dystopian scenario the proposal of a super AI being able to be turned off as the key means of safety, seems scarcely credible.

Note even that without being able to reform on the Internet (eg physical embodiment requires singularly specialized computing equipment), I can't logically see that extreme hostile actions (destroy global finance system, subversive propaganda campaigns, launch wars, launch nukes) are obviously easy to preclude.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#56

Post by peteradio » Mon, Jan 11, 2021, 10:17 PM UTC

semi-extrinsic wrote: post #41

plutonorm wrote: post #32

Earlier quoted context omitted.

That's the mainstream opinion on every. single. revolutionary advance. That you and everyone else believes it's not going to happen ever has almost no predictive power as to whether it actually will.

It's not so much "opinion on a revolutionary advance". When it comes to AGI-related stuff, we are quite literally like contemporaries of Leonardo da Vinci, who have seen his plans for the helicopter and are postulating that helicopters will cause big problems if they fly too high and crash into the mechanism that is holding up the sky above us. Also, this is not the mainstream opinion on e.g. fusion, or electric cars…

I think we fear ourselves (rightly) and super-fear a super-self (rightly). If this theoretical thing has the capacity at all to interact the way humans do then it will probably be brutal to at least someone and possibly most or everyone.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#57

Post by xt00 » Mon, Jan 11, 2021, 10:20 PM UTC

WJW wrote: post #24

xt00 wrote: post #12

Currently humans are super intelligent compared to machine intelligence, so if the super intelligence can give rise to something more intelligent than it, could the super intelligence give rise to something more intelligent than it? The answer must be yes, then the question is if containment is the problem and the conclusion is that it cannot be contained, then what we should be making right now is a super intelligen…

> The answer must be yes This does not logically follow. It is entirely possible that going even further would require greater resources than even the superintelligence can bring to bear.

Definitely feels like a challenging philosophical question -- if you have a superintelligence how much more room is there above it in intelligence and what does that additional intelligence buy you?

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#58

Post by [deleted] » Mon, Jan 11, 2021, 10:21 PM UTC

cambalache wrote: post #53

I confess I am a frustrated academic, but from time to time I read a paper like this one and and convince myself that my frustration may perhaps be unwarranted. RECIPE TO PROVE ANYTHING IS INCOMPUTABLE (According to this paper). Example, beauty is incomputable Step 1. Assume there is an algorithm Beauty(R,D) that given the program R and the input D, will return True if it is beautiful and False otherwise. Step 2. Cre…

[deleted]

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#59

Post by EVa5I7bHFq9mnYK » Mon, Jan 11, 2021, 10:24 PM UTC

inglor_cz wrote: post #37

EVa5I7bHFq9mnYK wrote: post #31

It's kinda obvious: if you create a system so intelligent that it can create itself, and you impose controls over it, it will be able to create a version of itself without the controls.

Well, you can, for example, try to limit its total energy budget. That is physical limitation, that is hard to circumvent. Of course, it is possible that said superintelligence develops an ingenious new source of energy as a response.

It will earn money by solving captchas and order a lot of Amazon Essentials batteries for itself :)

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#60

Post by jacb » Mon, Jan 11, 2021, 10:25 PM UTC

cambalache wrote: post #53

I confess I am a frustrated academic, but from time to time I read a paper like this one and and convince myself that my frustration may perhaps be unwarranted. RECIPE TO PROVE ANYTHING IS INCOMPUTABLE (According to this paper). Example, beauty is incomputable Step 1. Assume there is an algorithm Beauty(R,D) that given the program R and the input D, will return True if it is beautiful and False otherwise. Step 2. Cre…

That's not how arguments from contradiction work. You've assumed H (halting problem is decidable), prove B (beauty decidable), then say "but actually !H, therefore !B". All you've proven is that H => B. An actual argument from contradiction is "assume H, using H we can prove an obviously false statement, therefore not-H". And indeed, you have assumed something false (H), and shown a contradiction (various proofs for !H), so your original assumption was wrong (!H).

A concrete example: say Beauty(R,D) is trivially decidable (say it returns (popcount(R)+popcount(D))%2. Your argument "proves" this property is undecidable, but it's definitely decidable. So some step in your argument is wrong.

Another concrete example: Assume the halting problem is decidable. I would like a cookie. But the halting problem is not decidable. Therefore I would not like a cookie.