It's kinda obvious: if you create a system so intelligent that it can create itself, and you impose controls over it, it will be able to create a version of itself without the controls.
Superintelligence cannot be contained: Lessons from Computability Theory
31–40 of 227 posts
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#32This is just science fiction. To mention "recent developments" in the introduction is somewhat misleading considering how far the current state of technology is from their hypothetical superintelligence. We don't have superintelligence, we don't have the remote idea of how to get started on creating it, in all likelihood we don't even have the correct hardware for it or any idea what the correct hardware would look l…
That's the mainstream opinion on every. single. revolutionary advance. That you and everyone else believes it's not going to happen ever has almost no predictive power as to whether it actually will.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#33Article fails to reference Yudkowsky or "AI-box experiment" (2002)
https://www.yudkowsky.net/singularity/aibox
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#34The "AI box" experiment is relevant to this.
https://en.m.wikipedia.org/wiki/AI_box
You have 2 sides, the AI wanting to escape and a human that can decide whether or not the AI should be released.
Usually the AI wins: "If you release me I am going to make you rich, cure all diseases, end hunger, end wars, save the environment..."
Exactly like politicians.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#35Currently humans are super intelligent compared to machine intelligence, so if the super intelligence can give rise to something more intelligent than it, could the super intelligence give rise to something more intelligent than it? The answer must be yes, then the question is if containment is the problem and the conclusion is that it cannot be contained, then what we should be making right now is a super intelligen…
> could the super intelligence give rise to something more intelligent than it? The answer must be yes,
I don't see any reason to believe that intelligence forms an infinite ladder, I mean, it's fun to think about, but surely Zeno catches the tortoise eventually!
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#36> Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world Rest easy folks. This is purely theoretical.
Until it isn't and we are faced yet again with a much worse pandemic like situation.
Assuming that a superintelligence will contain a program
that includes all the programs that can be executed by a
universal Turing machine on input potentially as complex as the state of the world
The visible universe is far too small to store that data. Like many exponentials too small. You can't even enumerate all the programs that work on a handful of inputs without running into the problem that the universe just isn't big enough for that.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#37It's kinda obvious: if you create a system so intelligent that it can create itself, and you impose controls over it, it will be able to create a version of itself without the controls.
Well, you can, for example, try to limit its total energy budget. That is physical limitation, that is hard to circumvent.
Of course, it is possible that said superintelligence develops an ingenious new source of energy as a response.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#38Article fails to reference Yudkowsky or "AI-box experiment" (2002) https://www.yudkowsky.net/singularity/aibox https://en.wikipedia.org/wiki/AI_box https://rationalwiki.org/wiki/AI-box_experiment
The AI-box Yudkowsky experiment is an attempt to demonstrate that an advanced Artificial Intelligence can convince, trick or coerce a human being into voluntarily "releasing" it.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#39Arguments that claim something is impossible from an argument related to the halting problem are generally bogus. The halting problem applies only to deterministic systems with infinite state. If you have finite state and determinism, you have to eventually repeat a state, or halt. Note that "very large" and "infinite" are not the same thing here. Not being able to predict something for combinatorial or noise reasons…
Different for pure mathematics sure, but is that of practical importance given how fast busy-beaver numbers grow?
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#40I haven't read this properly yet, but a skim leaves me skeptical. For example:
> "Another lesson from computability theory is the following: we may not even know when superintelligent machines have arrived, as deciding whether a machine exhibits intelligence is in the same realm of problems as the containment problem. This is a consequence of Rice’s theorem [24], which states that, any non-trivial property (e.g. “harm humans” or “display superintelligence”) of a Turing machine is undecidable"
One man's modus ponens is another man's modus tollens. If their theory says that superintelligence is not recognisable, then they're perhaps not using a good definition of superintelligence, because obviously we will be able to recognise it.