In my opinion, all the talks about the potential danger of advanced AI is highly speculative and revolves around a very simple thing: fear of the unknown, that's all. We simply don't know. And some people are also afraid of creation by accident, because intelligence is seen as an emergent property of complex networks, but again, this is because we don't understand much about it. Tldr; Nothing to see here, move along.
Superintelligence cannot be contained: Lessons from Computability Theory
101–110 of 227 posts
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#102Arguments that claim something is impossible from an argument related to the halting problem are generally bogus. The halting problem applies only to deterministic systems with infinite state. If you have finite state and determinism, you have to eventually repeat a state, or halt. Note that "very large" and "infinite" are not the same thing here. Not being able to predict something for combinatorial or noise reasons…
Different for pure mathematics sure, but is that of practical importance given how fast busy-beaver numbers grow?
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#103Earlier quoted context omitted.
You need to start by proving that brains and all idealized computers we can build are inherently on different levels. Intuitively it seems like intuition is just a probabilistic analysis on incomplete data. For example that shadow is "probably" a predator that may eat me or that sound is probably a prey animal I can kill and eat. Current AI which probably doesn't fit either of our definitions of intelligence can desi…
> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer... Not commenting on what OP said, but I don't think this is correct. Even in principle, how can any computational process produce conscious experiences, which are by nature subjective and unquantifiable?
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#104Sorry for the snark, but Douglas Adams also demonstrated this: the earth as a super-intelligent computing device ended up getting a piece of itself onto a passing space ship, avoiding complete destruction. I just like the idea of thinking about all of earth, including what we'd consider having or not having life, as a single super intelligence. Of course you could scale up to include the solar system, galaxy or even…
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#105Earlier quoted context omitted.
Different for pure mathematics sure, but is that of practical importance given how fast busy-beaver numbers grow?
Busy-beaver only grows fast given unlimited memory. On a finite machine it’s limited by the total number of states the device has. Granted that can be 10^1,000,000,000,000+ these days, but it’s still limited so BB(10^10) on actual hardware = BB(10^100).
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#106Currently humans are super intelligent compared to machine intelligence, so if the super intelligence can give rise to something more intelligent than it, could the super intelligence give rise to something more intelligent than it? The answer must be yes, then the question is if containment is the problem and the conclusion is that it cannot be contained, then what we should be making right now is a super intelligen…
Even if the hypotheses hold, a sequence x, f(x), f(f(x)), ... doesn't have to diverge even with f(x)>x for all x. A chain of super intelligences could feasibly have a small upper bound not much greater than that for a single person. That idea even lines up with some crude experimental data -- vast additional resources barely make a dent in the frontier for chess, image recognition, a breadth of graph problems, etc...…
It seems similarly ridiculous to imagine that the human brain just happens to be close to the maximum computational power because it was what we suppose to be the most powerful instrument produced thus far on our planet.
Wouldn't it be equal parts unlikely and strange if the most effective and compact process that can be produced in any conditions just happened to be the one that random evolution happened on in a random walk through what is practical with earths organic chemistry.
Even if we imagine the incredibly unlikely idea that you can't make a more effective mind you would think you could run the same process faster than real time or more instances of it for increasingly small resources on a better computer.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#107Re: Superintelligence cannot be contained: Lessons from Computability Theory
#108Earlier quoted context omitted.
> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer... Not commenting on what OP said, but I don't think this is correct. Even in principle, how can any computational process produce conscious experiences, which are by nature subjective and unquantifiable?
Calling consciousness “subjective” feels kind of like calling water “wet”. But anyways, life itself is built on computational processes. Most of these processes are “purpose-built” to accomplish very specific tasks, but in humans the development of a massive cortex created something of a general-purpose computer. Consciousness seems to be the necessary compromise to run such an embodied computer on top of all the oth…
The Wikipedia definition of "computation" is "any type of calculation that includes both arithmetical and non-arithmetical steps and which follows a well-defined model". But this only makes sense in the context of a designer or observer external to the computation who can identify what that model is and thereby make sense of the output. So you can't say that brain processes are computational, much less life itself, without committing some variation of the homunculus fallacy.
John Searle (famously known for his Chinese Room thought experiment) made this argument in a paper called "Is the Brain a Digital Computer?" [1] He points out that "if we are to suppose that the brain is a digital computer, we are still faced with the question 'And who is the user?'"
A related problem is qualia. There is no computational process that will produce the sensations of colour or sound or touch. At best you will have some representation that requires having actually experienced those sensations to understand it. So a computational process cannot be the basis of or an explanation for those sensations, and therefore consciousness generally.
[1] https://philosophy.as.uky.edu/sites/default/files/Is%20the%2...
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#109Earlier quoted context omitted.
what you need to understand about the halting problem is that at is core it is an epistemological problem. An other expresion of it are Gödel's incompleteness theorems. Imagine a blank sheet of paper the area of the paper is what is knowable now start building logic as a data structure. We start with the first nodes which are the axioms now everything derived conects to other nodes etc as it expands as mold its going…
You need to start by proving that brains and all idealized computers we can build are inherently on different levels. Intuitively it seems like intuition is just a probabilistic analysis on incomplete data. For example that shadow is "probably" a predator that may eat me or that sound is probably a prey animal I can kill and eat. Current AI which probably doesn't fit either of our definitions of intelligence can desi…
We don't need to know. It suffices to expand "reason" to include not just inference, but properly those things which inference depends on, namely, conceptualization+abstraction and judgement. You could never infer that "all triangles have angles adding to 180 degrees" unless you were in possession of the signified concepts. And here is where the temptation to frame brains in terms of computers becomes plainly, let's say, problematic. Concepts like "triangularity" or "the color green" are abstract in that they do not exist as concrete things in their own right. You can have concrete green triangular objects in the world, but not triangularity or green as such...except in a mind that is able to abstract universals from particulars. So images (including the vulgarized CS sense) won't help you because they're always concrete impressions (one green triangle at the expense of all other possible green triangles). Formal structures expressed in what amounts to some notation won't help you either because they only get their meaning from an interpreting mind. Often, it seems attributions of intelligence to computers stem from a tacit failure to remove ourselves from the picture, projecting our capacities onto a hunk of cleverly arranged metal.
> If silicon chips prove an insufficient mechanism there is no particular reason we couldn't somehow use a biological substrate or indeed even one we haven't thought of.
Computation is a formalism independent of substrate. You can implement a Turing machine using anything that can maintain the appropriate interpretive correspondence. But even then, a computer is not an objective kind of thing like a tree. Unlike a tree, a computer is only a computer because the observer has chosen to interpret it as such, or to use it as such. (The naive materialists in the room might try to expand that claim to encompass anything, but in the very act they've made things worse. The mind's capacity to comprehend the world is severely crippled if not destroyed, and now you have to explain this mind that somehow can entertain the existence of wondrous things like trees and elephants that the rest of the universe is unable to evince, thus making the mind even more mysterious than it was before the reductionist took to his hatchet.) So if substrate makes a difference, then it's the substrate that matters, not some correspondence with a computational formalism.
> You need to start by proving that brains and all idealized computers we can build are inherently on different levels.
The burden of proof is on those claiming minds and computers are the same thing.
Re: Superintelligence cannot be contained: Lessons from Computability Theory
#110Earlier quoted context omitted.
You need to start by proving that brains and all idealized computers we can build are inherently on different levels. Intuitively it seems like intuition is just a probabilistic analysis on incomplete data. For example that shadow is "probably" a predator that may eat me or that sound is probably a prey animal I can kill and eat. Current AI which probably doesn't fit either of our definitions of intelligence can desi…
> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer We don't need to know. It suffices to expand "reason" to include not just inference, but properly those things which inference depends on, namely, conceptualization+abstraction and judgement. You could never infer that "all triangles have angles adding to 180 degrees" unless…