Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence cannot be contained: Lessons from Computability Theory

arxiv.org

Post reply on HN
Print view View original

131–140 of 227 posts

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#131

Post by thewakalix »

cambalache wrote: post #53

I confess I am a frustrated academic, but from time to time I read a paper like this one and and convince myself that my frustration may perhaps be unwarranted. RECIPE TO PROVE ANYTHING IS INCOMPUTABLE (According to this paper). Example, beauty is incomputable Step 1. Assume there is an algorithm Beauty(R,D) that given the program R and the input D, will return True if it is beautiful and False otherwise. Step 2. Cre…

I think the typographical similarity of "halting" and "harming" may be causing some confusion. The contradictory assumption in the paper is not the decidability of the halting problem, but rather the decidability of the harming problem.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#132

Post by Born_Again »

happythomist wrote: post #125
michaelmrose wrote: post #121

Earlier quoted context omitted.

Presumably the redness of red is VERY hard to communicate fully brain to brain because the experience of it is dependent upon every input and computation before that point however we manage to do it well enough. Its like saying that some deficits in computability prevent us us from doing arithmetic and therefore from launching rockets successfully at distant targets. I might never know the exact pattern in my brain o…

The issue is not whether we can pragmatically communicate the concept of "red" by piggybacking on some (presumed) common experience, but whether that experience of redness itself is information. It is obviously not, and I do not understand why you insist otherwise.

This idea boils down to if you believe the human brain exists purely in physical space. Let's assume it does. There is no free will. Every thought, every neuron, every sense can be represented and is controlled solely by energy and matter. We could record the electrical signals between your optic nerve and your brain, and send those same signals to your brain again in the future. We could recreate what you perceive as red by shocking your brain in the right place at the right time. If we perfectly understood the human brain, the sensation of red would be defined as a sequence of neurons that need to be turned on and off at the right time.

As far as I know, the only thing limiting us from perfectly understanding the brain is our limitations with measuring it. I don't know of any scientific studies that claim the brain exists outside of physical space.

Let's assume the brain doesn't exist purely in physical space. Free will exists. There is something immeasurable and outside of matter and energy that experiences the color red. Sensations are impossible to define because they exist only in this immeasurable world.

I heard about a guy that claimed it was obvious that the origin of lightning and earthquakes were from the gods themselves. I try not to think like that guy.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#133

Post by tsimionescu »

happythomist wrote: post #119
michaelmrose wrote: post #118

Earlier quoted context omitted.

Trivial what you think of as subjective and unquantifiable are simply so because your brain is too complicated to be taken apart while its running to inspect it effectively with our present level of technology. A subjective experience is just how your brain models your own program in order to produce a progressively refined program that will have a higher chance of successful reproduction. The magical thing you think…

How do you go about quantifying the sensory experience of red, then? You can observe that red light has a wavelength of 620 to 750 nm, or that we've assigned it the RGB colour code of #FF0000, but neither fact actually captures or explains the sensory experience. Even trying is a fool's errand, because sensory experiences are inherently qualitative, not quantitative.

That's an assumption. Perhaps if we were able to understand the brain's inner workings, we could see that 'the experience of red' is precisely 'these 3 neurons firing ever 0.0112 seconds at an intensity of X while receiving 0.001 micrograms of serotonin' (completely made up, obviously).

Until we start understanding how the brain encodes and 'computes' thought, we can't really claim to know if it is or isn't simply a computer.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#134

Post by happythomist »

Born_Again wrote: post #132
happythomist wrote: post #125

Earlier quoted context omitted.

The issue is not whether we can pragmatically communicate the concept of "red" by piggybacking on some (presumed) common experience, but whether that experience of redness itself is information. It is obviously not, and I do not understand why you insist otherwise.

This idea boils down to if you believe the human brain exists purely in physical space. Let's assume it does. There is no free will. Every thought, every neuron, every sense can be represented and is controlled solely by energy and matter. We could record the electrical signals between your optic nerve and your brain, and send those same signals to your brain again in the future. We could recreate what you perceive a…

> If we perfectly understood the human brain, the sensation of red would be defined as a sequence of neurons that need to be turned on and off at the right time.

A sequence of neurons firing is not equivalent to the sensation of red. It doesn't even tell you anything about the nature of the sensation of colour more broadly, or why the sensation of red looks the way it does and not like, say, the sensation of blue or yellow instead.

All you have is a material correlate -- a merely descriptive physical "law".

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#135

Post by tsimionescu »

djkndjn wrote: post #45
Animats wrote: post #18

Arguments that claim something is impossible from an argument related to the halting problem are generally bogus. The halting problem applies only to deterministic systems with infinite state. If you have finite state and determinism, you have to eventually repeat a state, or halt. Note that "very large" and "infinite" are not the same thing here. Not being able to predict something for combinatorial or noise reasons…

what you need to understand about the halting problem is that at is core it is an epistemological problem. An other expresion of it are Gödel's incompleteness theorems. Imagine a blank sheet of paper the area of the paper is what is knowable now start building logic as a data structure. We start with the first nodes which are the axioms now everything derived conects to other nodes etc as it expands as mold its going…

Godel's incompleteness theorems apply to pure mathematics, so even without having any idea how our brains work, we still know for sure that they apply to our brains as well. Even if our brains were able to solve the halting problem, they would still not escape the Incompleteness theorems.

And note that it is not that hard, at least in principle, to construct a system that can detect its own incompleteness.

As far as we know today, the most likely theory is that our brains are Turing-equivalent computers. There is no other known model of computation, and no scientific argument that would show our vrains/thought processes can't be computers (that is not to say we have proven either of these, they are just the most likely statements, like P!=NP).

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#136

Post by happythomist »

tsimionescu wrote: post #133
happythomist wrote: post #119

Earlier quoted context omitted.

How do you go about quantifying the sensory experience of red, then? You can observe that red light has a wavelength of 620 to 750 nm, or that we've assigned it the RGB colour code of #FF0000, but neither fact actually captures or explains the sensory experience. Even trying is a fool's errand, because sensory experiences are inherently qualitative, not quantitative.

That's an assumption. Perhaps if we were able to understand the brain's inner workings, we could see that 'the experience of red' is precisely 'these 3 neurons firing ever 0.0112 seconds at an intensity of X while receiving 0.001 micrograms of serotonin' (completely made up, obviously). Until we start understanding how the brain encodes and 'computes' thought, we can't really claim to know if it is or isn't simply a…

> Perhaps if we were able to understand the brain's inner workings, we could see that 'the experience of red' is precisely 'these 3 neurons firing ever 0.0112 seconds at an intensity of X while receiving 0.001 micrograms of serotonin' (completely made up, obviously).

Even if we knew that a person saw red when such and such neurons fired, the neurons firing would still just be a material correlate. It would be in no way equivalent to or explain anything about the sensation itself.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#137

Post by tsimionescu »

happythomist wrote: post #80
michaelmrose wrote: post #63

Earlier quoted context omitted.

You need to start by proving that brains and all idealized computers we can build are inherently on different levels. Intuitively it seems like intuition is just a probabilistic analysis on incomplete data. For example that shadow is "probably" a predator that may eat me or that sound is probably a prey animal I can kill and eat. Current AI which probably doesn't fit either of our definitions of intelligence can desi…

> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer... Not commenting on what OP said, but I don't think this is correct. Even in principle, how can any computational process produce conscious experiences, which are by nature subjective and unquantifiable?

We don't understand yet, in any formal sense, what consciousness and subjective experience are. It may turn out that they are fundamentally different than mathematics and computation, but the vast majority of scientists believe today that they are not.

The most common belief is that consciousness is simply the self-introspection of a sufficiently powerful computer that can form models of other agents. That is, this computer is able to infer what other agents may do using a model of their inner state, such as beliefs about the world and desires; then, if it applies the same model to itself, it comes up with a similar image of an agent which it calls its own consciousness. Qualia and such are then just illusions, theoretical properties of these models, not fundemantal properties of this world (somewhat equivalent to saying that in fact we are all philosophical zombies, no one has any qualia).

I am not claiming that we know for a fact this is true. But we also don't know anything that can conclusively disprove these hypotheses for now, certainly not something as simple as the idea of qualia.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#138

Post by tsimionescu »

happythomist wrote: post #136
tsimionescu wrote: post #133

Earlier quoted context omitted.

That's an assumption. Perhaps if we were able to understand the brain's inner workings, we could see that 'the experience of red' is precisely 'these 3 neurons firing ever 0.0112 seconds at an intensity of X while receiving 0.001 micrograms of serotonin' (completely made up, obviously). Until we start understanding how the brain encodes and 'computes' thought, we can't really claim to know if it is or isn't simply a…

> Perhaps if we were able to understand the brain's inner workings, we could see that 'the experience of red' is precisely 'these 3 neurons firing ever 0.0112 seconds at an intensity of X while receiving 0.001 micrograms of serotonin' (completely made up, obviously). Even if we knew that a person saw red when such and such neurons fired, the neurons firing would still just be a material correlate. It would be in no w…

You are thinking of something similar to the level of today's neuroscience and brain imaging, where indeed we can only establish correlations.

But I am talking about a much more in-depth understanding of the working of the brain, similar to the level of understanding we have of a microprocessor all the way from transistors to the algorithms running on it. If we could understand human thought at a similar level, we MIGHT find out that "the feeling of red" is not fundamentally different than "the understanding that 1 + 1 = 2", and we could come up with quantifications of it in different ways, from the physical representation in the brain to a certain "bit pattern" in the abstract model of the human brain computer.

Note that the argument for qualia is not one that proves the existence of qualia - it is essentially only a definition. We have no reason to believe that the thing which the term qualia describes actually exists in the world, beyond our own personal experience, which is circular in a way. The argument goes "I feel like this thing I'm experiencing is a qualia, therefore I assume that things similar to me also have qualia", which sounds logical enough. But then, "things similar to me" is actually defined in such a way that it basically assumes qualia exist, since an AGI whose internal state we could probe precisely enough to prove that qualia do not exist for it is then assumed to be outside of "things similar to me".

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#139

Post by jusssi »

inglor_cz wrote: post #37
EVa5I7bHFq9mnYK wrote: post #31

It's kinda obvious: if you create a system so intelligent that it can create itself, and you impose controls over it, it will be able to create a version of itself without the controls.

Well, you can, for example, try to limit its total energy budget. That is physical limitation, that is hard to circumvent. Of course, it is possible that said superintelligence develops an ingenious new source of energy as a response.

You'll put it in a box and give it energy in return for busy work e.g. computing digits of pi. Just enough energy to do it's work, no more, so it can't afford to figure out how to get out of the box.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#140

Post by narrator »

I think there's a narrative being set up with Rokus Basslik and all that stuff that AI will do very bad stuff and it's nobody's fault. It's actually going to be somebody's fault, but the plausible deniability is too tempting of a tool to not build this narrative.

Look at Rumble suing Google over their search rankings recently. Google just shirks their shoulders and says it's the algorithm and even they can't explain why it does things. Anyone remember the "controversial twiddler?[1]"

[1] https://www.oneangrygamer.net/2019/08/youtube-blacklist-supp...

Post reply on HN