Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence cannot be contained: Lessons from Computability Theory

arxiv.org

Post reply on HN
Print view View original

121–130 of 227 posts

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#121

Post by michaelmrose »

happythomist wrote: post #117
astrophysician wrote: post #114

Earlier quoted context omitted.

> I don't think you can say that life is built on computational processes unless you use a definition of "computation" that is so vague and all-encompassing that it becomes effectively meaningless. I mean, there's nothing physically stopping us from simulating a brain, right? It's a finite object with a finite amount of physical ingredients, and therefore with a finite amount of computing power we can simulate what i…

I think you're missing the central point, which is that computation is observer relative. Anything can be interpreted as a computational process. Searle: "Thus for example the wall behind my back is right now implementing the Wordstar program, because there is some pattern of molecule movements which is isomorphic with the formal structure of Wordstar. But if the wall is implementing Wordstar then if it is a big enou…

Presumably the redness of red is VERY hard to communicate fully brain to brain because the experience of it is dependent upon every input and computation before that point however we manage to do it well enough.

Its like saying that some deficits in computability prevent us us from doing arithmetic and therefore from launching rockets successfully at distant targets.

I might never know the exact pattern in my brain of the "redness of red" as experienced by you but it seemed to work well enough for my brain to form a pattern similar enough to communicate thoughts just as the incompleteness or inherent lack of precision of measurement don't prevent the rocket from being launched successfully.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#122

Post by goatlover »

michaelmrose wrote: post #118
happythomist wrote: post #80

Earlier quoted context omitted.

> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer... Not commenting on what OP said, but I don't think this is correct. Even in principle, how can any computational process produce conscious experiences, which are by nature subjective and unquantifiable?

Trivial what you think of as subjective and unquantifiable are simply so because your brain is too complicated to be taken apart while its running to inspect it effectively with our present level of technology. A subjective experience is just how your brain models your own program in order to produce a progressively refined program that will have a higher chance of successful reproduction. The magical thing you think…

All you've done is replace subjective experience with talk of modeling one's own program. What you haven't done is shown how the two are in any way equivalent.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#123

Post by goatlover »

cma wrote: post #91
nmca wrote: post #49

Earlier quoted context omitted.

This is known as a "treacherous turn", a phenomenon I'm aware of. But I don't really see how that's relevant, my point is that a lack of physical grounding or pragmatism can lead to spurious conclusions about the superintelligence that humans will very likely build in the not too distant future. It will be smart, but contain no infinities.

> It will be smart, but contain no infinities. We think we can say that from physics, but we don't know what we don't know.

Computational limits are already known from physics. But before any device can get close to those limits, there are likely practical ones in terms of cost and energy to construct such devices.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#124

Post by dwaltrip »

bobthechef wrote: post #109
michaelmrose wrote: post #63

Earlier quoted context omitted.

You need to start by proving that brains and all idealized computers we can build are inherently on different levels. Intuitively it seems like intuition is just a probabilistic analysis on incomplete data. For example that shadow is "probably" a predator that may eat me or that sound is probably a prey animal I can kill and eat. Current AI which probably doesn't fit either of our definitions of intelligence can desi…

> It's not even clear exactly how our brains work so its hard to imagine that they couldn't be implemented with a sufficiently powerful computer We don't need to know. It suffices to expand "reason" to include not just inference, but properly those things which inference depends on, namely, conceptualization+abstraction and judgement. You could never infer that "all triangles have angles adding to 180 degrees" unless…

It seems you are asserting that no machine intelligence can possess or develop abstract representations, no matter how sophisticated it is. Can you explain your reasoning?

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#125

Post by happythomist »

michaelmrose wrote: post #121
happythomist wrote: post #117

Earlier quoted context omitted.

I think you're missing the central point, which is that computation is observer relative. Anything can be interpreted as a computational process. Searle: "Thus for example the wall behind my back is right now implementing the Wordstar program, because there is some pattern of molecule movements which is isomorphic with the formal structure of Wordstar. But if the wall is implementing Wordstar then if it is a big enou…

Presumably the redness of red is VERY hard to communicate fully brain to brain because the experience of it is dependent upon every input and computation before that point however we manage to do it well enough. Its like saying that some deficits in computability prevent us us from doing arithmetic and therefore from launching rockets successfully at distant targets. I might never know the exact pattern in my brain o…

The issue is not whether we can pragmatically communicate the concept of "red" by piggybacking on some (presumed) common experience, but whether that experience of redness itself is information. It is obviously not, and I do not understand why you insist otherwise.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#126

Post by michaelmrose »

happythomist wrote: post #125
michaelmrose wrote: post #121

Earlier quoted context omitted.

Presumably the redness of red is VERY hard to communicate fully brain to brain because the experience of it is dependent upon every input and computation before that point however we manage to do it well enough. Its like saying that some deficits in computability prevent us us from doing arithmetic and therefore from launching rockets successfully at distant targets. I might never know the exact pattern in my brain o…

The issue is not whether we can pragmatically communicate the concept of "red" by piggybacking on some (presumed) common experience, but whether that experience of redness itself is information. It is obviously not, and I do not understand why you insist otherwise.

Making up something you can't define and then describing the thing you invented as uniquely human is a bad argument.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#127

Post by drdeca »

happythomist wrote: post #117
astrophysician wrote: post #114

Earlier quoted context omitted.

> I don't think you can say that life is built on computational processes unless you use a definition of "computation" that is so vague and all-encompassing that it becomes effectively meaningless. I mean, there's nothing physically stopping us from simulating a brain, right? It's a finite object with a finite amount of physical ingredients, and therefore with a finite amount of computing power we can simulate what i…

I think you're missing the central point, which is that computation is observer relative. Anything can be interpreted as a computational process. Searle: "Thus for example the wall behind my back is right now implementing the Wordstar program, because there is some pattern of molecule movements which is isomorphic with the formal structure of Wordstar. But if the wall is implementing Wordstar then if it is a big enou…

Scott Aaronson has, iirc, suggested the idea that the complexity of such an isomorphism could be the distinguishing thing between whether or not something should be said to be computing a particular think. Sounds plausible to me.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#129

Post by cambalache »

gbjw wrote: post #98
cambalache wrote: post #74

Earlier quoted context omitted.

You are just repeating my critique. I am not the one making the mistake.

So you mean your frustration is warranted then?

If that is the kind of work published by academics associated with the MIT and La Pontificia Universidad de Madrid, then maybe I should not feel frustrated after all by not being an academic.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#130

Post by happythomist »

drdeca wrote: post #127
happythomist wrote: post #117

Earlier quoted context omitted.

I think you're missing the central point, which is that computation is observer relative. Anything can be interpreted as a computational process. Searle: "Thus for example the wall behind my back is right now implementing the Wordstar program, because there is some pattern of molecule movements which is isomorphic with the formal structure of Wordstar. But if the wall is implementing Wordstar then if it is a big enou…

Scott Aaronson has, iirc, suggested the idea that the complexity of such an isomorphism could be the distinguishing thing between whether or not something should be said to be computing a particular think. Sounds plausible to me.

Why should the complexity matter, so long as the computation is recognizable as such?
Post reply on HN