Live data from Hacker News

The impossibility of intelligence explosion

medium.com

141–143 of 143 posts

Re: The impossibility of intelligence explosion

#141
post #108

Earlier quoted context omitted.

You are still making the assumption that the brain is a computer. Is there any theoretical proof that our brains are a permutation of a Turing machine? That to me would seem to be the first step towards your conclusion.

The Bekenstein bound requires any finite volume to contain a finite amount of information, otherwise it would collapse into a black hole. Therefore our brains are, at best, finite state automata, and not even Turing complete.

Wat?

Have you just proven that computers don't exist? I'm suffering a bit of cognitive dissonance here...

Re: The impossibility of intelligence explosion

#142

This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that it structures the argument in a way designed to avoid discussion of the issue. It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? It's specialized for "being human". The autho…

> The author is apparently unaware that this tautological response eliminates the distinction between general and specialized intelligence, as one could just as validly (or vacuously) say that a superhuman intelligence is specialized in being what it is and doing what it does.

It's not a tautology. "Generality" and "specificity" are artifacts of the human experience. What is tautological is to say, "it's general to me, therefore it is general."

You think that more and more progress comes by way of more and more optimization. This is not the whole story. Accidentation is the missing ingredient. Humans - as well as all life on earth - have a knack for creating more and more problems. It is this never-ending fountain of new, accidental problems that allows for what appears to us to be a chain of "progress" stretching into the past.

Our "generality" is in fact a hairball collection of specific functions that have accreted into the human animal over millions and millions of years. Some abstract Java class called `Agent` with an `.optimize()` method hanging off of it simply does not have that context.

If you want a really high quality, generally intelligent function in silicon though, it's hard to beat the XOR function ;)

Re: The impossibility of intelligence explosion

#143

Earlier quoted context omitted.

Your reply gives me the impression that you are not up to date with Reinforcement Learning. If you did, you would know that the author really understands this domain and was not merely tautological. "Specialized at being human" - this is a deep intuition. We are reinforcement learning agents that are pre-programed with a certain number of reward responses. We learn from rewards to keep ourselves alive, to find food,…

What are humans specialized in doing? Because it seems to me that humans are pretty good at chess, calculus, social manipulation, flying to the moon, building machines that take us to the bottom of the ocean, discovering fundamental physics, etc. A fish, no matter what environment and upbringing you give it, can't do any of those things. So it seems like there's some dimension in which the human brain is more general…

That dimension is still on a thin film around a little ball floating in one of a great number of possible universes within a great number of possible rule systems. Compared to that space, we are quite similar to fish, in terms of the purposes for which our machinery functions.

But the question is not, "is intelligence explosion possible?" The question is, "explode into what?"

Post reply on HN