Live data from Hacker News

A Reply to François Chollet on Intelligence Explosion

intelligence.org

31–40 of 89 posts

Re: A Reply to François Chollet on Intelligence Explosion

#31
These kinds of discussions aren't very rigorous and reveal a basic philosophical illiteracy and philistinism at work. There's quite a bit of question begging going on. The elephant in the room is that the prevailing materialistic/naturalistic (MN) understanding of the world is completely impotent where intentionality, qualia, consciousness, etc, are concerned. Philosophers like Thomas Nagel talk about it; the incorrigible Dennetts of the world prefer to shutter the windows and live in their intellectual safe spaces. To say that the notion that computers are intelligent is problematic is putting it very lightly.

MN is rooted in the expulsion of the mind from the reality under consideration, in the process sweeping many things under the “subjective” rug that don’t fit the methodologies used to investigate reality. But now, when the mind itself becomes the object of explanation, when someone remembers that minds are, after all, part of reality, it is no longer possible to play the game of deference and one must deal with all of those things we’ve been exiling to the “domain of the subjective”. MN is wholly impotent here, by definition. Qualia? Forget it. That’s why MN tends to collapse into either some form of dualism or eliminativism, the latter of which is a non-starter, the former of which has its own problems.

And yet, despite the terminal philosophical crisis MN finds itself in, the chattering priesthood of Silicon Valley remains blissfully unaware, hoping to conjure up some fantastical reality through handwaving.

Re: A Reply to François Chollet on Intelligence Explosion

#32

These kinds of discussions aren't very rigorous and reveal a basic philosophical illiteracy and philistinism at work. There's quite a bit of question begging going on. The elephant in the room is that the prevailing materialistic/naturalistic (MN) understanding of the world is completely impotent where intentionality, qualia, consciousness, etc, are concerned. Philosophers like Thomas Nagel talk about it; the incorri…

I haven't seen anyone argue that intentionality, qualia, or consciousness would necessarily be either a precondition or a result of developing AGI. In fact, thought experiments like the "paperclip maximizer" are often brought up to argue that a machine could be very alien in its internal experience or lack thereof, but still pose an existential threat.

Re: A Reply to François Chollet on Intelligence Explosion

#33
I can't remember where, but I saw a fairly damning argument against the intelligence explosion hypothesis on the grounds that it only works if the algorithm used to design a mind with n units of intelligence (whatever these are) scales linearly with units of intelligence. If it scales faster than linear, then your recursive bootstrapping operation takes longer and longer each time, so that eventually your next bootstrapping step will take longer than the amount of time left in the universe, meaning there is some finite intelligence cap for any such bootstrapped mind. It seems quite implausible to me that the problem of designing a mind would scale linearly, given that ostensibly much simpler problems, like sorting a list of strings, require polynomial time or log-linear time algorithms.

Re: A Reply to François Chollet on Intelligence Explosion

#34

These kinds of discussions aren't very rigorous and reveal a basic philosophical illiteracy and philistinism at work. There's quite a bit of question begging going on. The elephant in the room is that the prevailing materialistic/naturalistic (MN) understanding of the world is completely impotent where intentionality, qualia, consciousness, etc, are concerned. Philosophers like Thomas Nagel talk about it; the incorri…

None of the gameplaying AIs that have shown superhuman performance recently have needed to be conscious or aware of what they were doing in order to perform at that level. They likely don't experience qualia but they can still trounce human players. Similarly, Google's amazing improvements in machine translation haven't required consciousness or qualia.

There are certainly lots of people in Silicon Valley who are fascinated by the prospect of making machines with conscious experience, as you think is impossible, but few indications that this would be necessary to make non-conscious forms of AI into a factor that powerfully affects our future.

Re: A Reply to François Chollet on Intelligence Explosion

#35

I can't remember where, but I saw a fairly damning argument against the intelligence explosion hypothesis on the grounds that it only works if the algorithm used to design a mind with n units of intelligence (whatever these are) scales linearly with units of intelligence. If it scales faster than linear, then your recursive bootstrapping operation takes longer and longer each time, so that eventually your next bootst…

>meaning there is some finite intelligence cap for any such bootstrapped mind

This does not preclude an intelligence explosion, this cap could be many (say, 100) times higher than human intelligence. We could still see many features of an explosion in that case.

Re: A Reply to François Chollet on Intelligence Explosion

#36

Lots of points can be reduced to the ability to simulate a physical environment very fast, and much faster than the actual occuring of events. But it doesn't look like it's easy to simulate physics at high speeds, let alone faster than what happens in our environment. Therefore we are bound by our environment and our limited ability to simulate it.

This is probably a good criticism if it turns out that the right level of abstraction for most problems is Physics. And it seems like your argument would apply equally well against the idea of human intelligence. Luckily, our minds have developed other abstractions that allow us to solve problems much faster than if we had to simulate them as physics problems. For example, I don't need a physics-level simulation of my friend's brain when I want to predict how they'll react to a gift I'm giving them.

Re: A Reply to François Chollet on Intelligence Explosion

#37

I can't remember where, but I saw a fairly damning argument against the intelligence explosion hypothesis on the grounds that it only works if the algorithm used to design a mind with n units of intelligence (whatever these are) scales linearly with units of intelligence. If it scales faster than linear, then your recursive bootstrapping operation takes longer and longer each time, so that eventually your next bootst…

We should be mainly concerned with the unit of capability rather than of intelligence (for which there is no widely accepted standard measurement for non-human beings [1]) As we know, there are tasks that a less intelligent being can never accomplish no matter how much time and other resources it has.

If we use brain size as a rough proxy, ours is only three times as large as a chimp's but our capabilities for creation and destruction are vastly greater both in degree and range.

[1] IQ indicates the location in the distribution of intelligence within human population and it is flawed in many ways. The concept does not really apply to other beings.

Re: A Reply to François Chollet on Intelligence Explosion

#38
post #23

Earlier quoted context omitted.

sin(x) = x for very small x, this is a useful tool for several proofs.

for all functions, f(x) = f(0) + f'(0) * x for x very close to 0

yeah I didn't know that part, just the more specific one.

Re: A Reply to François Chollet on Intelligence Explosion

#39

I can't remember where, but I saw a fairly damning argument against the intelligence explosion hypothesis on the grounds that it only works if the algorithm used to design a mind with n units of intelligence (whatever these are) scales linearly with units of intelligence. If it scales faster than linear, then your recursive bootstrapping operation takes longer and longer each time, so that eventually your next bootst…

>meaning there is some finite intelligence cap for any such bootstrapped mind This does not preclude an intelligence explosion, this cap could be many (say, 100) times higher than human intelligence. We could still see many features of an explosion in that case.

True, but I think that the results of such a "weak intelligence explosion" (where the linear/sublinear scaling case would be a "strong intelligence explosion"), while still remarkable, would fall far short of some of the expectations placed on general AI by the singularity / superintelligence crowd. For the sake of argument, extrapolating from our current energy consumption using a (simplistic) linear model and some rough back of the envelope calculations, the intelligence required to harness the total energy output of the sun would be 40 trillion times the aggregate intelligence of the entire human species today. Several hundred times just isn't going to cut it. Now, if our ability to harness energy increases exponentially with intelligence, then maybe it could work, but that's just an assumption, and, given that there are hard physical limits on the efficiency of energy generation due to thermodynamics, seems very unlikely.

Re: A Reply to François Chollet on Intelligence Explosion

#40

I can't remember where, but I saw a fairly damning argument against the intelligence explosion hypothesis on the grounds that it only works if the algorithm used to design a mind with n units of intelligence (whatever these are) scales linearly with units of intelligence. If it scales faster than linear, then your recursive bootstrapping operation takes longer and longer each time, so that eventually your next bootst…

We should be mainly concerned with the unit of capability rather than of intelligence (for which there is no widely accepted standard measurement for non-human beings [1]) As we know, there are tasks that a less intelligent being can never accomplish no matter how much time and other resources it has. If we use brain size as a rough proxy, ours is only three times as large as a chimp's but our capabilities for creati…

An analogous argument would hold in the case of "units of capability". Why do you think the problem of producing capable minds is any easier than the problem of producing intelligent minds? I'm not sure what the point of your chimp analogy is, unless you think I think that brain size is the unit of intelligence. Anyway, I'm also skeptical of the idea that there are such things as general "units of intelligence", but the entire notion of a superintelligence is predicated on the idea that intelligence is a general quality and can be meaningfully quantified. It's not clear to me how we'd even define superintelligence without reference to some quantitative measure of intelligence, although if you'd like to try, I'd be interested.
Post reply on HN