Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The impossibility of intelligence explosion

medium.com

Post reply on HN
Print view View original

101–110 of 143 posts

Re: The impossibility of intelligence explosion

#101

Post by jessaustin »

btilly wrote: post #42
pbkhrv wrote: post #40

Earlier quoted context omitted.

Wait, so if I run a Go-playing program from 10 years ago on the AlphaGo cluster then it'll produce better results than it did 10 years ago?

Yes. Nowhere near as good as AlphaGo, but yes it would do better. When Deep Blue beat Kasparov at chess, the program was not significantly better than what had been state of the art for the previous decade. They just threw enough hardware at it. For chess programs there is an almost linear relationship between your search depth and effective ELO rating, and search depth went up by a constant with each generation of M…

For chess programs there is an almost linear relationship between your search depth and effective ELO rating...

Maybe that's why chess has been "solved" by AI, and as of yet no real problems that trouble humanity have?

Re: The impossibility of intelligence explosion

#102

Post by naasking »

spuz wrote: post #98
naasking wrote: post #94

Earlier quoted context omitted.

> Automation of jobs and tasks is very real, but completely replacing humans and potentially destroying us all is a joke and only the people who no nothing about AI/brains think this. Don't be absurd. Brains are simply biochemical computers. The idea of electrical or other types of computers superceding us is perfectly realizable. The only real debate is over the timeline.

It's not absurd to believe that humans will never be able to invent machines that match human level intelligence. Firstly, it's not necessarily logical to assume that brains are simply computers because we've never created or even imagined a computer that works anything like a brain. Even if it's possible, it might be beyond human intelligence to invent such a computer. I like to think of AGI like the Warp Drive. It…

> It's not absurd to believe that humans will never be able to invent machines that match human level intelligence

It actually is absurd and it's easy to see why: nature evolved a series of progressively more intelligent biochemical computers by accident, simply by trial and error.

If that's all we ever did with computers, like enumerating all Turing machines, we would eventually find one more intelligent than us. The fact that we can do even better than simple trial and error, for instance, by building machines that help us build machines, we drastically accelerate that inevitability.

Re: The impossibility of intelligence explosion

#103

Post by naasking »

amag wrote: post #99
naasking wrote: post #94

Earlier quoted context omitted.

> Automation of jobs and tasks is very real, but completely replacing humans and potentially destroying us all is a joke and only the people who no nothing about AI/brains think this. Don't be absurd. Brains are simply biochemical computers. The idea of electrical or other types of computers superceding us is perfectly realizable. The only real debate is over the timeline.

Superseding us in what way? The machine is already better at chess and Go, clearly not at classifying images[1], but the issue here is that the author argues that there is no such thing as a general intelligence. It's all situational. On a separate note, does it even matter? Some people argue it's important so that we can put AI safe-guards in place, to protect mankind. But either: A) Intelligence explosion won't hap…

A superintelligence can't circumvent mathematical proof, so your scenarios are not exhaustive and safeguards are not intrinsically useless.

> but the issue here is that the author argues that there is no such thing as a general intelligence. It's all situational

An intelligence that can adapt to changing situations seems pretty general to me.

Re: The impossibility of intelligence explosion

#104

Post by YeGoblynQueenne »

mannykannot wrote: post #49

This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that it structures the argument in a way designed to avoid discussion of the issue. It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? It's specialized for "being human". The autho…

>> A lot of column-inches are expended on repeatedly restating that animal intelligences have co-evolved with their sensorimotor systems, which is a contingent fact of history, not a fundamental necessity for intelligence (as far as we know;

How do we know that? We only know of intelligences that co-evolved with their sensoriotor etc, so how do we know that's not the only way to do it?

Re: The impossibility of intelligence explosion

#105

Post by YeGoblynQueenne »

tigershark wrote: post #39

I'm not sure that I ever read a piece written with so much certainty and arrogance on a field that is completely unexplored. Just as an example: "The basic premise of intelligence explosion — that a “seed AI” will arise, with greater-than-human problem solving ability, leading to a sudden, recursive, runaway intelligence improvement loop — is false." From the little that we know as of today I call bullshit. Even Alph…

Alpha Go is good at playing Go. It can't do anything else. That's the point the author is making at the start of the article, that intelligence develops by focusing on specific tasks.

There's a lot that is hardly substantiated in the OP, but the truth is that just because you have a machine that's smart enough to play Go better than any human being, doesn't mean you can anticipate a machine that can learn to play the bassoon better than any human being.

The argument about the no free lunch theorem is informative and one of the few good points in the article. An algorithm that is good at Χ is eventually going to be pretty bad at Y. A superintelligence would have to beat humans in all possible X, even the ones it would be really bad at. And that sounds like an impossibility.

Re: The impossibility of intelligence explosion

#106

Post by YeGoblynQueenne »

SilasX wrote: post #25
titzer wrote: post #14

> If intelligence is fundamentally linked to specific sensorimotor modalities, a specific environment, a specific upbringing, and a specific problem to solve, then you cannot hope to arbitrarily increase the intelligence of an agent merely by tuning its brain — no more than you can increase the throughput of a factory line by speeding up the conveyor belt. The latter part makes no sense at all. Of course you can incr…

Also, intelligence includes the ability to identify other bottlenecks and remove them. Of course you can't expect that that a faster conveyor belt, by itself, can yield ever-greater output, but you can expect further improvements if the factory is feeding engineers who are working on ways to further re-design the factory, and they are currently constrained by the factory's output. The whole reason that some people pr…

>> The whole reason that some people predict an intelligence explosion is because intelligence is the resource that can find arbitrary ways to self-improve, other than tuning one specific parameter.

But not even a super-intelligence can find ways to self-improve that don't exist. We don't know what we don't know and we don't know if it's possible for intelligence to improve indefinitely. If there is a hard limit to the amount of improvement an intelligence can acquire, then superintelligence is not going to happen.

As an analogy, think of the speed of light. No amount of technology will get you past that. You might find a way around it for the purposes of interestellar travel but nothing will ever move faster than light.

How can we know that there don't exist similar impassable barriers for the development of intelligence? Maybe it is an -yet undiscoered- law of intelligence that an intelligent species cannot create an artificial intelligence more intelligent than itself. Who knows?

The problem is that we can sit around thinking of possibilities and impossibilities for ever, but the fact of the matter is that we don't, currently, have any evidence to the point that super-intelligence is possible. We don't have any evidence to the contrary either- but the discussions of superintelligence start with people assuming it is possible and that is an assumption that must first be substantiated (but rarely is).

Re: The impossibility of intelligence explosion

#107

Post by ddxxdd »

Can anyone attest to the validity of the No Free Lunch theorem?

What if I created an algorithm that had access to every other possible algorithm ever created, and chose to use the optimal algorithm for the given task? Would that algorithm-seeking algorithm do better at every task than random chance? Would that algorithm-seeking algorithm break the No Free Lunch theorem?

Re: The impossibility of intelligence explosion

#108

Post by spuz »

naasking wrote: post #102
spuz wrote: post #98

Earlier quoted context omitted.

It's not absurd to believe that humans will never be able to invent machines that match human level intelligence. Firstly, it's not necessarily logical to assume that brains are simply computers because we've never created or even imagined a computer that works anything like a brain. Even if it's possible, it might be beyond human intelligence to invent such a computer. I like to think of AGI like the Warp Drive. It…

> It's not absurd to believe that humans will never be able to invent machines that match human level intelligence It actually is absurd and it's easy to see why: nature evolved a series of progressively more intelligent biochemical computers by accident , simply by trial and error. If that's all we ever did with computers, like enumerating all Turing machines, we would eventually find one more intelligent than us. T…

You are still making the assumption that the brain is a computer. Is there any theoretical proof that our brains are a permutation of a Turing machine? That to me would seem to be the first step towards your conclusion.

Re: The impossibility of intelligence explosion

#109

Post by SilasX »

YeGoblynQueenne wrote: post #106
SilasX wrote: post #25

Earlier quoted context omitted.

Also, intelligence includes the ability to identify other bottlenecks and remove them. Of course you can't expect that that a faster conveyor belt, by itself, can yield ever-greater output, but you can expect further improvements if the factory is feeding engineers who are working on ways to further re-design the factory, and they are currently constrained by the factory's output. The whole reason that some people pr…

>> The whole reason that some people predict an intelligence explosion is because intelligence is the resource that can find arbitrary ways to self-improve, other than tuning one specific parameter. But not even a super-intelligence can find ways to self-improve that don't exist. We don't know what we don't know and we don't know if it's possible for intelligence to improve indefinitely. If there is a hard limit to t…

Those are all good points, and good reasons to be skeptical of the Intelligence Explosion narrative, but the author wasn't basing his objection on those reasons.

Re: The impossibility of intelligence explosion

#110

Post by Veedrac »

Bartweiss wrote: post #92
akvadrako wrote: post #76

Earlier quoted context omitted.

> I've seen lots of essays (not this one) claim that morality will "inherently" emerge from intelligence, which I think is absurd. Shit, my moral views aren't an 'inherent' product of anything except my evolution-shaped brain that feels empathy. You're evolution-shaped brain is what an AI will become too. It won't start out super-human smart; it will evolve to an equal level. Then keep going. Morals are basically low…

...huh? AI will reach parity with humans before it exceeds them, agreed. But intelligence parity is not the same as "matching my evolution-shaped brain". Feeling empathy for other humans is not a requirement for intelligence; I routinely make supposedly-irrational choices to help humans in non-reciprocal settings because I feel empathy. There's no reason to think an AI that can write code as well as me will feel the…

> AI will reach parity with humans before it exceeds them, agreed.

Intelligence is a hugely dimensional space, and AI constraints look very little like those on humans. It is unlikely there will ever be a time when AI is similarly smart, as much as differently smart to a similar context-dependent magnitude.

Post reply on HN