Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The impossibility of intelligence explosion

medium.com

Post reply on HN
Print view View original

61–70 of 143 posts

Re: The impossibility of intelligence explosion

#61

Post by Smaug123 »

noam87 wrote: post #50

This article has too many holes to count and reads more like someone in denial. --- BUT, as an aside: I think a decent argument exists for the non-certainty of intelligence explosion. The argument goes like this: it takes an intelligence of level X to engineer an intelligence of level X+1. First, it may well be that humans are not an intelligence of level X, and reach our limit before we engineer an intelligence supe…

I think everyone agrees about "non-certainty". Where people disagree is on how likely an intelligence explosion is; and in particular, whether it is likely enough to warrant expending effort to plan for it.

Re: The impossibility of intelligence explosion

#62

Post by Smaug123 »

nabla9 wrote: post #46

Very good piece from actual researcher in the field. I can see practical intelligence explosion when visuospatial intelligence develops and can be connected to rudimentary reasoning. Most of human intelligence seems to be bootstrapped from our ability to comprehend and visualize 3d space and objects moving there. It's also interesting how almost all problems look like boxes and arrows or connecting lines when you dra…

Some people (~3% of the population) completely lack the ability to visualise. http://slatestarcodex.com/2014/03/17/what-universal-human-ex...

Re: The impossibility of intelligence explosion

#63

Post by Retric »

tigershark wrote: post #39

I'm not sure that I ever read a piece written with so much certainty and arrogance on a field that is completely unexplored. Just as an example: "The basic premise of intelligence explosion — that a “seed AI” will arise, with greater-than-human problem solving ability, leading to a sudden, recursive, runaway intelligence improvement loop — is false." From the little that we know as of today I call bullshit. Even Alph…

Games have a clear ends goal. How to you measure getting better at ethics?

Re: The impossibility of intelligence explosion

#64

Post by manux »

10-6 wrote: post #53
manux wrote: post #48

Earlier quoted context omitted.

> It can understand "the given problem" but there is FAR MORE to our minds than optimizing and learning how to solve a task. Humans are bound by the size of their brains. Machines can "duplicate themselves", in a way, it can freeze its Go-playing part and learn to do something else to perfection. Humans can't do that. If pro Go players stop playing Go and decide to master sword-making, they will forget about Go. A ma…

"it can freeze its Go-playing part and learn to do something else to perfection" People like Elon Musk will write sentences like this but hidden away in it is a subtle reference to an AI program understanding what it's doing. An AI program cannot jump from mastery of Go (with its neural networks and weighted matrices) to all of a sudden decide to figure out what the best course of action is to stop global warming or…

I believe that we are not close to "general AI", at all.

Yet, that is a very different position than believing that "general AI" is not possible at all, which I think is a foolish position considering highly adaptative beings exist "biomechanically" in the form of humans.

Re: The impossibility of intelligence explosion

#65

Post by naasking »

Retric wrote: post #63
tigershark wrote: post #39

I'm not sure that I ever read a piece written with so much certainty and arrogance on a field that is completely unexplored. Just as an example: "The basic premise of intelligence explosion — that a “seed AI” will arise, with greater-than-human problem solving ability, leading to a sudden, recursive, runaway intelligence improvement loop — is false." From the little that we know as of today I call bullshit. Even Alph…

Games have a clear ends goal. How to you measure getting better at ethics?

Even without assuming some value system, we can measure progress in ethics by accumulating a growing set of necessarily false moral propositions. For instance, propositions which necessarily contradict themselves.

Re: The impossibility of intelligence explosion

#66

Post by [deleted] »

naasking wrote: post #65
Retric wrote: post #63

Earlier quoted context omitted.

Games have a clear ends goal. How to you measure getting better at ethics?

Even without assuming some value system, we can measure progress in ethics by accumulating a growing set of necessarily false moral propositions. For instance, propositions which necessarily contradict themselves.

[deleted]

Re: The impossibility of intelligence explosion

#67

Post by rihegher »

"A person with an IQ of 130 is statistically far more likely to succeed in navigating the problem of life than a person with an IQ of 70 — although this is never guaranteed at the individual level — but the correlation breaks down after a certain point. There is no evidence that a person with an IQ of 200 is in any way more likely to achieve a greater impact in their field than a person with an IQ of 130. How comes?"

Actually there is as well no evidence that a person with an IQ of 130 is in any way more likely to achieve a greater impact in their field than a person with an IQ of 100. And that is probably why the majority of humans have an IQ close to 100 while people with an IQ of 130 or more are less than 5%.

That said the proportion of people with IQ greater than 130 is growing slowly, that may means we are already seing an intelligence growth with mankind.

Re: The impossibility of intelligence explosion

#68

Post by drngdds »

10-6 wrote: post #11

The author makes several points about AI/intelligence and recursive systems that I think a lot of people who think "AI will take over the world and replace" humans [1][2][3] don't understand. He argues that general intelligence by itself is not really something that exists, our brains exist within a broader system (environment, culture, bodies, etc.) which is something you can read more about in embodied cognition: h…

>[AI] completely replacing humans and potentially destroying us all is a joke and only the people who no nothing about AI/brains think this.

This isn't true at all.

http://slatestarcodex.com/2015/05/22/ai-researchers-on-ai-ri...

Re: The impossibility of intelligence explosion

#69

Post by erasemus »

mannykannot wrote: post #49

This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that it structures the argument in a way designed to avoid discussion of the issue. It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? It's specialized for "being human". The autho…

>It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence?

I think there are two usages of the term 'general intelligence' floating around:

(1) the ability that humans possess (but which animals don't) to create universal theories,

(2) the measure of one human's general cognitive ability or potential (in all fields) relative to another human's.

Note that IQ tests are concerned with (2). The quest for AGI is concerned with (1), though the additional prediction of intelligence explosion or singularity assumes the validity of (2).

I think the author would claim that (1) exists but (2) doesn't. He explains the predictive power of IQ tests by claiming that general intelligence is a threshold ability and that people who score highly on an (arbitrary) test are more likely to have exceeded that threshold. Beyond the threshold, achievement is limited only by other factors.

Re: The impossibility of intelligence explosion

#70

Post by AnimalMuppet »

azakai wrote: post #55

> Intelligence expansion can only come from a co-evolution of the mind, its sensorimotor modalities, and its environment. This is misleading. It's true in a sense - the environment does matter - but artificial intelligence can create artificial environments in which to learn, and simulate them faster than humanly possible. Those environments could be evolved together with the intelligence. So there is still the possi…

You're assuming that IQ will scale linearly with processing power. So far as I know, there is no reason to suppose that is true.
Post reply on HN