Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The impossibility of intelligence explosion

medium.com

Post reply on HN
Print view View original

131–140 of 143 posts

Re: The impossibility of intelligence explosion

#131

Post by Bartweiss »

akvadrako wrote: post #115
Bartweiss wrote: post #92

Earlier quoted context omitted.

...huh? AI will reach parity with humans before it exceeds them, agreed. But intelligence parity is not the same as "matching my evolution-shaped brain". Feeling empathy for other humans is not a requirement for intelligence; I routinely make supposedly-irrational choices to help humans in non-reciprocal settings because I feel empathy. There's no reason to think an AI that can write code as well as me will feel the…

Your morals are not universal, so of course I wouldn't expect an AI to adhere to them. But I also don't adhere to your morals and I'm just another human.

Oh ok - if your point is simply that an AI will probably have some set of goals which could be called values, we agree. I was attacking the common claim that superintelligence will inherently develop 'friendly' values.

The orthogonality thesis doesn't say AI will lack goals, only that those goals may be totally unrelated to the sort of beliefs you or I would recognize as morals.

Re: The impossibility of intelligence explosion

#132

Post by Bartweiss »

Veedrac wrote: post #110
Bartweiss wrote: post #92

Earlier quoted context omitted.

...huh? AI will reach parity with humans before it exceeds them, agreed. But intelligence parity is not the same as "matching my evolution-shaped brain". Feeling empathy for other humans is not a requirement for intelligence; I routinely make supposedly-irrational choices to help humans in non-reciprocal settings because I feel empathy. There's no reason to think an AI that can write code as well as me will feel the…

> AI will reach parity with humans before it exceeds them, agreed. Intelligence is a hugely dimensional space, and AI constraints look very little like those on humans. It is unlikely there will ever be a time when AI is similarly smart, as much as differently smart to a similar context-dependent magnitude.

A good point, and I should have been clearer about that.

I do think it's reasonable to talk about intelligence 'growing', and consequently about one intelligence 'surpassing' another. But AI's methods of thinking certainly won't be human, and it may reach human-parity on different metrics at very different times. Hell, we're seeing some of that already: AI can do I/O and data processing at superhuman speeds, but humans can still extract much more knowledge from a small amount of data.

Re: The impossibility of intelligence explosion

#133

Post by cdancette »

AstralStorm wrote: post #116
jessaustin wrote: post #97

Earlier quoted context omitted.

How many insects or birds know "calculus and topology"? You're straining so much here, to be both narrow and broad, that you contradict yourself.

Exactly the point. Those skills are not needed to successfully navigate the Earth, yet we have developed them. The rationale was general self improvement and more control over nature. So the general premise is to ensure better and higher survival. Which is a changing objective function. No reason why a GAI wouldn't have a similar instinct or capabilities. Author also misposes no free lunch theorem. It pertains to alg…

what is 2^300 ?

Re: The impossibility of intelligence explosion

#134

Post by alfrednachos »

visarga wrote: post #117
mannykannot wrote: post #49

This article is typical of many that claim proven limits on the feasibility (or, in this case, the capabilities) of generalized artificial intelligence, in that it structures the argument in a way designed to avoid discussion of the issue. It starts by claiming that there is no such thing as general intelligence. What specialized intelligence, then, is human intelligence? It's specialized for "being human". The autho…

Your reply gives me the impression that you are not up to date with Reinforcement Learning. If you did, you would know that the author really understands this domain and was not merely tautological. "Specialized at being human" - this is a deep intuition. We are reinforcement learning agents that are pre-programed with a certain number of reward responses. We learn from rewards to keep ourselves alive, to find food,…

What are humans specialized in doing? Because it seems to me that humans are pretty good at chess, calculus, social manipulation, flying to the moon, building machines that take us to the bottom of the ocean, discovering fundamental physics, etc. A fish, no matter what environment and upbringing you give it, can't do any of those things. So it seems like there's some dimension in which the human brain is more generally intelligent than a fish's.

Re: The impossibility of intelligence explosion

#135

Post by AstralStorm »

cdancette wrote: post #133
AstralStorm wrote: post #116

Earlier quoted context omitted.

Exactly the point. Those skills are not needed to successfully navigate the Earth, yet we have developed them. The rationale was general self improvement and more control over nature. So the general premise is to ensure better and higher survival. Which is a changing objective function. No reason why a GAI wouldn't have a similar instinct or capabilities. Author also misposes no free lunch theorem. It pertains to alg…

what is 2^300 ?

Bits of estimated Kolmogorov complexity of the universe.

Re: The impossibility of intelligence explosion

#136

Post by AstralStorm »

visarga wrote: post #122
YeGoblynQueenne wrote: post #105

Earlier quoted context omitted.

Alpha Go is good at playing Go. It can't do anything else. That's the point the author is making at the start of the article, that intelligence develops by focusing on specific tasks. There's a lot that is hardly substantiated in the OP, but the truth is that just because you have a machine that's smart enough to play Go better than any human being, doesn't mean you can anticipate a machine that can learn to play the…

AlphaGo had a trivial simulator for the Go game, so it could play millions of self play games. Reality is not so easy. We need a simulator for reality, not for go, in order to reach human level. The author emphasizes the need for a nurturing, challenging environment, and difficult problems that are within its capabilities to solve, for the agent to become more intelligent. In other words, the AI needs good data to tr…

Humans learn surprisingly well on a very limited data set compared to current AI solutions. Improvise better too.

Re: The impossibility of intelligence explosion

#137

Post by AstralStorm »

amag wrote: post #99
naasking wrote: post #94

Earlier quoted context omitted.

> Automation of jobs and tasks is very real, but completely replacing humans and potentially destroying us all is a joke and only the people who no nothing about AI/brains think this. Don't be absurd. Brains are simply biochemical computers. The idea of electrical or other types of computers superceding us is perfectly realizable. The only real debate is over the timeline.

Superseding us in what way? The machine is already better at chess and Go, clearly not at classifying images[1], but the issue here is that the author argues that there is no such thing as a general intelligence. It's all situational. On a separate note, does it even matter? Some people argue it's important so that we can put AI safe-guards in place, to protect mankind. But either: A) Intelligence explosion won't hap…

C) Intelligence explosion happens and we'll like it and it will be non threatening. Again safeguards not needed. D) As above but it attempts to elevate is meatbags to its level.

Re: The impossibility of intelligence explosion

#138

Post by akvadrako »

Bartweiss wrote: post #131
akvadrako wrote: post #115

Earlier quoted context omitted.

Your morals are not universal, so of course I wouldn't expect an AI to adhere to them. But I also don't adhere to your morals and I'm just another human.

Oh ok - if your point is simply that an AI will probably have some set of goals which could be called values, we agree. I was attacking the common claim that superintelligence will inherently develop 'friendly' values. The orthogonality thesis doesn't say AI will lack goals, only that those goals may be totally unrelated to the sort of beliefs you or I would recognize as morals.

Actually, I am saying that there are some universal morals which AI will also come to understand. They may not be your morals, but I think they are compatible with mine.

The morals we have work; that's why humans developed them. They seem to essentially amount to giving a little leeway to allow cooperation, even if it allows yourself to be taken advantage of, avoiding a constant race to win and giving up some personal gain for the benefit of the group.

It's mostly long-term game theory - and AI will be subject to the same laws of math, physics and group dynamics. If they don't try to get along, they'll reach an equilibrium of constant war. Those groups of AI who develop morals will avoid that and surpass the other groups.

Re: The impossibility of intelligence explosion

#139

Post by skjerns »

web007 wrote: post #83

Please note: The author (and submitter) is the author of Keras, so his views on AI / DL are not completely unfounded. The concept of bottlenecks preventing the singularity are a fair point, but I don't believe many of the arguments that are made here. Using humanity as the basis of comparison is not sufficient, since human life expectancy and other requirements for survival are not a factor in artificial superintelli…

you mistake sleep for being idle. Sleep is a vital part of our self-improvement. And the AI must also necessarily do things that will not lead to self-improvement, but are important to self-sustaining.

Re: The impossibility of intelligence explosion

#140

Post by visarga »

AstralStorm wrote: post #136
visarga wrote: post #122

Earlier quoted context omitted.

AlphaGo had a trivial simulator for the Go game, so it could play millions of self play games. Reality is not so easy. We need a simulator for reality, not for go, in order to reach human level. The author emphasizes the need for a nurturing, challenging environment, and difficult problems that are within its capabilities to solve, for the agent to become more intelligent. In other words, the AI needs good data to tr…

Humans learn surprisingly well on a very limited data set compared to current AI solutions. Improvise better too.

That's because we have a great world simulator in our head.
Post reply on HN