Is Intelligence Self-Limiting? • Hacker News Board

Re: Is Intelligence Self-Limiting?

#11

Post by iamgopal » Sat, Mar 10, 2012, 4:36 PM UTC

Swizec wrote: post #8

Valid argument, but we have a clear example that it is wrong. Humans. A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business? Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.

We are driven by multiple different value system. e.g. I am extremely curious to know if god's existence can or can not be proved scientifically. I can go to great length to know it, I do not give a damn about money. On the other hand, there are people who do not give damn about god and can go to great length for money. So Humanity can not go towards single goal. Some of them could go for shortcut and are going.

Re: Is Intelligence Self-Limiting?

#12

Post by zerostar07 » Sat, Mar 10, 2012, 4:41 PM UTC

Human civilization is currently limited to planet Earth with a few minor exceptions, so it makes sense to consider it one big system.

Actually it makes sense to consider all life as one big system. It was created by the planet itself, so, who knows we might even have to ascribe motivations to the planet. It's as if the planet (kind of like Lem's Solaris) has been brewing organisms for millions of years in order to do something with them . We might not be able to conceive these purposes with our antrhropomorphic thinking.

So, the human race is now coming to the point where it can modify and advance itself by tinkering with its own circuits. What we don't know is 1) what the planet plans to do with us and b) what are its movitivation and reward signals. It's not explained in the article how the AI knows what are the reward signals of its creator or why it would ever want to change them.

Re: Is Intelligence Self-Limiting?

#13

Post by rbarooah » Sat, Mar 10, 2012, 4:43 PM UTC

Swizec wrote: post #8

Valid argument, but we have a clear example that it is wrong. Humans. A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business? Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.

I'm not sure humans are a counterexample. Drugs don't generally relieve the need to work to buy food, and most serious drug addicts don't look as though they enjoying themselves.

Re: Is Intelligence Self-Limiting?

#14

Post by zerostar07 » Sat, Mar 10, 2012, 4:44 PM UTC

Swizec wrote: post #8

Valid argument, but we have a clear example that it is wrong. Humans. A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business? Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.

It might not have to do with "figuring it out", in the case of humans it's natural selection: individuals who choose to stimulate their reward centers only had much lower probability of reproducing. (like mice who stimulate their nucleus accumbens till they starve to death)

Re: Is Intelligence Self-Limiting?

#15

Post by iamgopal » Sat, Mar 10, 2012, 4:48 PM UTC

goodside wrote: post #10

The fact the author can think of one particular way to design an AI that would result it in falling over and dying once it figures out how to modify itself does not imply that all possible intelligences would behave similarly. Somehow, humans are able to "want to be happy" while still refusing to take heroin or pain killers, and we have no reason yet to believe that AI is fundamentally unable to grasp this distinctio…

The overlooked point here is, We do not have any control over how we build next generation. We have a little control by teaching them. The example author gave assumes each generation having lots of control over the next generation, and hence figured out what bugs them and solved it. I do not think we know what bugs us. For some people other people having different religions bugs them, so they kill them and be happy. For some people unknown unified theory bugs them, they will not be happy till it found. So It is not possible to find universal solution to different problem. But It could happen that people with similar problem join a group and find a shortcut way of being happy, without the problem being solved.

Re: Is Intelligence Self-Limiting?

#16

Post by iamgopal » Sat, Mar 10, 2012, 4:51 PM UTC

robertskmiles wrote: post #4

carsongross wrote: post #2

Reality is self-limiting. There are no exponentials, only sigmoid curves. Unfortunately there is a lot of money to be made convincing people that an early stage sigmoid is actually an exponential.

Obviously intelligence tops out at the high end of a sigmoid curve - there's only so much matter and energy in the universe to convert to computronium - but that's no reason to suspect any value smaller than that as the limit. It must eventually level off, but the point at which it levels off may be hundreds of billions of times more intelligent than a human. So the fact that intelligence is limited, doesn't in any w…

I like it. Intelligence more defer at Hope, Anger and Desire than at Innovativeness.

Re: Is Intelligence Self-Limiting?

#17

Post by randallsquared » Sat, Mar 10, 2012, 4:54 PM UTC

cousin_it wrote: post #3

Not all possible AI designs are based on reinforcement learning or reward channels. You can code the AI with a formally specified goal instead. For example, "is there a proof of the Riemann hypothesis in ZFC shorter than a million symbols?" If such a goal requires converting the solar system into computronium, then the AI will do that. It won't settle for wireheading because wireheading doesn't give an answer to the…

In fact there's ample reason to think that humans are at the lowest threshold of intelligence that makes a technological civilization possible, because if we'd reached that threshold earlier in our evolution, we'd have created civilization then instead of now. There's probably plenty of room above us.

While I don't disagree that humans are essentially at the lowest level of intelligence that make civilization possible (else why would it have taken hundreds of thousands of years to get started?), this claim has no bearing on the claim that the upper limit of intelligence is immediately above human genius level. You seem to be assuming that there is necessarily a wide gap between the lowest civilization-producing level and the highest practical level, and that's not at all clear. Some (weak) evidence that we're already near the top can be found in the higher incidence of mental health issues among very intelligent humans: perhaps this is a result of a limit on complexity rather than merely a feature of human brains.

Re: Is Intelligence Self-Limiting?

#18

Post by jimrandomh » Sat, Mar 10, 2012, 5:14 PM UTC

This is called "wireheading", and while it's an interesting possibility, unfortunately what we know indicates that AIs won't do it. See Stephen Omohundro, "The Basic AI Drives" (http://selfawaresystems.files.wordpress.com/2008/01/ai_drive...). The basic argument is that a sufficiently intelligent mind will make a distinction between the world being a certain way, and it perceiving that the world is a certain way, and that its goals will be defined in terms of the former; and since misperceiving the world impairs its ability to influence it, it will protect its perceptions from corruption.

Re: Is Intelligence Self-Limiting?

#19

Post by Androsynth » Sat, Mar 10, 2012, 5:46 PM UTC

robertskmiles wrote: post #6

What's the purpose of this 'pleasure' construct in the AI's mind? If it's able to calculate the value of a utility function in order to set the 'pleasure' variable, and it bases its actions on the value of this 'pleasure' variable, why not just cut out the middle man and have it base its actions directly on the result of the utility function? The variable functionally buys you nothing, but introduces this problem tha…

From a systems perspective, a pleasure center in the 'brain' is much more scalable than hardwiring something to 'do more x'. These are semi-sentient beings that he is describing, not dumb robots.

Re: Is Intelligence Self-Limiting?

#20

Post by Androsynth » Sat, Mar 10, 2012, 5:53 PM UTC

Swizec wrote: post #8

Valid argument, but we have a clear example that it is wrong. Humans. A lot of what we do is driven by internal value calculations and pleasure centers, so why aren't we all simply taking drugs and avoiding all this messy "doing things" business? Point is, if humans figured out a way of avoiding purely pressing the right buttons to enjoy themselves and actually being useful, so too will smart robots.

Drugs, in their current incarnations, are simply not that powerful a motivator. If you were to point at a heroin junky or basehead and say 'that person is in ecstasy right now, do you want their life?', the majority of people would say no, because all they see is a dirty, homeless junky.