Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Dawn of AI: How to ensure the promise outweighs the perils

economist.com

Post reply on HN
Print view View original

11–20 of 75 posts

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#11

Post by FeepingCreature »

maratd wrote: post #5
k__ wrote: post #4

Earlier quoted context omitted.

Totally. I often have the impression that super human entities just need us like we need cells. The market can't survive without humans, but it doesn't care about a few of them dying.

There is no market. No corporations, no governments, nothing. It's just people. The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do. Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they…

There is no people. No dreams, no hope, no love. It's just cells.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#12

Post by loup-vaillant »

return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else.

---

If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally unstoppable, then (iii) accomplish whatever it was programmed to accomplished.

If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…

If the goal is to answer some difficult math question, it could tile the planet with computer, destroying the ecosystem and all humans in it, so it can compute the answer, and display it on a screen, for nobody to see it. Or maybe you programmed it to tile the universe with paperclips, because that goal is easy to debug. Oops.

---

As long as we're limiting ourselves to narrow AI (ordinary programs, really), we should be okay, and most of the consequences should be manageable. But as soon as we get to genuine AGI, we must worry about intelligence explosion and value alignment, so we can avoid creating a god that want something we don't.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#13

Post by coldtea »

rndn wrote: post #7
iamcurious wrote: post #2

humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations. This. Superhuman intelligences already exi…

I hate this argument just as much as I hate the one-word phrase "This". It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans. What people are actually concerned about are superintelligences which operate in completely alien ways and outperform even cooperating humans.

>It's ignorant of the entire literature on superintelligence because it assumes that superintelligences will behave like cooperating humans.

That's at best a peripheral detail to the parallel the parent tries to draw.

Real AI might as well behave in any odd way -- the real point is that something like "more powerful than human intelligence" already exists, in the form of these aggregate entities.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#14

Post by return0 »

bwood wrote: post #9
return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer

i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#15

Post by coldtea »

TeMPOraL wrote: post #3
iamcurious wrote: post #2

humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations. This. Superhuman intelligences already exi…

I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.

>their intelligence is usually not the identifying characteristic

Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#16

Post by TeMPOraL »

maratd wrote: post #5
k__ wrote: post #4

Earlier quoted context omitted.

Totally. I often have the impression that super human entities just need us like we need cells. The market can't survive without humans, but it doesn't care about a few of them dying.

There is no market. No corporations, no governments, nothing. It's just people. The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do. Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they…

> Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they say ... and the majority of us do nothing or worse.

I'd say conclusion is the opposite - there's little "free will", people generally are slaves to the incentive structures they live in. People follow incentives so predictably that at scale there's little sense in talking about individuals - it's exactly corporations, governments and markets. Behaviour of system as a whole dominates over behaviour of individual components.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#17

Post by giltleaf »

>"Crucially, this capacity is narrow and specific. Today’s AI produces the semblance of intelligence through brute number-crunching force, without any great interest in approximating how minds equip humans with autonomy, interests and desires."

I'm not sure why we would want AI to have autonomy, interest, and desire? It's almost like solving a problem that doesn't exist.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#18

Post by TeMPOraL »

coldtea wrote: post #15
TeMPOraL wrote: post #3

Earlier quoted context omitted.

I wouldn't neccessary call all of them "superhuman", their intelligence is usually not the identifying characteristic. But they're definitely alien minds, pursuing their own alien values, thinking in an alien way.

> their intelligence is usually not the identifying characteristic Intelligence in this context in the sense of a means to perpetuate their existance and get more power, not as some moral characteristic. Besides they can employ tons of Nobel prize winners (e.g. Feynman and co in the Manhatan project), and build crazily-smart stuff (for their purposes).

I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#19

Post by jjaredsimpson »

loup-vaillant wrote: post #12
return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

The machine does not hate you, nor does it loves you. But you are made of atoms, that it could use for something else. --- If you create an AI with a genuine utility function. And that AI manage to be gazillion smarter than you are (think the difference between us and chimps, only much greater). Then it is likely the AI will (i) convince you to unleash it upon the internet, (ii) become even more powerful and totally…

>If the goal is to keep humans safe and happy, it might imprison everyone in playgrounds, and drug or lobotomise any unhappy people, or restraint any suicidal people. Or, if you specified "happiness" with pictures of smiling people, it might resort to facial reconstruction, or it might just tile the solar system with molecular smileys, because that maximises the "make people smile" goal much better than actually making people happy. And while we're at it, happiness isn't the only thing we care about…

I wish this would stop being repeated. The argument for a solar system tiling smiley face machine has been thoroughly refuted.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#20

Post by DanAndersen »

return0 wrote: post #14
bwood wrote: post #9

Earlier quoted context omitted.

It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer

i dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?

Don't think of "wanting" as a soul-like action of a little person in the machine. "Wanting" is a shorthand for anything that is the result of a goal system and autonomy. In the simplest sense, giving a drone a map of its surroundings and giving it a location as the goal for its path-finding optimization algorithms, the drone now "wants" to go to that location.
Post reply on HN