The Dawn of AI: How to ensure the promise outweighs the perils • Hacker News Board

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#41

Post by coldtea » Thu, May 07, 2015, 4:00 PM UTC

return0 wrote: post #24

coldtea wrote: post #10

Earlier quoted context omitted.

> Now we are scared of something we haven't even created I don't see the contradiction or strangeness in that. That's the whole idea behind "caution" -- to be wary of the potential outcome BEFORE it happens. > How could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it For one we are quite good consumers it. How about they get rid of us, so they…

> Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc. I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the…

>I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design.

The problem with this is that for real AI, you don't have control over what they feel. It's not like some program of the sixties or seventies AI, were stuff was "simulated", it can be modelling a real neural network that works like the brain, and into which you don't have much control over what will develop.

That is, the kind of AI people are afraid of is not "simulated", it's actually thinking for itself.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#42

Post by DanAndersen » Thu, May 07, 2015, 4:00 PM UTC

return0 wrote: post #33

SCHiM wrote: post #29

Earlier quoted context omitted.

It 'wants' what we programmed it to want. It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do. The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the…

if (nearby() == human) {dontTouch(); abort() ; } There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?

How do we define nearby? Do we mean physical contact? Does the fact that the atoms of objects don't "actually" touch each other due to repulsive forces come into play?

Considering that an AI system would be software and wouldn't be constrained to a particular robot body, what does nearby even mean?

Is it physical proximity? Is indirectly touching a human with a ten-foot pole allowed? What about manipulating the global economic system such that the programmer who could shut you down loses his job?

We can't just separate the whole system from the world, because the AI was presumably built with a practical purpose beyond "sit in a box and don't touch humans." Social engineering is a danger; imagine having it send an email to a lowly programmer, pretending to be an important executive, requesting that they remove this one pesky line of code from the AI's programming because all this "proximity alert" stuff is interfering with its ability to actually accomplish what we invested time/money to get it to do.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#43

Post by TeMPOraL » Thu, May 07, 2015, 4:00 PM UTC

coldtea wrote: post #37

TeMPOraL wrote: post #18

Earlier quoted context omitted.

I didn't mean intelligence as moral characteristic - the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things. As alien minds, they're not very smart - just smart enough to be dangerous.

> the observation is that many bureaucracies tend to do a lot of dumb, self-defeating things Isn't that true for people too, that can otherwise be extremely intelligent? From being terrible at social life, to sabotaging their careers and so on?

Yeah, I suppose it is.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#44

Post by sixQuarks » Thu, May 07, 2015, 4:02 PM UTC

Let's not kid ourselves, there is simply no way to ensure the promise outweighs the perils. I'm talking about superintelligence here, not artificial general intelligence, although it's almost certain superintelligence will arrive shortly after AGI.

Anyone who is not scared shitless of what's to come simply does not truly understand what AI is capable of. If you're one of those people, do not compare AI to ANYTHING that you are familiar with today in regards to computers/software.

Superintelligent AI will be like a god that knows your inner thoughts, desires, hopes and dreams as well as nearly everyone else on earth. It will manipulate people with ease - it's not even an issue. Whatever it wants to accomplish, it will, with or without human approval.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#45

Post by robotresearcher » Thu, May 07, 2015, 4:02 PM UTC

jarek-foksa wrote: post #21

return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

If we managed to create an AI that gets out of control, we would pose an existential threat to it due to our ability to create another even more powerful and uncontrollable AI.

That's a good point. But the AI could do that too, and maybe better.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#46

Post by coldtea » Thu, May 07, 2015, 4:03 PM UTC

DanAndersen wrote: post #31

return0 wrote: post #24

Earlier quoted context omitted.

> Fifth, as advanced AI, it could develop actual feelings, including sadism, greed, etc. I get that some AIs will have simulated reward signals, but letting them go haywire would just be bad design. We have nuclear heads, but we do not design them to explode at random. Now, our own reward mechanisms arise from the grander plan to keep organic life alive. Theirs have no underlying motive, and i don't think it goes the…

I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess. But I feel like this particular argument is (please correct me if I'm wrong) saying something like "Well of course we wouldn't let them go crazy! That'd be ridiculous!" I agree -- a sensible solution would be to have good design and introduce safeguards. T…

>I think there's some over-anthropomorphization of the AI here, in terms of it picking up human qualities as if they were Things outside our brains that we possess.

I don't see any issue with anthropomorphization with regard to intelligent being (at any level).

To have an issue with that would be to assume that humans are more special than other life forms (and AI would be like a life form in this sense).

Animals can have empathy, anger, anxiety and other traits we associate with humans (and why wouldn't they? It's not like our brains are unique, they're just "more"), so AI could too.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#47

Post by 0359B02149AB » Thu, May 07, 2015, 4:05 PM UTC

return0 wrote: post #8

Now we are scared of something we haven't even created . We are far from real AI, but even so, how could the "interests" of a machine conflict with ours? I mean, they work with electricity, and we are hardly good sources for it. I cannot see how our future electric horses are going to be anything but useful to us. I 'd be more afraid of modified humans than computing machines.

Because it's easy to construct angels and demons in one's mind, about anything. It polarizes the mind into fear and desire, forcing forcing a person to continuously oscillate between extremes. It makes it seem like there are always only two choices, when in reality, there are many, if not infinite.

Also, there are always going to be some people who understand the machine better than the machine understands itself. Even if it means taking those gigantic tables of probabilities and abstracting over them to form some kind of representational language that can be used to predict complex non-deterministic machine behavior (similar to how we already try to predict non-deterministic behavior, or not) in complex systems.

As a corollary, I would argue that machines have already modified humans. They affect the way we think and reason. They are an integral part of our life and the machine abstraction can be projected onto that which it does not describe completely (in which, the technology that exists now can not construct the technology we envision). We program machines, machines program us. This seems obvious.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#48

Post by k__ » Thu, May 07, 2015, 4:16 PM UTC

maratd wrote: post #5

k__ wrote: post #4

Earlier quoted context omitted.

Totally. I often have the impression that super human entities just need us like we need cells. The market can't survive without humans, but it doesn't care about a few of them dying.

There is no market. No corporations, no governments, nothing. It's just people. The uncomfortable truth is that as humans, we don't really care if a few of us die off as long as we don't know them and they're not in our vicinity. Which is why markets, governments, corporations, etc. act the way they do. Sure, if you ask a person, they'll weave a heartfelt story, etc. but the truth is in what they do, not in what they…

The theory about capitalism-critique has stopped to blame people for these problems long time ago.

It's about the systems and how they force people to do things they normally wouldn't do.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#49

Post by caligastia » Thu, May 07, 2015, 4:24 PM UTC

jerf wrote: post #6

iamcurious wrote: post #2

humans have been creating autonomous entities with superhuman capacities and unaligned interests for some time. Government bureaucracies, markets and armies: all can do things which unaided, unorganised humans cannot. All need autonomy to function, all can take on life of their own and all can do great harm if not set up in a just manner and governed by laws and regulations. This. Superhuman intelligences already exi…

These superhuman intelligences are indeed literally superhuman, though... they consist of humans. As much as some fellow commenters are calling them "alien" thinkers, their alienness is still profoundly human . Many of these "superhumans" can still be surprisingly accurately characterized with the human characteristics we are used to. Generally there's some small set of humans at the top of the hierarchy that are set…

Very good point, too many people consider AI to be a completely distinct entity that will appear fully formed in a lab one day. The reality is more fluid. Kevin Kelly calls it the Technium, as he sees it, it's a completely new kingdom that is being co-created - initially by man, and increasingly by itself.

This new intelligence is indeed a hybrid - it is not Twitter, it is millions of people and machines tweeting, it is not an iPhone, it is millions of people and software interacting with it and through it, it is not Google, it is billions of people exchanging information with it and through it.

To understand where this is going, we must ask, like KK does, what does technology want? Certain things are obvious - silicon and electrons are its flesh and blood, so the activity and interest of this new life form will be centered around semiconductor manufacturing and sources of electricity. How does a machine make sure it will always be fed? It must provide something useful to man so that it keeps getting plugged in and recharged.

So what is useful to man? At this moment in history the largest use of computation appears to be communication, and the machines have made themselves indispensable in this area - but at a cost. The machines demand our attention in exchange for revealing the messages we crave.

Stephenson uses the swordplay terms 'Actor' and 'Patient' to describe the initiator and the subject of an attack. Witness a person's reaction to their iPhone beeping with an incoming message - between man and machine, who is the Actor in this interaction? We jump at the sound and interrupt our work in order to service the machine, which then gives us our reward, our tweet, and in gratitude we place it back in its charger.

Digital viruses are where we can see the forefront of a new relationship emerging - a digital virus is more intelligent and more efficient than its host, the Operating System, so we have a very straightforward Darwinian process playing out between an 'old' species, the OS, and a new, small, intelligent interloper - its electricity is paid for by its host, another machine, therefore the virus does not have the host's social contract with the person who owns the computer - it has a different calculus for its survival, it owes only its creator, the virus writer.

Like the coming 'water wars', the new digital wars will be rooted in a struggle for limited resources such as human attention, electricity, and habitable areas of silicon.

Re: The Dawn of AI: How to ensure the promise outweighs the perils

#50

Post by DanAndersen » Thu, May 07, 2015, 4:29 PM UTC

return0 wrote: post #30

bwood wrote: post #22

Earlier quoted context omitted.

If the AI doesn't have its own intention (perhaps having intention is a prerequisite for being truly intelligent), it's conceivable that we would program it with an intention or goal, such "find the most efficient way to produce as many paperclips as possible". If you assigned such a task to a human, it's fairly obvious that the task is subject to other things which are important to humans (for example, don't enslave…

yeah but that's bad design or terrorism, there is always another person to blame here. I think we are talking here about machines that take the blame themselves.

I'm not sure what the purpose is of focusing on blame. You don't need to assign moral culpability to a machine for its catastrophically poor design to be something to worry about.