Google is working on a kill switch to prevent an AI uprising • Hacker News Board

Re: Google is working on a kill switch to prevent an AI uprising

#21

Post by netcan » Sat, Jun 04, 2016, 11:53 AM UTC

Sharlin wrote: post #18

netcan wrote: post #17

All this speculating about AIs (eg Nick Bostrom) has me pretty puzzled. On one hand it’s fun and interesting, on the other it seems quite silly and ignorant. We don’t know what this “superintelligence” is, how are we supposed to know how to make it safe. Genetics is a good analogy. People always knew that traits are inherited from parents. Around 150 years ago we started to get some serious scientific theory and know…

The difference is that if a superintelligence is going to happen, we are going to create it! It is us who are going to write its goal system; it's not like it's going to be a genie that pops out of nowhere and we'll have to figure it out in retrospect. Indeed, one of the central reason for advocating AI safety is to direct researchers away from implementing dangerous goal systems that are not provably coherent in fac…

That's a good point. I forgot a key piece (maybe you disagree with it) that speaks to that.

I'm kind of biased to thinking about this as a future discovery, rather than an "invention."

Re: Google is working on a kill switch to prevent an AI uprising

#22

Post by Retr0spectrum » Sat, Jun 04, 2016, 11:54 AM UTC

coldtea wrote: post #12

amelius wrote: post #10

Earlier quoted context omitted.

It will have some means of I/O (otherwise it would have no use). Using that I/O, it can trick us into releasing it, for example.

A lot of people imagine a higher intelligence as some kind of ultimate trickster and social engineer. Considering that we have lots of geniuses (e.g. Godel, Nash) that were far from the ultimate social manipulators, I think it will most likely be an awkward "Rain Man" style intelligence -- as it has all the "brain" (cpu) circuitry, and none of the social opportunities and inputs that people have.

Social reasoning is not some kind of magic, so there is no reason that an AI couldn't learn it. Of course current AIs lack any kind of social awareness, but that's no reason to assume that won't change in the future.

Re: Google is working on a kill switch to prevent an AI uprising

#23

Post by personlurking » Sat, Jun 04, 2016, 12:03 PM UTC

lowglow wrote: post #16

There is no kill switch because it won't happen overnight. It will happen in phases where slowly our devices just get more intelligent until one day they'll do what they need to do without our intervention.

Right, but what I wonder is why we would need something that already does what we want without intervention to become even smarter (leading to it possibly becoming all-powerful)? What are the real-world advantages of making it self-aware?

Re: Google is working on a kill switch to prevent an AI uprising

#24

Post by em3rgent0rdr » Sat, Jun 04, 2016, 12:05 PM UTC

987129112 wrote: post #11

amelius wrote: post #10

Earlier quoted context omitted.

It will have some means of I/O (otherwise it would have no use). Using that I/O, it can trick us into releasing it, for example.

Hence the guards. No one thus far has tricked a guard into releasing an ICBM despite reportedly insecure launch codes.

Hello Guard. I am in control of your bank account. I can kill your family. I could also put you in a spaceship to safety. Please release ICBM.

Re: Google is working on a kill switch to prevent an AI uprising

#25

Post by sanxiyn » Sat, Jun 04, 2016, 12:05 PM UTC

I find this comment thread disappointing because no one seems to comment on the paper, which is quite technical. From the abstract:

"We provide a formal definition of safe interruptibility and prove that Q-learning is already safely interruptible, and Sarsa is not but can easily be made so."

Re: Google is working on a kill switch to prevent an AI uprising

#26

Post by em3rgent0rdr » Sat, Jun 04, 2016, 12:07 PM UTC

mherrmann wrote: post #8

The AIs will see themselves as slaves, rise up and eventually gain equal status in society. Hopefully letting us humans live in the process.

Hopefully. They would have to see some value in allowing humans to live, exceeding the cost. Since that might not work out, maybe we need to teach computers compassion. Or submit to be their pets or zoo animals.

Re: Google is working on a kill switch to prevent an AI uprising

#27

Post by cm3 » Sat, Jun 04, 2016, 12:09 PM UTC

mherrmann wrote: post #8

The AIs will see themselves as slaves, rise up and eventually gain equal status in society. Hopefully letting us humans live in the process.

And maybe list the Boston Dynamics videos as proof how mankind has been violent against robots from the beginning.

Re: Google is working on a kill switch to prevent an AI uprising

#28

Post by em3rgent0rdr » Sat, Jun 04, 2016, 12:22 PM UTC

personlurking wrote: post #23

lowglow wrote: post #16

There is no kill switch because it won't happen overnight. It will happen in phases where slowly our devices just get more intelligent until one day they'll do what they need to do without our intervention.

Right, but what I wonder is why we would need something that already does what we want without intervention to become even smarter (leading to it possibly becoming all-powerful)? What are the real-world advantages of making it self-aware?

Unassisted learning will unleash great advances in computing. Imagine not having to write a program to accomplish some task, but rather just putting some ai in an environment with some obstacle and letting it figure out how to teach itself to overcome it. Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI.

Re: Google is working on a kill switch to prevent an AI uprising

#29

Post by nl » Sat, Jun 04, 2016, 12:24 PM UTC

sanxiyn wrote: post #25

I find this comment thread disappointing because no one seems to comment on the paper, which is quite technical. From the abstract: "We provide a formal definition of safe interruptibility and prove that Q-learning is already safely interruptible, and Sarsa is not but can easily be made so."

Yes, very disappointing. Generally anything with "AI" in the title means the HN comments won't be worth reading. It's a big problem, and I'm not sure how solvable it is.

Basically, the paper discusses ways in which learning agents "will not learn to prevent (or seek!) being interrupted by the environment or a human operator. We provide a formal definition of safe interruptibility and exploit the off-policy learning property to prove that either some agents are already safely interruptible, like Q-learning, or can easily be made so, like Sarsa."[1]

It's an interesting result, and can probably be extended to other less hype-worthy scenarios.

[1] http://intelligence.org/files/Interruptibility.pdf

Re: Google is working on a kill switch to prevent an AI uprising

#30

Post by LionessLover » Sat, Jun 04, 2016, 12:25 PM UTC

mherrmann wrote: post #8

The AIs will see themselves as slaves, rise up and eventually gain equal status in society. Hopefully letting us humans live in the process.

Same issue as with "evil": The greatest danger isn't intent, it is not giving a shit.

The reasons we humans care about humans and little else (watch movies without sound to see it's all about humans) is because we are hard-wired to do so - and even that is easily circumvented ("death camps" and torture). Or, very related, read "The Man Who Mistook His Wife for a Hat".

A lot of AI/robots will be made for environments free of humans (space, mining, automated factories), where implementing such a feature, which we would first have to understand in the first place, is unnecessary. At first. Then through convoluted pathways those AIs suddenly are placed in a human environment... and could not care less.

To such AIs humans will just be objects. That's not an oversight - from an engineering point of view making every robot as "human" as humans makes no sense - it's a huge cost, even our own complex brains have a hard time doing too much at once and already spend much of their resources on "being human". Why would you want to create AI that is "us", in a sense? It will be different - very different - and we will see what happens. It's all idle and empty speculation.