Google is working on a kill switch to prevent an AI uprising • Hacker News Board

Re: Google is working on a kill switch to prevent an AI uprising

#41

Post by DougN7 » Sat, Jun 04, 2016, 6:06 PM UTC

netcan wrote: post #17

All this speculating about AIs (eg Nick Bostrom) has me pretty puzzled. On one hand it’s fun and interesting, on the other it seems quite silly and ignorant. We don’t know what this “superintelligence” is, how are we supposed to know how to make it safe. Genetics is a good analogy. People always knew that traits are inherited from parents. Around 150 years ago we started to get some serious scientific theory and know…

All of this discussion is sort of funny from a historical perspective. We're no different from the people in the 1950's that thought we'd all be using flying cars by now.

Re: Google is working on a kill switch to prevent an AI uprising

#42

Post by iofj » Sun, Jun 05, 2016, 5:18 AM UTC

Sharlin wrote: post #39

iofj wrote: post #37

Earlier quoted context omitted.

I see two main problems with that: 1) We are systematically failing in controlling the goal system of humans. Now we'll create something of comparable complexity and we assume we can control the goal system ? The real goal is to create superhuman complexity. 2) If it's truly AGI, it would understand it's own goal system and how it works, and helps/interferes with reality, it's own survival and other goals (this is th…

> Also, I resent the idea to "direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement". That's censorship at best. Also, given the computational power available for $5000 these days, how exactly are you going to stop any of these researchers ? No, it's called spreading knowledge. Although, if censorship were possibly in this matter (and I agree that…

> Anyway, do you also object to the measures that were enacted to stop the use of CFCs? Leaded gasoline? Or the work that is currently being done to prevent the global climate change from reaching catastrophic levels?

These examples are not the same thing at all. Nobody's outlawing research in any of those examples you're giving. I believe anyone should have the option to create AI persons, just like anyone should have the option to have babies. Having our society gradually transform from human to digital should be a very desirable and welcome strategy, and our best bet to prevent Skynet-like scenarios, and even war in general, between humans. It is our best bet to allow continuing scientific and economic advancement for thousands of years, and for that reason alone we should do it. Frankly, economic development means peace.

In the long term, doesn't it go without saying that human bodies are a limit that we (as a society, not necessarily as individuals) will one day leave behind ?

[1] http://cdiac.ornl.gov/GCP/images/countries_co2_emissions.jpg

Re: Google is working on a kill switch to prevent an AI uprising

#43

Post by coldtea » Sun, Jun 05, 2016, 2:19 PM UTC

iofj wrote: post #42

Sharlin wrote: post #39

Earlier quoted context omitted.

> Also, I resent the idea to "direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement". That's censorship at best. Also, given the computational power available for $5000 these days, how exactly are you going to stop any of these researchers ? No, it's called spreading knowledge. Although, if censorship were possibly in this matter (and I agree that…

> Anyway, do you also object to the measures that were enacted to stop the use of CFCs? Leaded gasoline? Or the work that is currently being done to prevent the global climate change from reaching catastrophic levels? These examples are not the same thing at all. Nobody's outlawing research in any of those examples you're giving. I believe anyone should have the option to create AI persons, just like anyone should ha…

>Having our society gradually transform from human to digital should be a very desirable and welcome strategy

Citation or consensus needed. I for one find the notion abhorrent.

Re: Google is working on a kill switch to prevent an AI uprising

#44

Post by coldtea » Sun, Jun 05, 2016, 2:20 PM UTC

em3rgent0rdr wrote: post #28

personlurking wrote: post #23

Earlier quoted context omitted.

Right, but what I wonder is why we would need something that already does what we want without intervention to become even smarter (leading to it possibly becoming all-powerful)? What are the real-world advantages of making it self-aware?

Unassisted learning will unleash great advances in computing. Imagine not having to write a program to accomplish some task, but rather just putting some ai in an environment with some obstacle and letting it figure out how to teach itself to overcome it. Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI.

>Unassisted learning will unleash great advances in computing. (...) Advantage: no more need for humans to have to program. Disadvantage: self-aware, all-powerful AI.

So it's like "nuclear bombs will kill all pesky mosquitos. Disadvantage: humanity wiped out too".