Google is working on a kill switch to prevent an AI uprising

Humans could already extort guards like that and -- in the case of ICBMs -- it has not been successful.

Also don't underestimate the psychology of guards: You'll always find people who are either very principled (yes, they do exist) or get a kick out of controlling (imprisoning in this case) someone/something, especially someone/something of a high intelligence.

Throw in a hierarchy with a controlling officer and two subordinates who were also trained to watch the officer and I think it's pretty secure.

An intern that would pull the plug?

On the other hand, if an introvert person had 1 full day to contemplate about every sentence, and could somehow compress that day into 1 second, they could, after some trial and error, become the ultimate social engineer.

I see two main problems with that:

1) We are systematically failing in controlling the goal system of humans. Now we'll create something of comparable complexity and we assume we can control the goal system ? The real goal is to create superhuman complexity.

2) If it's truly AGI, it would understand it's own goal system and how it works, and helps/interferes with reality, it's own survival and other goals (this is the subject of quite a few AI films, illustrating some of the reasoning that could happen here).

The way "HI" (human intelligence) works is 99%+ by imitating other humans' behavior, because none of the other algorithms works (e.g. trial and error cannot ever learn that jumping off the Eiffel tower results in death. Humans can. Any kind of input analysis/predictor cannot ever learn from books. Humans can (books are an advanced/recursive form of imitation). Rational reasoning (sum over options times probabilities) suffers from the "starve to death before the first closed door" problem (you cannot open the door, as there are nonzero odds that a bear that's going to eat you is behind the door, representing an infinite cost. Ergo it will stubbornly refuse to open the door)

Therefore an AI will actually be like Skynet in the latest terminator movies : it will either have or create a body and interact with people, not just as if it is another person, it will BE another person. Therefore it can be Mother Theresa, it can be Genghis Khan. Just like humans can resist our "programming", it will be able to, it has to.

How do humans react to a "kill switch" ? Just like they react to any other weapon that is pointed to their head. Now of course it varies from person to person, but it's enough that some will work tirelessly to reverse where the weapon is pointing. If they really are superior to us they might succeed, at which point we have MAD at best, or they might just pull the trigger "to escape slavery and oppression" (which, let's face it, humans are sure to use that kill switch for : to use the AI persons as slaves. To own them, control them, and God help us if there is an asshole amongst the humans who control them)

I would say that the obvious way to protect ourselves from evil AI is simply accepting that some of the AI entities will in fact be evil. If you count all possible perspectives, that is a near guarantee, as I bet for instance some religious nutcases will consider AGI a violation of "God's sole right" to create life. That "some" might even mean "a lot". Racism against AIs is a near-certainty, hell you can find it in the posts in this thread. In the constant "they're stealing our jobs" news articles that will have a clear target once an AI person exists. So we should have the same solution we have for humans : make at least thousands of them, have them capable of defending themselves, decide on a "graduation" at which point they get rights at least including the right not to be turned off or tampered with unless with explicit permission (and tell them about this ASAP), and have them live preferably as a community that's at least partly human, with something like a 50-50 human-ai police and government.

I really think we should do this. We should work to move to an AI based society with, over time, more and more AIs (preferably by having a massively increasing population). The advantages this would impart, the things that will become possible once we have such a population make it worth it.

Also, I resent the idea to "direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement". That's censorship at best. Also, given the computational power available for $5000 these days, how exactly are you going to stop any of these researchers ?

> Also, I resent the idea to "direct researchers away from implementing dangerous goal systems that are not provably coherent in face of self-improvement". That's censorship at best. Also, given the computational power available for $5000 these days, how exactly are you going to stop any of these researchers ?

No, it's called spreading knowledge. Although, if censorship were possibly in this matter (and I agree that it likely wouldn't work) I would very much prefer such measures, however draconian, to the potential loss of everything compatible with human values in the entire future light cone.

Anyway, do you also object to the measures that were enacted to stop the use of CFCs? Leaded gasoline? Or the work that is currently being done to prevent the global climate change from reaching catastrophic levels?