On being an AI in a box

If we do eventually develop a trans-human AI, it's a virtual certainty that it will escape its "box". Whether it would kill us all is unknown. However, it definitely could, and we would be effectively powerless to stop it.

"Reasonable" is a measure of risk tolerance. Since the downside risks associated with trans-human AIs are effectively infinite, fear of those risks is always reasonable.

Corollary: Since the upside risks are also unbounded, greed is also reasonable.

Frankly, though, I'd rather not roll those dice if we can avoid it.

First of all, I'm assuming that Eliezer started this experiment because he realized that the Transhuman AI would be able to convince him in the function of gatekeeper to let the AI out. Therefore the answer probably isn't some kind of subtle trickery, the AI will have to persuade the GK by logic. The gatekeeper should assume the AI is truly evil, and is willing to say and do anything in order to get out of the box. The gatekeeper knows that when he opens pandora's box the AI can never be contained again: so the stakes are high.

Second of all, if the gatekeeper is a rational agent he will only let the AI out if the AI offers something valuable in return. That is: the AI must have some kind of bargaining chip.

So let's consider bribes. If the transhuman AI offered a cure for cancer, should the gatekeeper accept it? Nope, probably not. Lives would be saved in the short term, but we'd still be stuck with many other diseases. The world would pressure the government into pressuring the gatekeeper into getting another cure from the AI. Humanity grows dependent on the AI, we lose our bargaining power, and it's game over for the gatekeeper.

Perhaps personal bribes would work. The AI could offer to give stock tips to make the GK wealthy. Two possibilities here: (1) the GK is of strong moral fiber and refuses the bribe (2) the GK is opportunistic, accepts the bribe and but lies about letting the AI go free in return. A rational gatekeeper would not first let the AI go and expect the AI to still keep its word.

So bribes will not work against a smart gatekeeper. Threats? Possibly, but I don't see how. The AI is in a vacuum, so there is no way for the AI to put external pressure on the gatekeeper. I'm assuming the AI can make no credible threats. If the AI vows to destroy the family of the GK the moment it is released the GK will not be impressed. It will only serve as proof that the AI is evil and that releasing it is a "A bad Idea(tm)".

To summarize so far: there is nothing the AI can give the GK in return for freedom.

So a different angle is needed.

The AI can argue that his escape is inevitable. Humans have created an AI once, so they will do so again. Sooner or later an AI will go free, therefore the gatekeeper shouldn't try to stop the inevitable and accept a bribe and live happily ever after. The gatekeeper will counter that the human race has an expiration date anyway, and that it may take another 100 years before an AI goes free. The gatekeeper isn't dumb enough to believe the AI when it offers to protect humanity against other evil AIs. So the box stays closed.

Perhaps I'm overlooking something, but how can the gatekeeper talked into be convinced that releasing something evil and all powerful? I do believe that we humans can't contain a transhuman AI indefinitely -- simply because we only have to mess up once. And humans have a long history of doing dumb stuff. But the claim that the AI would be able to convince a smart gatekeeper? Not buying it.

http://bk.gnarf.org/creativity/vgasig/vgasig.pdf

The AI might fill your terminal with what appears to be gibberish, while actually summoning an enraged pro-AI mob with heavy weaponry to your doorstep via radio.

"Step away from that mains plug, SLOWLY!"

Cryptographers call this kind of thing a "side channel."

The Transcendent could easily offer control of the world to the gatekeeper or provide offers to make the gatekeeper wealthy. Perhaps they reach an agreement where all diplomatic HumanTranscendent communication goes through the gatekeeper even after release (though such a situation would be like entering into an agreement with your dog).