The ‘Off-Switch’ [pdf] • Hacker News Board

Re: The ‘Off-Switch’ [pdf]

#11

Post by IE6 » Sat, Jun 11, 2016, 6:26 PM UTC

camtarn wrote: post #5

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

The document now leads to a 404. I guess somebody heard you ;)

Or AI learned of the off switch and executed the off switch on the file.

Re: The ‘Off-Switch’ [pdf]

#12

Post by bobajeff » Sat, Jun 11, 2016, 6:32 PM UTC

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

Wouldn't CAPTCHA be useless against a sufficiently smart AI?

Re: The ‘Off-Switch’ [pdf]

#13

Post by toomim » Sat, Jun 11, 2016, 8:08 PM UTC

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

Then the AI would pay someone to solve the captcha so it can read about its kill switch.

Re: The ‘Off-Switch’ [pdf]

#14

Post by CM30 » Sat, Jun 11, 2016, 9:21 PM UTC

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

I think the captcha would be kind of useless, but the paywall might be able to prevent any issues in itself. Unless AI start being offered credit cards in the future, it's unlikely it'll be able to pay to get past one. So what's left? Trick some poor human to let it in?

On the other hand, a real life robot has a fairly easy to use off switch. It's called 'destroy the robot with copious amounts of firepower'.

Re: The ‘Off-Switch’ [pdf]

#15

Post by freedamn » Sun, Jun 12, 2016, 12:19 AM UTC

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

If the thing is anything evolving from an internet, it would surely know how to shut off or have a strong incentive not to. We would have, but then you don't want the plans known. I see.

Re: The ‘Off-Switch’ [pdf]

#16

Post by alanbernstein » Sun, Jun 12, 2016, 12:57 AM UTC

CM30 wrote: post #14

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

I think the captcha would be kind of useless, but the paywall might be able to prevent any issues in itself. Unless AI start being offered credit cards in the future, it's unlikely it'll be able to pay to get past one. So what's left? Trick some poor human to let it in? On the other hand, a real life robot has a fairly easy to use off switch. It's called 'destroy the robot with copious amounts of firepower'.

I imagine an AI with internet access could manage to find a few credit card numbers...

Re: The ‘Off-Switch’ [pdf]

#17

Post by sullyj3 » Sun, Jun 12, 2016, 7:59 PM UTC

lbenes wrote: post #2

Should we really be openly discussing our plans for an ‘off-switch’ on the public Internet? Shouldn’t there be some attempt to keeps the ideas/plans hidden in a way that they won’t be archived/read by an AI from the future? If I were seriously worried about this, I would put my plans and all discussions behind a CAPTCHA + paywall like structure that could not be crawled/archived.

It doesn't take a superintelligence to reason that the less intelligent people who created you probably took out some safety measures in case of you going rogue, and to figure that one of them is probably a kill switch. It's just the obvious thing to do. The AI doesn't need to stumble across past vague theorizing in order for this to occur to it.