On being an AI in a box • Hacker News Board

Re: On being an AI in a box

#61

Post by wlievens » Tue, Dec 08, 2009, 10:26 AM UTC

gizmo wrote: post #20

giardini wrote: post #15

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug. I wish Eliezer and others would set aside meta-AI (dire predictions and worries about AI, the coming singularity, etc.) and concentrate on the problem of creating AI Guess there's no money in that. If only someone would pull the plug on this nonsense...

Oh no, somebody is doing research in an as of yet unexplored area! That must be a humongous waste of time! Pull the plug before too many papers have been written! Before I read Eliezer's work I thought the design an AI first, think about details & security afterward was a viable strategy. Eliezer illustrates how such an approach can go very wrong. Valuable research, if you ask me. PS: pulling the plug misses the poin…

Now, let me play devil's advocate here, as I do think that Eliezer writes interesting stuff; but what is he contributing to the field of AI beyond what movies like Terminator, Space Odyssey or Eagle Eye also illustrate (AI takes over humans)?

Re: On being an AI in a box

#62

Post by wlievens » Tue, Dec 08, 2009, 10:30 AM UTC

MarkPNeyer wrote: post #36

asciilifeform wrote: post #35

Earlier quoted context omitted.

> a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust…

There any many possible ways civilization could end. There are plenty of natural disasters (think supervolcanoes, or asteroid impacts) that could also destroy civilization. I don't see the harm in thinking about preventing one of them.

We know a lot more about how those work, which may well be a necessary condition when organizing prevention.

Re: On being an AI in a box

#63

Post by wlievens » Tue, Dec 08, 2009, 10:31 AM UTC

asciilifeform wrote: post #40

MarkPNeyer wrote: post #39

Earlier quoted context omitted.

If they're not taking any money from the government, what do you care what other people study? Research is like buying lottery tickets, except you have no idea how big the payoff could be. If you think 'our goose is cooked' and a transhuman intelligence could help us out, doesn't it make sense to support the development of a transhumanist intelligence?

> what do you care what other people study? I watched people with genuine potential (Eliezer Y., for instance) turn from groundbreaking AI work to writing "AI might kill us all!" screeds and recycled mathematics. A decade ago I was half-certain that he would eventually invent an artificial general intelligence. Now I am equally certain that he never will. Philosophizing and screaming "Caution!" is simply too much fun…

I partly agree with your points, except the last. I doubt any researcher is ever discouraged by the fearmongering you describe.

Re: On being an AI in a box

#64

Post by wlievens » Tue, Dec 08, 2009, 10:35 AM UTC

michael_dorfman wrote: post #6

lsb wrote: post #4

This basically boils down to how can a dangerous entity in captivity use bribes to attain freedom? which seems like a common enough trope between humans that any transhuman AI, having read all of recorded literature, and having accessed all of that person's conversations, would have enough data points to pattern match to the right bribe methodology.

Bribes and/or threats-- there's more than one possible motivator here.

Threats aren't possible in the scenario. Unless you refer to threatening not to give valuable information (cure for cancer) - but we're calling that a bribe here.

Re: On being an AI in a box

#65

Post by Eliezer » Tue, Dec 08, 2009, 3:05 PM UTC

giardini wrote: post #58

Eliezer wrote: post #42

Earlier quoted context omitted.

If you think that you can wait until AI looks like it's going to be developed now , and then suddenly develop all the math you need for the actually quite different design problem of Friendly AI, you have a very romantic view of how long it takes to do new basic math. I sometimes call this the intuitive theory of "science by press release", i.e., when science is needed, you just need someone to issue one of those pre…

If you think that, with our current limited knowledge and relatively limited resources, we can anticipate what is necessary to constrain AI to be "friendly" or that we can somehow protect ourselves from something that can reproduce the entire logical thought process of mankind in minutes then you have a very romantic view of human accomplishment. Indeed, that would make you the "John Henry" of AI.

On this I can only refer you to e.g. http://singinst.org/AIRisk.pdf. This is a standard reaction, and the standard reply is "The goal is to build something that doesn't want to hurt you in the first place." No one's trying to "constrain" anything, except in the sense that a programmer "constrains" code, not by fighting it, but by writing that particular piece of code into existence out of an exponentially vast space of possible alternatives.