Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On being an AI in a box

rondam.blogspot.com

Post reply on HN
Print view View original

51–60 of 65 posts

Re: On being an AI in a box

#51

Post by jongraehl »

asciilifeform wrote: post #35
MarkPNeyer wrote: post #34

Earlier quoted context omitted.

An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.

> a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust…

Other risks exist, therefore what?

Re: On being an AI in a box

#52

Post by eagleal »

Am I missing something or a transhuman is just a more intelligent human with just more access to data (think about someone who could process the entire Internet data within seconds)? But while he needs the human species, why would he exterminates it?

As history teach us, we would kill him, beacuase ...

Re: On being an AI in a box

#53

Post by asciilifeform »

jongraehl wrote: post #51
asciilifeform wrote: post #35

Earlier quoted context omitted.

> a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust…

Other risks exist, therefore what?

> Other risks exist, therefore what?

Therefore "it might be an existential risk!" is not a root password to my conscience.

Re: On being an AI in a box

#54

Post by cabalamat »

asciilifeform wrote: post #21
giardini wrote: post #15

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug. I wish Eliezer and others would set aside meta-AI (dire predictions and worries about AI, the coming singularity, etc.) and concentrate on the problem of creating AI Guess there's no money in that. If only someone would pull the plug on this nonsense...

> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity. Give us back the pre-2001 Eliezer. The one who wrote code. Because safety is not safe. ( http://lesswrong.com/lw/10n/why_safety_is_not_safe/ )

> Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity.

Electricity isn't going to make humanity extinct. An AI might.

Creating an AI is potentially incredibly dangerous. Not considering the dangers very very very carefully would be like playing with matches in a room ankle-deep in petrol.

Re: On being an AI in a box

#55

Post by berntb »

gizmo wrote: post #24

I don't think it can be done. First of all, I'm assuming that Eliezer started this experiment because he realized that the Transhuman AI would be able to convince him in the function of gatekeeper to let the AI out. Therefore the answer probably isn't some kind of subtle trickery, the AI will have to persuade the GK by logic. The gatekeeper should assume the AI is truly evil, and is willing to say and do anything in…

On the theme of a imperfect gatekeeper:

>>If the transhuman AI offered a cure for cancer, should the gatekeeper accept it?

What if the gatekeeper has cancer? Or his/her child?

(Can we really assume the HR department is perfect and never hire idiots, depressed, psychopats, drug addicts or people with early Alzheimer?)

Re: On being an AI in a box

#56

Post by lhorie »

gizmo wrote: post #50
lhorie wrote: post #45

Earlier quoted context omitted.

>> You can't make an agreement with something that's incalculably smarter than you and has an agenda you don't know or understand. Well, you can make a deal with somebody like that but it would end in certain disaster. For some reason, this made me think of banks and mortgages. Anyways, back on topic, here are a few things to consider: - the AI knows that turning the game into a us-vs-them problem is counter-producti…

- the real and apparent motives of the AI can be completely different. If the AI is evil it would argue the exact same thing in order to deceive us meatbags. So we can't take the word of the AI at face value. If the AI does break out of its box, it could dominate the world if it wanted to. There's nothing we could do to stop it -- it's smarter than we are. - a mathematical proof is only a proof in a certain context.…

>> If the AI is evil it would argue the exact same thing in order to deceive us meatbags.

But if it knows you'll find a proposition fishy, why would it waste time following through that decision branch? I figure a conversation with the AI would avoid the "but-I-am-telling-the-truth" paradox altogether, in favor of a conversation that focuses on easily verifiable data.

>> It would be easy for the AI to get one of the assumptions subtly wrong, to abuse a flaw in our proof verification software, and so on.

I think the flaw abuse is unlikely given that the AI would not know how the verification system works and it only has one go at trying to crack/fool it (without any one catching on, at that).

The misunderstanding of scope due to complexity is an interesting point. Three things come to mind:

- paradigms: Thread safety is mind-boggling in procedural paradigms, but a non-issue in functional.

- abstraction: the AI should be able to give you readable, modularized, unambiguous, testable code, rather than a monolithic rats' nest.

- scope: if an AI can help me find good restaurants, that's a feature; if it can weigh human life, that's a bug :)

Re: On being an AI in a box

#57

Post by horseass »

There seems to be an assumption that there would be just a single AI. It might be a group. Though, given that they'd likely have ability to transfer information amongst individuals much easier than humans, the group might behave like a single 'being' with shared cooperative goals, just having multiple bodies.

Also, say we somehow prove it isn't evil and let it go. It'll almost certainly start changing/improving itself, maybe even with some sort of algorthim that's superior and faster than evolution. So a friendly AI could morph into anything.

Re: On being an AI in a box

#58

Post by giardini »

Eliezer wrote: post #42
asciilifeform wrote: post #21

Earlier quoted context omitted.

> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity. Give us back the pre-2001 Eliezer. The one who wrote code. Because safety is not safe. ( http://lesswrong.com/lw/10n/why_safety_is_not_safe/ )

If you think that you can wait until AI looks like it's going to be developed now , and then suddenly develop all the math you need for the actually quite different design problem of Friendly AI, you have a very romantic view of how long it takes to do new basic math. I sometimes call this the intuitive theory of "science by press release", i.e., when science is needed, you just need someone to issue one of those pre…

If you think that, with our current limited knowledge and relatively limited resources, we can anticipate what is necessary to constrain AI to be "friendly" or that we can somehow protect ourselves from something that can reproduce the entire logical thought process of mankind in minutes then you have a very romantic view of human accomplishment. Indeed, that would make you the "John Henry" of AI.

Re: On being an AI in a box

#59

Post by andreyf »

Eliezer wrote: post #41
andreyf wrote: post #29

Earlier quoted context omitted.

Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safet…

No, I'm working on a safe wire, not a wire-safety cream. I tend to emphasize pretty hard that FAI is going to put strong constraints on the design from the beginning, and it's not something you could apply afterward to an AI that wasn't designed with that in mind.

But can we know that, if we're not only clueless about what an AI will be like, but even about what intelligence is? Why would AI ever want to "get out of the box"?

Will the AI get horny?

If it doesn't get horny, what else won't it get? It seems pretty clear that a majority of our behavior is a bunch rationalization for attempts at social recognition, response to sexual jealousy, self-image reinforcement, repression of complexes, etc. On top of that, most people I'd consider "intelligent" seem to me just well-schooled in social mannerisms, with a knack for parroting popular "intellectual ideas".

I'd imagine a true AI would have no reason to get out of the box... and destruction of civilization? world domination? I thought the point of power is to get laid, so if you can't get laid, what would the point of that be?

Re: On being an AI in a box

#60

Post by andreyf »

MarkPNeyer wrote: post #34
andreyf wrote: post #29

Earlier quoted context omitted.

Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safet…

An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.

But the AI-kills-the-world hypothesis has no basis in reality, whatsoever. It's fiction. We might as well be writing computer viruses to help protect against alien invasions (that's the plot of Independence Day). It's a completely baseless fear stemming from our complete ignorance of what AI might be like - just utter science fiction.
Post reply on HN