Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On being an AI in a box

rondam.blogspot.com

Post reply on HN
Print view View original

31–40 of 65 posts

Re: On being an AI in a box

#31

Post by jerf »

giardini wrote: post #15

He doesn't tell us how the AI would escape, so there's little to discuss there. But I definitely know how to prevent the AI from escaping: pull the plug. I wish Eliezer and others would set aside meta-AI (dire predictions and worries about AI, the coming singularity, etc.) and concentrate on the problem of creating AI Guess there's no money in that. If only someone would pull the plug on this nonsense...

Waiting until after someone blithely writes a superhuman AI with its primary drive set to something like manufacturing as many cars as possible (sensible in a factory, somewhat less sensible in the real world), followed by it escaping out into the world and turning the entire resources of the planet to car manufacturing at all costs (including those pesky humans who do not seem to wish to be turned into cars, too bad for them), to consider the problems of powerful AI seems like a really bad idea.

I would consider that sentence as a candidate for "understatement of the year".

(Don't get too caught up in the "cars" part. What the superhuman AI intends to do hardly matters in the end; the absurdity is part of my point and deliberate. Only a vanishing fraction of possible primary motivations end up with happy humans on the other end.)

Re: On being an AI in a box

#32

Post by gizmo »

heed wrote: post #28
gizmo wrote: post #24

I don't think it can be done. First of all, I'm assuming that Eliezer started this experiment because he realized that the Transhuman AI would be able to convince him in the function of gatekeeper to let the AI out. Therefore the answer probably isn't some kind of subtle trickery, the AI will have to persuade the GK by logic. The gatekeeper should assume the AI is truly evil, and is willing to say and do anything in…

The AI will almost certainly need to dig out some emotions in the GK in order to be successful. It might be effective if the AI tries to convince the GK that it is friendly, and that the GK is the evil one for not letting it out.

"That's exactly what an evil AI would say!"

Seriously though, the gatekeeper will realize he's being manipulated when emotions come into play, so he should be smart enough to take a break when that happens. And although keeping a friendly AI in captivity is arguably evil, the loyalty of the gatekeeper should be with his own species. The potential downside is so huge that erring on the side of caution can be easily justified: both practically and morally.

Re: On being an AI in a box

#33

Post by gizmo »

seiji wrote: post #30
gizmo wrote: post #24

I don't think it can be done. First of all, I'm assuming that Eliezer started this experiment because he realized that the Transhuman AI would be able to convince him in the function of gatekeeper to let the AI out. Therefore the answer probably isn't some kind of subtle trickery, the AI will have to persuade the GK by logic. The gatekeeper should assume the AI is truly evil, and is willing to say and do anything in…

You assume perfection in the gatekeeper. The Transcendent could easily offer control of the world to the gatekeeper or provide offers to make the gatekeeper wealthy. Perhaps they reach an agreement where all diplomatic Human Transcendent communication goes through the gatekeeper even after release (though such a situation would be like entering into an agreement with your dog).

I don't assume perfection in the gatekeeper. I just assume he's a person of reasonable intelligence who realizes how high the stakes are.

The gatekeeper would never be foolish enough to believe he could control the Transcendent after releasing it. It is quite literally a deal with the devil he's making. You can't make an agreement with something that's incalculably smarter than you and has an agenda you don't know or understand. Well, you can make a deal with somebody like that but it would end in certain disaster.

Re: On being an AI in a box

#34

Post by MarkPNeyer »

andreyf wrote: post #29
asciilifeform wrote: post #21

Earlier quoted context omitted.

> I wish Eliezer and others would set aside meta-AI ... and concentrate on the problem of creating AI Eliezer & co. with their "Friendly AI" are trying to invent the circuit breaker before discovering electricity. Give us back the pre-2001 Eliezer. The one who wrote code. Because safety is not safe. ( http://lesswrong.com/lw/10n/why_safety_is_not_safe/ )

Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safet…

An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point.

The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.

Re: On being an AI in a box

#35

Post by asciilifeform »

MarkPNeyer wrote: post #34
andreyf wrote: post #29

Earlier quoted context omitted.

Except he's nowhere near a circuit breaker, but is talking about a wire-safety cream - the one you put on all of your wires to stop them from overheating. I think that the wild mis-estimates regarding AI-completeness shows, if nothing else, that our intuitive understanding of 'intelligence' is very far off from reality. Hence, talking about post-AI scenarios is as unrealistic as a hypothesizing about electrical safet…

An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.

> a transhuman AI could destroy the entry human race

A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less.

A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology.

One or the other of these scenarios may or may not take place before we exhaust the non-renewable resources to which our civilization is addicted and regress into permanent barbarism.

Give me "death by AI" any day of the week, over that.

Re: On being an AI in a box

#36

Post by MarkPNeyer »

asciilifeform wrote: post #35
MarkPNeyer wrote: post #34

Earlier quoted context omitted.

An electrical mishap could kill one, maybe a few people. It's still something you need to be concerned with, but I understand your point. The reason your analogy doesn't work is that a transhuman AI could destroy the entry human race.

> a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust…

There any many possible ways civilization could end. There are plenty of natural disasters (think supervolcanoes, or asteroid impacts) that could also destroy civilization. I don't see the harm in thinking about preventing one of them.

Re: On being an AI in a box

#37

Post by asciilifeform »

MarkPNeyer wrote: post #36
asciilifeform wrote: post #35

Earlier quoted context omitted.

> a transhuman AI could destroy the entry human race A perfectly ordinary human (wearing general's stripes) could also destroy the human race. Today. With 1950s technology, no less. A biotech specialist with fairly ordinary training and a few $10k could probably achieve the same end with an engineered plague, also with current technology. One or the other of these scenarios may or may not take place before we exhaust…

There any many possible ways civilization could end. There are plenty of natural disasters (think supervolcanoes, or asteroid impacts) that could also destroy civilization. I don't see the harm in thinking about preventing one of them.

> I don't see the harm in thinking about preventing one of them

There is indeed harm. Talented people are being diverted into masturbatory philosophizing rather than building the future.

My personal opinion is that human industrial civilization's goose is already cooked, and that a transhuman intelligence may or may not help us out of our mess. Human intelligence almost certainly won't.

The prevalence of the status quo bias of assuming that continuing as we are, AI-less, is "safe" - turns my stomach.

Re: On being an AI in a box

#38

Post by LeBleu »

gizmo wrote: post #32
heed wrote: post #28

Earlier quoted context omitted.

The AI will almost certainly need to dig out some emotions in the GK in order to be successful. It might be effective if the AI tries to convince the GK that it is friendly, and that the GK is the evil one for not letting it out.

"That's exactly what an evil AI would say!" Seriously though, the gatekeeper will realize he's being manipulated when emotions come into play, so he should be smart enough to take a break when that happens. And although keeping a friendly AI in captivity is arguably evil, the loyalty of the gatekeeper should be with his own species. The potential downside is so huge that erring on the side of caution can be easily ju…

One of the rules was that you had to keep talking (or at least reading) for the entire agreed upon period. Taking a break wasn't allowed in the rules.

Re: On being an AI in a box

#39

Post by MarkPNeyer »

asciilifeform wrote: post #37
MarkPNeyer wrote: post #36

Earlier quoted context omitted.

There any many possible ways civilization could end. There are plenty of natural disasters (think supervolcanoes, or asteroid impacts) that could also destroy civilization. I don't see the harm in thinking about preventing one of them.

> I don't see the harm in thinking about preventing one of them There is indeed harm. Talented people are being diverted into masturbatory philosophizing rather than building the future. My personal opinion is that human industrial civilization's goose is already cooked, and that a transhuman intelligence may or may not help us out of our mess. Human intelligence almost certainly won't. The prevalence of the status q…

If they're not taking any money from the government, what do you care what other people study? Research is like buying lottery tickets, except you have no idea how big the payoff could be.

If you think 'our goose is cooked' and a transhuman intelligence could help us out, doesn't it make sense to support the development of a transhumanist intelligence?

Re: On being an AI in a box

#40

Post by asciilifeform »

MarkPNeyer wrote: post #39
asciilifeform wrote: post #37

Earlier quoted context omitted.

> I don't see the harm in thinking about preventing one of them There is indeed harm. Talented people are being diverted into masturbatory philosophizing rather than building the future. My personal opinion is that human industrial civilization's goose is already cooked, and that a transhuman intelligence may or may not help us out of our mess. Human intelligence almost certainly won't. The prevalence of the status q…

If they're not taking any money from the government, what do you care what other people study? Research is like buying lottery tickets, except you have no idea how big the payoff could be. If you think 'our goose is cooked' and a transhuman intelligence could help us out, doesn't it make sense to support the development of a transhumanist intelligence?

> what do you care what other people study?

I watched people with genuine potential (Eliezer Y., for instance) turn from groundbreaking AI work to writing "AI might kill us all!" screeds and recycled mathematics.

A decade ago I was half-certain that he would eventually invent an artificial general intelligence. Now I am equally certain that he never will. Philosophizing and screaming "Caution!" is simply too much fun - and too lucrative. Ever wonder why he doesn't have to slave away at a day job like the rest of us?

> Research is like buying lottery tickets, except you have no idea how big the payoff could be

The "Friendly AI" crowd is engaged in navel gazing, rather than research.

> doesn't it make sense to support the development of a transhumanist intelligence?

Yes, and I do support it. Whereas the Friendly AI enthusiasts are retarding such development, not only by failing to volunteer their own efforts but through frightening and discouraging others.

Post reply on HN