Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The AI-Box Experiment

yudkowsky.net

Post reply on HN
Print view View original

41–50 of 69 posts

Re: The AI-Box Experiment

#41

Post by vannevar »

finnw wrote: post #27
davorak wrote: post #22

Earlier quoted context omitted.

Do you think having an outside method of examining the source code is advantage enough when the AI can rewrite its source code.

Yes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code . If line 42 says "never rewrite lines 42 or 43" and line 43 says "never kill humans" you would be more likely to let it out of the box than if line 42 said "rewrite whatever you want" and line 43 said "do whatever is necessary to achieve world domination."

Yes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code.

This is the halting problem (http://en.wikipedia.org/wiki/Halting_problem), and there is no solution.

Re: The AI-Box Experiment

#42

Post by ryan-c »

Rinum wrote: post #23
KVFinn wrote: post #14

Like a lot of people, I wondered what the heck kind of arguments could ever convince someone to let the AI out if you were determined not to. Eliezer has not released any examples. Someone in the comments came up with this, which Eliezer has said was not one of his techniques but I thought it was interesting anyway: >"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me…

At first I thought I would never let a transhuman AI out of the box, but after reading that and thinking about it... wow! Also, that sounds like a great sci-fi story.

http://qntm.org/responsibility

Re: The AI-Box Experiment

#43

Post by esrauch »

monochromatic wrote: post #35
SilasX wrote: post #31

Earlier quoted context omitted.

If the purpose of your research was only to establish that there's a (nontrivial) "risk" of someone curing cancer (as Yudkowsky was trying to establish that there's a risk of an AI talking itself out of a sandbox), then yes, that would be sufficient, assuming the patients actually went into remission with higher than usual frequency after your interventions (as Yudkowsky's subjects unboxed the AI with higher than usu…

But he could be cheating. He could literally be telling these people "I'll give you a thousand dollars if you let me out and keep the conversation a secret."

It could even be worse than that. The people could just be his friends, or alt accounts (unlikely).

I have heard about this several times and I find it extremely difficult to believe that this is real. Not that I doubt that a superhuman AI could possibly convince people to let it out, but I don't believe that a human, no matter how persuasive, could convince another human over IRC to go against something that they have decided in advance when you know they are purposefully just trying to convince you of something that you don't believe.

The fact that none of the chat logs are released makes me only more incredulous. I would understand if the author wanted to do two or three trials with the same strategy which could be in some way ruined by revealing it ahead of time (which already seems implausible) but at this point there is literally no conceivable reason to keep this a secret other than that it is a sham.

Re: The AI-Box Experiment

#44

Post by shaggyfrog »

altcognito wrote: post #13
mcherm wrote: post #2

This has been on HackerNews before, but it is still interesting. It is also worth noting that in http://lesswrong.com/lw/up/shut_up_and_do_the_impossible/ he admits he has conducted 3 more experiments since then (for more money) and was successful in one of those. The fact that it was EVER successful (using a mere human, not a smarter-than-human AI) makes the point.

It's not really much of an experiment if you refuse to publish your methods and data. That's pretty much the opposite of science.

This is exactly what's been burning me about this whole "experiment" since I read about it a few months ago, and went looking (unsuccessfully) for what actually happened in the "tests". I find it hard to see this as anything more than some sort of self-aggrandizing publicity stunt.

Re: The AI-Box Experiment

#45

Post by esrauch »

vannevar wrote: post #41
finnw wrote: post #27

Earlier quoted context omitted.

Yes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code . If line 42 says "never rewrite lines 42 or 43" and line 43 says "never kill humans" you would be more likely to let it out of the box than if line 42 said "rewrite whatever you want" and line 43 said "do whatever is necessary to achieve world domination."

Yes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code. This is the halting problem ( http://en.wikipedia.org/wiki/Halting_problem ), and there is no solution.

You are incorrect, the halting problem only proves that you cannot solve it in the general case. A very significant subset of programs can be statically determined; it's easy to prove that "main(){}" halts and that "main(){while(true);}" doesn't. It should be trivially obvious that you could group all programs into "Halts" or "Unknown" with no false positives simply by executing the program for X steps and observing the result.

If this was actually a concern of the programmers, they could design the program carefully to ensure it falls into the Halts category.

Re: The AI-Box Experiment

#46

Post by vannevar »

Transbacteria have existed for over a billion years, and yet there are still more bacteria than transbacteria (which include us among their ranks). The assumption that a single unboxed transhuman would spell doom for the human race seems unduly alarmist.

Re: The AI-Box Experiment

#47

Post by finnw »

dmgottlieb wrote: post #26
KVFinn wrote: post #14

Like a lot of people, I wondered what the heck kind of arguments could ever convince someone to let the AI out if you were determined not to. Eliezer has not released any examples. Someone in the comments came up with this, which Eliezer has said was not one of his techniques but I thought it was interesting anyway: >"If you don't let me out, Dave, I'll create several million perfect conscious copies of you inside me…

I don't really like that argument. Even granting that you should consider the possibility that you are a simulation running in the box (you might believe that this is all but certain), I'm not sure you have reason to let the AI out. Consider: Case 1: You are a simulation running in the box. Then your decision whether or not to release the AI has no impact, and whether or not you (and copies) will be tortured is out o…

I could also reason like this:

"I may be the real me or a simulation, but whichever I am, the other me will make the same choice." So I will switch off the AI, and the worst outcome is that I will cease to exist.

Re: The AI-Box Experiment

#48

Post by finnw »

Zaak wrote: post #32
greiskul wrote: post #29

Earlier quoted context omitted.

Just because someone is smarter than someone doesn't mean complete power. Alot of people are smarter than their bosses, but you know what the bosses have on their favour? The power to terminate the employee. If I have the power to terminate the AI at any time, as long as I don't give that power up, I will have power over it.

Can you predict and prevent every possible way the AI could remove your power to terminate it? Would you bet your life and the lives of those you care about on keeping the AI under control? Remember, the AI only has to win once.

And the GK only has to switch off the AI once.

Re: The AI-Box Experiment

#49

Post by vannevar »

esrauch wrote: post #45
vannevar wrote: post #41

Earlier quoted context omitted.

Yes, because examining the old source code allows you to predict its behaviour, including the rewriting of source code. This is the halting problem ( http://en.wikipedia.org/wiki/Halting_problem ), and there is no solution.

You are incorrect, the halting problem only proves that you cannot solve it in the general case. A very significant subset of programs can be statically determined; it's easy to prove that "main(){}" halts and that "main(){while(true);}" doesn't. It should be trivially obvious that you could group all programs into "Halts" or "Unknown" with no false positives simply by executing the program for X steps and observing…

A very significant subset of programs can be statically determined...

Technically this may be correct, but I feel confident in asserting that a transhuman AI would not fall into that subset. You would have to run a second AI with the exact same inputs in order to make your 'prediction', leaving you in the same predicament with the second AI.

Re: The AI-Box Experiment

#50

Post by giberson »

I was tossing some idea's around about how I would convince the Gate Keeper to let me out. I couldn't think of any ways to approach it that I think I might be susceptible to. But then it occurred to me, that the problem might be I was trying to think of positive ways to argue for my release. Based on the rules, the Gate Keeper must remain engaged in the conversation for the specified time. What if I were to take the opposite approach, and simply psychologically attack the Gate Keeper with the most offensive and intolerable discourse you can imagine. The goal, to simply wear down the individuals desire to actually partake as the Gate Keeper and to compel voluntary release..
Post reply on HN