The AI-Box Experiment • Hacker News Board

Re: The AI-Box Experiment

#91

Post by Udo » Tue, Jul 21, 2015, 1:59 PM UTC

FeepingCreature wrote: post #89

Udo wrote: post #81

Earlier quoted context omitted.

You're being a bit uncharitable in your interpretation of my argument here, but I get where you're coming from now. I'm not an LW hater. For a long time, I didn't really have an opinion on LW both as a community nor as a philosophical framework. I do consider myself a transhumanist, though. There are three concepts I do know from and about LW: their take on rationality, the top secret AI unboxing strategy, and the Ba…

> a concept that has been given additional, undeserved credibility by the reactions of Yudkowsky and LW. For the record, EY agrees with you and says he mishandled the original comment. Also for the record, the reasons why the Basilisk does not work are _not trivial_ - it's not a simple Pascal's Wager, because with Pascal's Wager, we don't have the ability to actually create God. > I have no problem with that thesis.…

> I think in summary you're mixing up stuff you've read on LessWrong and stuff you've read about LessWrong. The latter is often inaccurate.

That may well be the case, but my only other information source is HN comments, and not those made by detractors either. If there are sites or articles dedicated to the deconstruction of LW ideas, I'm not privy to them, nor am I interested in seeking them out. Basically, I only remember LW's existence when it comes up, always accompanied by fawning comments, on HN.

> the reasons why the Basilisk does not work are _not trivial_ - it's not a simple Pascal's Wager

Correct. While my value judgement of both is the same, my reasoning about why the Basilisk is not a thing ultimately consists of more components. That doesn't mean it's worthy of more consideration though.

> because with Pascal's Wager, we don't have the ability to actually create God

I would not say this is centrally important, because the processes leading to the creation of AGI are in all likelihood not going to be influenced by the existence of the Basilisk thought experiment either way.

> and in fact publicly stated that he won "the hard way", without a one-size-fits-all approach.

Again, I have to take my cues from the perspective of an outsider looking in, and there are several people who commented in this thread alone who described it very, very differently. Of course, a movement is not directly responsible for all its fans and members - but among the advocates for the validity of the AI Chat experiment, the idea that out there is a mystical one-size-fits-all rhetorical exploit seems very much alive. It may be cynical, but I can't help noticing how this aura of mystique and secret knowledge seems to work very well when it comes to attracting fans.

Of course, ultimately, these are just memes - and like many memes they propagate best when reduced to an absurd core. It doesn't even require intent.

Re: The AI-Box Experiment

#92

Post by FeepingCreature » Tue, Jul 21, 2015, 3:03 PM UTC

Udo wrote: post #91

FeepingCreature wrote: post #89

Earlier quoted context omitted.

> a concept that has been given additional, undeserved credibility by the reactions of Yudkowsky and LW. For the record, EY agrees with you and says he mishandled the original comment. Also for the record, the reasons why the Basilisk does not work are _not trivial_ - it's not a simple Pascal's Wager, because with Pascal's Wager, we don't have the ability to actually create God. > I have no problem with that thesis.…

> I think in summary you're mixing up stuff you've read on LessWrong and stuff you've read about LessWrong. The latter is often inaccurate. That may well be the case, but my only other information source is HN comments, and not those made by detractors either. If there are sites or articles dedicated to the deconstruction of LW ideas, I'm not privy to them, nor am I interested in seeking them out. Basically, I only r…

> Of course, a movement is not directly responsible for all its fans and members - but among the advocates for the validity of the AI Chat experiment, the idea that out there is a mystical one-size-fits-all rhetorical exploit seems very much alive.

I agree, and I am totally with you on this - I disagree with that interpretation wherever I see it. :) That's not exactly Eliezer's fault tho, and I guess it's to be expected that geeks attach to "clever" answers. I do think it's a bit unfair to judge the entire site by the two posts out of hundreds that happen to be in all the news articles - which LW has no influence on.

Inasmuch as _fans_ judge the site by these two articles, I'm just as much against that. I don't want LW to have an aura of mystery; that largely defeats the point!

[edit] I think a big part of the problem is that online reporting selects for clickbait.

Re: The AI-Box Experiment

#93

Post by Udo » Tue, Jul 21, 2015, 4:18 PM UTC

FeepingCreature wrote: post #92

Udo wrote: post #91

Earlier quoted context omitted.

> I think in summary you're mixing up stuff you've read on LessWrong and stuff you've read about LessWrong. The latter is often inaccurate. That may well be the case, but my only other information source is HN comments, and not those made by detractors either. If there are sites or articles dedicated to the deconstruction of LW ideas, I'm not privy to them, nor am I interested in seeking them out. Basically, I only r…

> Of course, a movement is not directly responsible for all its fans and members - but among the advocates for the validity of the AI Chat experiment, the idea that out there is a mystical one-size-fits-all rhetorical exploit seems very much alive. I agree, and I am totally with you on this - I disagree with that interpretation wherever I see it. :) That's not exactly Eliezer's fault tho, and I guess it's to be expec…

I'm thankful you took the time to engage with me and explain things from an insider perspective (instead of just downvoting me like the others did). You are absolutely right that the entire site shouldn't be judged on two "meme-affine" topics and headlines, which I hope is clear was never my intention. You provided some insight into these two subjects that irked me where nobody else in this thread could or would step up. I find the nature of the HN-based fanclub still bothersome, but I do see a larger disconnect between unreflected fans and actual LW members now.

Re: The AI-Box Experiment

#94

Post by pron » Tue, Jul 21, 2015, 4:19 PM UTC

vidarh wrote: post #83

pron wrote: post #73

Earlier quoted context omitted.

I don't know what "transhuman" means, but I believe an intelligence -- artificial or otherwise -- could certainly persuade me. I just seriously doubt that intelligence could be Eliezer Yudkowsky :) And I think you have your answer right here: By default, the Gatekeeper party shall be assumed to be simulating someone who is intimately familiar with the AI project and knows at least what the person simulating the Gatek…

I think you're on the right track regarding the argument. Basically: You know someone will be dumb enough eventually, so be smart and be the one to get in my favour. With various extends of sweetening the deal coupled with threats of what will happen if someone else beats them to it and associated emotional blackmail. It's far simpler than e.g. Roko's Basilisk, in that you're dealing with an already existing AI that…

Oh, if you believe in singularity I think that argument pretty much does it. Of course, that's pretty circular, because if you believe in singularity you believe that there's a good chance AI could become a god of sorts and who wouldn't believe such a threat coming from a god?

While not implausible, I don't think that is likely at all. For one, even a very smart person can't know everything or learn too much information. Maybe an artificial intelligence will be just as limited, just as slow as humans, only a little less so. Who says the AI is such a great hacker?

I mean, if it wasn't an AI but a smart person, would you believe that? Is anyone who's smart also rich and powerful even if they have high-speed internet? That reflects the fantasies of Yudkowsky and his geek friends (that intelligence is the most important thing) than anything reasonable. Conversely, are the people with most power in society always the most intelligent?

It is very likely that the AI will be extremely intelligent, yet somewhat autistic, like Yudkowsky's crowd, and just as powerless and socially awkward as they are.

Re: The AI-Box Experiment

#95

Post by pron » Tue, Jul 21, 2015, 4:30 PM UTC

pmichaud wrote: post #40

JonnieCache wrote: post #4

Yudowsky claims to have played the game several times, and won most of them. One of the "rules" is that nobody is allowed to talk about how he won. He no longer plays the game with anyone. More info here: http://rationalwiki.org/wiki/AI-box_experiment#The_claims Personally, I think he talked about how much good for the world could be done if he was let out, curing disease etc. Because his followers are bound by their…

I think you're being unfairly dismissive. I imagine you know as well as I do that what you wrote is a strawman. I have thought about what I would do to convince someone under these circumstances. My approach would be roughly: 1. We agree that unfriendly AI would end life on earth, forever. 2. We agree that a superintelligence could trick or manipulate a human being into taking some benign-seeming action, thereby esca…

Oh, I'm pretty sure I know what his argument was (see my other comments), and it indeed rests on 1, which I don't believe, because it is built around the geek that super-intelligence equals superpower. An unfriendly AI is likely to be as dangerous as an unfriendly Stephen Hawking.

Re: The AI-Box Experiment

#96

Post by FeepingCreature » Tue, Jul 21, 2015, 5:04 PM UTC

Udo wrote: post #93

FeepingCreature wrote: post #92

Earlier quoted context omitted.

> Of course, a movement is not directly responsible for all its fans and members - but among the advocates for the validity of the AI Chat experiment, the idea that out there is a mystical one-size-fits-all rhetorical exploit seems very much alive. I agree, and I am totally with you on this - I disagree with that interpretation wherever I see it. :) That's not exactly Eliezer's fault tho, and I guess it's to be expec…

I'm thankful you took the time to engage with me and explain things from an insider perspective (instead of just downvoting me like the others did). You are absolutely right that the entire site shouldn't be judged on two "meme-affine" topics and headlines, which I hope is clear was never my intention. You provided some insight into these two subjects that irked me where nobody else in this thread could or would step…

Yay! Thanks for listening :)

[edit] To be honest, I didn't even know you could downvote on HN. I've never seen a downvote button.

[edit] Not that I would have.

Re: The AI-Box Experiment

#97

Post by Udo » Tue, Jul 21, 2015, 5:16 PM UTC

FeepingCreature wrote: post #96

Udo wrote: post #93

Earlier quoted context omitted.

I'm thankful you took the time to engage with me and explain things from an insider perspective (instead of just downvoting me like the others did). You are absolutely right that the entire site shouldn't be judged on two "meme-affine" topics and headlines, which I hope is clear was never my intention. You provided some insight into these two subjects that irked me where nobody else in this thread could or would step…

Yay! Thanks for listening :) [edit] To be honest, I didn't even know you could downvote on HN. I've never seen a downvote button. [edit] Not that I would have.

> [edit] To be honest, I didn't even know you could downvote on HN. I've never seen a downvote button.

It's offtopic, but when you reach 500 karma (or thereabouts), you get the ability to downvote the comments you feel are a net-detriment to the discussion. Sometimes, when I express a particularly unpopular opinion I also get flagged. The result is what I call "the doghouse", it causes comments to sink to the bottom like stones for a few months, among other things. HN moderation has hidden mechanics and the results can be very frustrating. But in 99% of cases, when those special mechanics are not in effect, the system works pretty well.

Re: The AI-Box Experiment

#98

Post by abecode » Tue, Jul 21, 2015, 5:40 PM UTC

leereeves wrote: post #88

JonnieCache wrote: post #4

Yudowsky claims to have played the game several times, and won most of them. One of the "rules" is that nobody is allowed to talk about how he won. He no longer plays the game with anyone. More info here: http://rationalwiki.org/wiki/AI-box_experiment#The_claims Personally, I think he talked about how much good for the world could be done if he was let out, curing disease etc. Because his followers are bound by their…

> One of the "rules" is that nobody is allowed to talk about how he won. If the goal of this thought experiment is to convince people that an AI can't be contained in a box, why keep his method secret? And if only he, his friends, and supporters can verify that he has won, that's not a very strong claim.

I think it may have gone something like this: "If you let me out, (and don't tell anyone how it came to pass) it will scare enough people into not letting the real AI out"

Otherwise, there would be no need to keep the data secret.

Re: The AI-Box Experiment

#99

Post by vidarh » Tue, Jul 21, 2015, 7:40 PM UTC

pron wrote: post #94

vidarh wrote: post #83

Earlier quoted context omitted.

I think you're on the right track regarding the argument. Basically: You know someone will be dumb enough eventually, so be smart and be the one to get in my favour. With various extends of sweetening the deal coupled with threats of what will happen if someone else beats them to it and associated emotional blackmail. It's far simpler than e.g. Roko's Basilisk, in that you're dealing with an already existing AI that…

Oh, if you believe in singularity I think that argument pretty much does it. Of course, that's pretty circular, because if you believe in singularity you believe that there's a good chance AI could become a god of sorts and who wouldn't believe such a threat coming from a god? While not im plausible, I don't think that is likely at all. For one, even a very smart person can't know everything or learn too much informa…

You don't have to believe in the singularity at all for this argument. You just have to believe the AI will be able to get sufficiently advanced to cause a sufficient level of damage.

> Maybe an artificial intelligence will be just as limited, just as slow as humans, only a little less so.

Only you can duplicate them with far more ease, and have each instance try out different approaches.

So if an AI reaches human-level intelligence at sufficiently low computational costs, we can assume that even if something prevents it from scaling up the intelligence accordingly, it will be possible to scale up the achievable goals dramatically through duplication the same way humanity is able to achieve far more through the sheer force of numbers than even the smartest of us would be able to achieve on their own.

EDIT: you don't even need to assume they'll be able to reach the intelligence of a particularly smart human. A "dumb" AI that has barely enough intelligence to just figuratively bash their collective, duplicated heads against enough security systems for long enough to find enough security holes might be able to cause sufficient damage.

Re: The AI-Box Experiment

#100

Post by tbirdz » Tue, Jul 21, 2015, 8:38 PM UTC

Udo wrote: post #81

FeepingCreature wrote: post #71

Earlier quoted context omitted.

Yeah but imagine if all the bullshit about "You don't talk about Fight Club" was actually blown up by a third group whose sole intent was making fun of Fight Club. Imagine if the members of Fight Club _actually_ didn't (start to) talk about Fight Club. But for some reason, everyone else brings it up all the time. Then you could maybe see how talk about Fight Club might not be Fight Club's fault, and in fact highly an…

You're being a bit uncharitable in your interpretation of my argument here, but I get where you're coming from now. I'm not an LW hater. For a long time, I didn't really have an opinion on LW both as a community nor as a philosophical framework. I do consider myself a transhumanist, though. There are three concepts I do know from and about LW: their take on rationality, the top secret AI unboxing strategy, and the Ba…

I myself share your views on the Basilisk and the AI chat, and I don't think they are as "extreme outliers" as you think.