Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rationality: From AI to Zombies

intelligence.org

Post reply on HN
Print view View original

71–80 of 142 posts

Re: Rationality: From AI to Zombies

#71

Post by kenbellows »

unbudgingprawn wrote: post #23
zyxley wrote: post #21

Earlier quoted context omitted.

> Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilis…

It seems like he deeply believes in it. I don't think he realizes it's just a modern version of Pascal's wager

In attempting to debunk this sort of statement, he has literally written the following: "In the hands of RationalWiki generally, and RationalWiki leader David Gerard particularly ..., [the fictional version of the Roko's Basilisk event] somehow metamorphosed into a Singularity cult that tried to get people to believe a Pascal's Wager argument to donate to their AI god on pain of torture. This cult that has literally never existed anywhere except in the imagination of David Gerard."[1]

[1] https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibo...

Re: Rationality: From AI to Zombies

#72

Post by ikeboy »

maxerickson wrote: post #63
ikeboy wrote: post #55

Earlier quoted context omitted.

And yet you posted this to the internet. Some people never learn, I guess.

Anything that somehow includes my saying "Think safe thoughts!" in a causal chain is already essentially inevitable. The future is not easily shaped.

What you posted, according to your own premises, is worse than just saying "Think safe thoughts!"

If I modded this, I would delete your post.

Re: Rationality: From AI to Zombies

#73

Post by pyrois »

ikeboy wrote: post #60
pyrois wrote: post #36

Earlier quoted context omitted.

I'll give it a shot, but first, a list of disclaimers: Yudkowsky gets a ton of unjustified internet hate and scorn, and I disagree with a lot of it. I read a number of the sequences, and quite enjoyed them. I also think his reaction to Roko's Basilisk was pretty reasonable: someone on your form comes up with a way to basically guarantee eternal torture for anyone who reads it, and then posts it, thus guaranteeing ete…

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ : >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose betwe…

He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true.

To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of discomfort

And I don't agree that you can. The difference between dust-mote and torture is not one of degree, but one of _kind_. It's a discontinuous function (in my opinion). I don't know where the discontinuity is, but it's there.

Re: Rationality: From AI to Zombies

#74

Post by maxerickson »

TeMPOraL wrote: post #69
maxerickson wrote: post #67

Earlier quoted context omitted.

My point is largely that I don't think you learn much until you actually go ahead and twist the knife yourself (or not!). Short of that, it's just a mind puzzle, "how does what I think my beliefs are match up with what I think my beliefs are" or so.

I always found it as the "how much my gut reaction matches with what the mathematical model suggests" kind of puzzle - useful for both exploring your own biases and the limitations of the model.

That's just what I mean, you end up examining your beliefs about what your gut reaction would be. Holding the knife in your hand and deciding what to do with it is when you find out your real gut reaction.

Re: Rationality: From AI to Zombies

#75

Post by TeMPOraL »

Houshalter wrote: post #66
spacehome wrote: post #50

Earlier quoted context omitted.

The visible universe? Some back-of-the-napkin estimates are that no more than 2^500 bits will be flipped in the region of space we can see, but there's a crucial assumption about minimum temperature you need to make to make that that estimate finite. On the other hand, there's some pretty compelling reasoning that the entire universe is actually infinite in extent, so ... infinite computational power?

What an absurd thing to consider. There is no possibility of building a computer the size of the universe. This website is full of crazy people talking about crazy things!

Because? How do you know? Show us the reasoning.

Also, there is this concept of "thought experiments". Building a universe-sized computer may sound as likely as sitting on the light wave and looking around, and yet the second one brought us Relativity.

Re: Rationality: From AI to Zombies

#76

Post by Houshalter »

DanAndersen wrote: post #34
philh wrote: post #30

Earlier quoted context omitted.

I think he does, although he doesn't say so explicitly in the original post. http://lesswrong.com/lw/kn/torture_vs_dust_specks/ At any rate, I choose torture, and I think that's the majority view at the LW meetups I attend, and I think we mostly take it for granted that Eliezer chooses torture.

Would you be able to help me understand your reasoning? From my perspective, "torture vs dust specks" doesn't seem to be a thing you can just "shut up and multiply" on, because the impact of torture vs the impact of a dust speck seem fundamentally different. To me the negative impact of pain is not just the sensory experience on a purely linear scale, but the effect it has on a person's life even afterward. It seems…

I wrote this as a reply to someone else above:

I think the dust speck argument would be better if first framed as a preference for yourself:

If you were going to live for 3^^^3 lifetimes, would you like to be tortured for 50 years in exchange for one fewer dust specks in your eye during each of those lifetimes.

I think most people would answer no. So the debate has nothing to do with morality in general. Instead the debate is about Scope Insensitivity (http://lesswrong.com/lw/hw/scope_insensitivity/), that our emotional reaction to things isn't diminished by even infinitesimally small quantities (see also Useless Medical Disclaimers http://lesswrong.com/lw/h4/useless_medical_disclaimers/).

I think I can make that position seem more reasonable if framed another way: Would you risk a 1/(3^^^3) chance at 50 years of torture, in exchange for getting rid of one dust speck out of your eye? I think most people would say yes. You couldn't even get out of bed in the morning if you weren't willing to take even incredibly small risks.

There is a much higher probability than 1/(3^^^3) that you could get in a car accident with injuries that cause 50 years of incredible pain, yet you will still probably risk driving for even trivial things.

And if you lived for 3^^^3 lifetimes and took this risk each time, you would likely suffer 50 years of torture during at least one lifetime.

Re: Rationality: From AI to Zombies

#77

Post by TheOtherHobbes »

ikeboy wrote: post #60
pyrois wrote: post #36

Earlier quoted context omitted.

I'll give it a shot, but first, a list of disclaimers: Yudkowsky gets a ton of unjustified internet hate and scorn, and I disagree with a lot of it. I read a number of the sequences, and quite enjoyed them. I also think his reaction to Roko's Basilisk was pretty reasonable: someone on your form comes up with a way to basically guarantee eternal torture for anyone who reads it, and then posts it, thus guaranteeing ete…

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ : >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose betwe…

Actually my decision probably depends on the person. [cough]

But anyway. This isn't even philosophy - it's a digital remix of medieval scholasticism pretending to be philosophy.

The irony is that politics proves empirically that ideas actually can be dangerous and harmful. And some ideas - actually narratives - can be very dangerous and harmful indeed.

There's over a century of "persuasion technology" (Bernays, etc) that exploits this.

Nothing I've seen Y write deals with the problem of politics as a social exploit in an insightful - never mind a useful - way.

Meanwhile real people are being tortured in real ways. What's his proposed rational solution to that problem?

Re: Rationality: From AI to Zombies

#78

Post by maxerickson »

ikeboy wrote: post #72
maxerickson wrote: post #63

Earlier quoted context omitted.

Anything that somehow includes my saying "Think safe thoughts!" in a causal chain is already essentially inevitable. The future is not easily shaped.

What you posted, according to your own premises, is worse than just saying "Think safe thoughts!" If I modded this, I would delete your post.

What are my premises?

If I don't take info hazards as a serious threat, there isn't anything to sort those statements with.

Re: Rationality: From AI to Zombies

#79

Post by TeMPOraL »

nsxwolf wrote: post #62
convexfunction wrote: post #43

Earlier quoted context omitted.

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real ; that was why it was a big deal.

You don't what? What policy? Information hazards? What?! I feel like I just stumbled into a Scientology auditing session or something reading your comment.

It's a concept coined by Nick Bostrom. Basically, some information can be harmful. As a simple, non-controvertial information hazard, imagine you learned an identity of a covert assasin of a foreign government, and you start telling about it to people around you. Don't be surprised if not only you drop dead, but the people you interact with as well.

http://www.nickbostrom.com/information-hazards.pdf

http://wiki.lesswrong.com/wiki/Information_hazard

Re: Rationality: From AI to Zombies

#80

Post by Houshalter »

pyrois wrote: post #68
Houshalter wrote: post #65

Earlier quoted context omitted.

I think the dust speck argument would be better if first framed as a preference for yourself: If you were going to live for 3^^^3 lifetimes, would you like to be tortured for 50 years in exchange for one fewer dust specks in your eye during each of those lifetimes. I think most people would answer no . So the debate has nothing to do with morality in general. Instead the debate is about Scope Insensitivity ( http://l…

Yes, I agree, I would take the risk, but those are two entirely different things. It's not the same argument framed in a different way at all. In Yudkowsky's argument, it's the option between one person _absolutely guaranteed_ to have 50 years of torture, vs 3^^^3 people _absolutely guaranteed_ to be dust-moted. I think the guarantee changes things significantly.

If you consistently take low risks that might result in getting tortured, then you are pretty much guaranteed to eventually lose the bet and suffer. The probability of eventually losing even a small bet approaches 1 if you take it enough times.

Just think of it as if you were going to other people and deciding to take this bet for them. 1/3^^^3 chance of torture in exchange for removing a speck of dust from their eye. I think you would be ok with that because it's an obvious choice for ourselves.

And if you continue to do this for enough people, eventually one of them will get tortured. After removing dust specks from approximately 3^^^3 people's eye. But you are right, there is a chance no one will get tortured. So do it for 3^^^^3 people then and the probability is 0.99999...

Post reply on HN