Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rationality: From AI to Zombies

intelligence.org

Post reply on HN
Print view View original

81–90 of 142 posts

Re: Rationality: From AI to Zombies

#81

Post by ikeboy »

pyrois wrote: post #73
ikeboy wrote: post #60

Earlier quoted context omitted.

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ : >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose betwe…

He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true. To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of disc…

>It's a discontinuous function (in my opinion).

There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that.

You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture).

At some point you need to say that you'd pick specks even over 1 minute of torture, even over 1 second, on which I think most people's intuitions would stop saying that. Or you need to find some point between 50 years and 1 second where it become commensurate.

Re: Rationality: From AI to Zombies

#82

Post by Houshalter »

maxerickson wrote: post #74
TeMPOraL wrote: post #69

Earlier quoted context omitted.

I always found it as the "how much my gut reaction matches with what the mathematical model suggests" kind of puzzle - useful for both exploring your own biases and the limitations of the model.

That's just what I mean, you end up examining your beliefs about what your gut reaction would be. Holding the knife in your hand and deciding what to do with it is when you find out your real gut reaction.

But I don't really know if we want to build a moral theory upon my gut reactions to things. My gut reactions are probably inconsistent. They are optimized by experience, culture, and evolution, not for things vastly outside the scope of common everyday situations.

I know I'm not a 100% selfless altruist. I don't donate all my money to charity and I'm not going to risk going to prison to sabotage a power plant. But that doesn't mean I can't aspire to being more altruistic or want the world to be more utilitarian.

Almost all of EY's posts are about how your gut reaction to things is often wrong, inconsistent, or leads to outcomes you probably don't desire. E.g. Scope Insensitivity (http://lesswrong.com/lw/hw/scope_insensitivity/) where people will pay more money to save 4,500 people out of 11,000, rather than 4,500 out of 250,000.

Re: Rationality: From AI to Zombies

#83

Post by convexfunction »

nsxwolf wrote: post #62
convexfunction wrote: post #43

Earlier quoted context omitted.

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real ; that was why it was a big deal.

You don't what? What policy? Information hazards? What?! I feel like I just stumbled into a Scientology auditing session or something reading your comment.

You should familiarize yourself with that feeling so you can recognize it later; it's the feeling of falling into an inferential gap. (The thing might be worthless, true, like Scientology. I usually find that resolving confusion is worth my time, though.)

Re: Rationality: From AI to Zombies

#84

Post by Neslit »

TheOtherHobbes wrote: post #77
ikeboy wrote: post #60

Earlier quoted context omitted.

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ : >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose betwe…

Actually my decision probably depends on the person. [cough] But anyway. This isn't even philosophy - it's a digital remix of medieval scholasticism pretending to be philosophy. The irony is that politics proves empirically that ideas actually can be dangerous and harmful. And some ideas - actually narratives - can be very dangerous and harmful indeed. There's over a century of "persuasion technology" (Bernays, etc)…

I notice you didn't actually say at which point you would prefer torturing (10^100)*X people for Y years each, over torturing X people for Y+0.0000001 years each, for X and Y at least 0.0000001.

(You may assume you don't know anything in particular about these people, other than that they are adult humans.)

Re: Rationality: From AI to Zombies

#85

Post by [deleted] »

Neslit wrote: post #84
TheOtherHobbes wrote: post #77

Earlier quoted context omitted.

Actually my decision probably depends on the person. [cough] But anyway. This isn't even philosophy - it's a digital remix of medieval scholasticism pretending to be philosophy. The irony is that politics proves empirically that ideas actually can be dangerous and harmful. And some ideas - actually narratives - can be very dangerous and harmful indeed. There's over a century of "persuasion technology" (Bernays, etc)…

I notice you didn't actually say at which point you would prefer torturing (10^100)*X people for Y years each, over torturing X people for Y+0.0000001 years each, for X and Y at least 0.0000001. (You may assume you don't know anything in particular about these people, other than that they are adult humans.)

[deleted]

Re: Rationality: From AI to Zombies

#86

Post by Kutta »

I'm happy to see this in the preface (where EY looks back and details what he thinks he did wrong):

"Today I would write more courteously, I think. The discourtesy did serve a function, and I think there were people who were helped by reading it; but I now take more seriously the risk of building communities where the normal and expected reaction to low-status outsider views is open mockery and contempt.

Despite my mistake, I am happy to say that my readership has so far been amazingly good about not using my rhetoric as an excuse to bully or belittle others. (I want to single out Scott Alexander in particular here, who is a nicer person than I am and an increasingly amazing writer on these topics, and may deserve a part of the credit for making the culture of Less Wrong a healthy one.)"

Re: Rationality: From AI to Zombies

#87

Post by pyrois »

ikeboy wrote: post #81
pyrois wrote: post #73

Earlier quoted context omitted.

He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true. To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of disc…

>It's a discontinuous function (in my opinion). There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that. You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture). A…

Sure, they are commensurate at some point. I'd pick one nanosecond of torture over the dust-motes, for instance. But I'm not certain that I would ever choose the 50 years of torture over the dust motes for any number of dust-moted people. That's because it's not continuous, so you can't do the sort of epsilon-delta proofs that Yudkowsky's argument depends on.

At least, in my opinion.

Re: Rationality: From AI to Zombies

#88

Post by ikeboy »

pyrois wrote: post #87
ikeboy wrote: post #81

Earlier quoted context omitted.

>It's a discontinuous function (in my opinion). There are a finite number of possible brain states, far less than the 3||3 number. All possible "feelings" is likewise less than that. You need to be dividing all possible feelings of pain into two groups, and asserting that the absolute worst of the first group (containing dust specks) is incommensurate with the absolute best of the second group (containing torture). A…

Sure, they are commensurate at some point. I'd pick one nanosecond of torture over the dust-motes, for instance. But I'm not certain that I would ever choose the 50 years of torture over the dust motes for any number of dust-moted people. That's because it's not continuous, so you can't do the sort of epsilon-delta proofs that Yudkowsky's argument depends on. At least, in my opinion.

That means that for some amount of time, say X seconds, you would prefer X-1 seconds of torture for one person over specks, but prefer specks over X seconds of torture for one person.

But let's double each side; presumably you would make the same decision if asked again, right? So now you prefer X-1 seconds of torture done to each of two people, over twice as much specks.

Now, unless X is very low, you should prefer X for a single person over X-1 for two people. (If you disagree with this, please give a plausible value for X that makes it false.)

So you prefer X on a single person over double!specks, but prefer single!specks over X. This seems extremely unlikely. Or even if true, we should be able to make you pick torture for 50 years just by multiplying specks another couple of orders of magnitude.

Does this make his argument any clearer?

Re: Rationality: From AI to Zombies

#89

Post by zyxley »

Strilanc wrote: post #58
zyxley wrote: post #19

Earlier quoted context omitted.

Yudkowsky's "good", at that of his fans, also tends to be naively utilitarian, ignoring modern development of that philosophy (see his whole "it's better for one person to be tortured for 50 years than for a very large number of people to each get one speck of dust in their eye" thing, for example).

It's disingenuous to characterize moral disagreement as ignorance. Eliezer is aware that people think the torture is worse and of the arguments they make to justify it, he simply thinks they're wrong. Why does he think they're wrong? Probably because the resulting decision theories are exploitable or inconsistent in well understood ways when operating under uncertainty. If you prefer 100000000000000000000000000000000…

> If you prefer 10000000000000000000000000000000000 dust-specks over 1 tortures, then you should prefer to continuously blind someone with dust specks over taking a 0.0000000000000001% chance that they will torture someone.

I fail to see how the first part of this sentence implies the second.

Re: Rationality: From AI to Zombies

#90

Post by kybernetikos »

pyrois wrote: post #73
ikeboy wrote: post #60

Earlier quoted context omitted.

What do you say to the argument in http://lesswrong.com/lw/n3/circular_altruism/ : >But let me ask you this. Suppose you had to choose between one person being tortured for 50 years, and a googol people being tortured for 49 years, 364 days, 23 hours, 59 minutes and 59 seconds. You would choose one person being tortured for 50 years, I do presume; otherwise I give up on you. >And similarly, if you had to choose betwe…

He's assuming linearity, which at the very least needs justification. He's assuming that the function that maps from the pair (number of people, type of torture) to suffering is linear in the number of people, and also linear in the type of torture. I don't believe either of those things are true. To put it more clearly, he says: > So we can keep doing this, gradually - very gradually - diminishing the degree of disc…

We're social animals. There's an amount of discomfort (maybe not large) I would be prepared to undergo to help someone else, and I find myself thinking that others "should" also be prepared to undergo such an amount of discomfort to assist others. Given the choice to accept a dust mote temporarily in my eye as part of a huge crowd in order to save another person from torture, I would gladly accept that, and I think all reasonable people would too, therefore there is no number of people such that I believe the utility of dustmote vs torture turns out in favour of torture.
Post reply on HN