Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rationality: From AI to Zombies

intelligence.org

Post reply on HN
Print view View original

31–40 of 142 posts

Re: Rationality: From AI to Zombies

#31

Post by Retric »

Houshalter wrote: post #17
danbmil99 wrote: post #14

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

The Roko's Basilisk thing is way overblown. The author addressed the issue here: https://www.reddit.com/r/xkcd/comments/2myg86/xkcd_1450_aibo... And of course some of the topics he writes about are always going to be controversial. But utilitarianism is hardly radical.

People tend to ignore that for a logical trail to work every stated and unstated assumption needs to be 100% true or it's pointless. Roko's Basilisk is great example where even the slightest weakness in any part of the argument makes the whole idea pointless. (And there are a lot of unstated assumptions.)

EX: What's the maximum computational power of the universe?

Re: Rationality: From AI to Zombies

#32

Post by zyxley »

TeMPOraL wrote: post #25
zyxley wrote: post #19

Earlier quoted context omitted.

Yudkowsky's "good", at that of his fans, also tends to be naively utilitarian, ignoring modern development of that philosophy (see his whole "it's better for one person to be tortured for 50 years than for a very large number of people to each get one speck of dust in their eye" thing, for example).

Check out that topic more carefully. As far as I remember, it was a toy example pointing out where the naive application of utilitarianism goes bonkers, and from what I can tell, Eliezer doesn't actually subscribe to "torture over dust specks".

He does. He says so explicitly in the comments.

> I'll go ahead and reveal my answer now: Robin Hanson was correct, I do think that TORTURE is the obvious option, and I think the main instinct behind SPECKS is scope insensitivity.

Re: Rationality: From AI to Zombies

#33

Post by TeMPOraL »

zyxley wrote: post #32
TeMPOraL wrote: post #25

Earlier quoted context omitted.

Check out that topic more carefully. As far as I remember, it was a toy example pointing out where the naive application of utilitarianism goes bonkers, and from what I can tell, Eliezer doesn't actually subscribe to "torture over dust specks".

He does. He says so explicitly in the comments. > I'll go ahead and reveal my answer now: Robin Hanson was correct, I do think that TORTURE is the obvious option, and I think the main instinct behind SPECKS is scope insensitivity.

I stand corrected.

Re: Rationality: From AI to Zombies

#34

Post by DanAndersen »

philh wrote: post #30
TeMPOraL wrote: post #25

Earlier quoted context omitted.

Check out that topic more carefully. As far as I remember, it was a toy example pointing out where the naive application of utilitarianism goes bonkers, and from what I can tell, Eliezer doesn't actually subscribe to "torture over dust specks".

I think he does, although he doesn't say so explicitly in the original post. http://lesswrong.com/lw/kn/torture_vs_dust_specks/ At any rate, I choose torture, and I think that's the majority view at the LW meetups I attend, and I think we mostly take it for granted that Eliezer chooses torture.

Would you be able to help me understand your reasoning? From my perspective, "torture vs dust specks" doesn't seem to be a thing you can just "shut up and multiply" on, because the impact of torture vs the impact of a dust speck seem fundamentally different. To me the negative impact of pain is not just the sensory experience on a purely linear scale, but the effect it has on a person's life even afterward. It seems an overly simplistic model to put it on a linear scale, when there seem to be negative experiences that we easily "bounce back from" and others that we don't.

When measuring the negative impacts of a speck in the eye, regarding its impact on any individual's ability to achieve their goals in life or forget about the discomfort in the future, I have trouble seeing how a dust speck is anything but zero in those regards.

Re: Rationality: From AI to Zombies

#35

Post by Strilanc »

philh wrote: post #22
danbmil99 wrote: post #14

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

> EY's definition of good is ruthlessly utilitarian, Note that people frequently misunderstand Eliezer, especially wrt the basilisk. (Though it's not clear whether you're thinking about the basilisk in this paragraph, so that "especially" may not be relevant.) With this in mind, can you give an example of a specific moral claim where you think you disagree with Eliezer?

(Different person here.)

The most common example I see is the dust-specks vs torture thing[1], where Eliezer thinks increasing dust specks eventually outweighs the torture and various other people think torture outweighs any number of dust specks. Eliezer is basing his position on taking a few decision theory axioms as obviously true, and others are basing their position on taking their moral intuitions as obviously true (i.e. a "One's modus ponens is another's modus tollens" situation).

1: http://lesswrong.com/lw/kn/torture_vs_dust_specks/

Re: Rationality: From AI to Zombies

#36

Post by pyrois »

philh wrote: post #22
danbmil99 wrote: post #14

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

> EY's definition of good is ruthlessly utilitarian, Note that people frequently misunderstand Eliezer, especially wrt the basilisk. (Though it's not clear whether you're thinking about the basilisk in this paragraph, so that "especially" may not be relevant.) With this in mind, can you give an example of a specific moral claim where you think you disagree with Eliezer?

I'll give it a shot, but first, a list of disclaimers: Yudkowsky gets a ton of unjustified internet hate and scorn, and I disagree with a lot of it. I read a number of the sequences, and quite enjoyed them. I also think his reaction to Roko's Basilisk was pretty reasonable: someone on your form comes up with a way to basically guarantee eternal torture for anyone who reads it, and then posts it, thus guaranteeing eternal torture for your forum readers? Who cares that the idea won't actually guarantee any such thing, Roko _thought_ that it might; I would be pissed as hell.

Anyway, my point is, I'm only offering a gentle and hopefully reasoned disagreement with someone I regard highly. I'm not jumping on the "fuck that guy" train. Moving on.

There was a post that boiled down to the question "Would you rather 3 ||| 3 people (where | is an ascii stand-in for Knuth's up-arrow notation) get a mote of dust in their eye, or one person be horrifically tortured for 50 years?" and his conclusion was basically "you can use math to assign some incredibly small epsilon of suffering to getting a mote of dust in your eye, but eventually, if you sum enough people, it's more suffering overall than one poor person getting horrifically tortured". My problem with his post was that I think that any morality scheme that results in the person getting tortured is fundamentally flawed, and I don't care how much math you throw at me to try and "prove" that it's better.

I don't think I've ever heard of an attempt to rigorously derive morality that I agree with. Morality is too contextual, too messy, for us to perfectly capture in those sorts of models. It's especially bad when we try and model morality mathematically, and then take as gospel the result of that model, rather than say "oh, um, that's not a great result, the model must be wrong".

Re: Rationality: From AI to Zombies

#37

Post by philh »

DanAndersen wrote: post #34
philh wrote: post #30

Earlier quoted context omitted.

I think he does, although he doesn't say so explicitly in the original post. http://lesswrong.com/lw/kn/torture_vs_dust_specks/ At any rate, I choose torture, and I think that's the majority view at the LW meetups I attend, and I think we mostly take it for granted that Eliezer chooses torture.

Would you be able to help me understand your reasoning? From my perspective, "torture vs dust specks" doesn't seem to be a thing you can just "shut up and multiply" on, because the impact of torture vs the impact of a dust speck seem fundamentally different. To me the negative impact of pain is not just the sensory experience on a purely linear scale, but the effect it has on a person's life even afterward. It seems…

Basically I think you're just rounding something down to zero which isn't quite zero. It's too small to express, but it's not zero, and when you multiply it by 3^^^3, it becomes far worse than fifty years of torture.

Eliezer asked a follow-up question in the thread: would you pay a penny to avoid the dust specks?

Re: Rationality: From AI to Zombies

#38

Post by DanAndersen »

philh wrote: post #37
DanAndersen wrote: post #34

Earlier quoted context omitted.

Would you be able to help me understand your reasoning? From my perspective, "torture vs dust specks" doesn't seem to be a thing you can just "shut up and multiply" on, because the impact of torture vs the impact of a dust speck seem fundamentally different. To me the negative impact of pain is not just the sensory experience on a purely linear scale, but the effect it has on a person's life even afterward. It seems…

Basically I think you're just rounding something down to zero which isn't quite zero. It's too small to express, but it's not zero, and when you multiply it by 3^^^3, it becomes far worse than fifty years of torture. Eliezer asked a follow-up question in the thread: would you pay a penny to avoid the dust specks?

Right, that's why I reject the assumption that you can multiply it like that.

Imagine a society of 3^^^3 people, first in a universe in which the dust speck thing happened to all of them, and then in one in which it didn't happen. What would we expect to see different between the societies, 1 minute, 1 day, 1 year afterward? Would there be any difference? Any new wars or suffering?

The only reasonable outcome in which the dust specks would be worse would be, admittedly, if there's a non-negligible portion of the population that, as a result of getting the dust speck while operating machinery or doing something important like surgery, ends up making some terrible mistake that causes lasting harm to someone or themselves. In that case, the outcome is "hey, some huge number of people got maimed or killed, so the torture of one is OK in comparison."

But my thought is that your line of reasoning would still be held even if all 3^^^3 people were given the dust speck when they are sitting alone in their room away from any potentially-dangerous objects. Is that the case? In that particular case I don't see what the dust speck would actually do to any person in the future (beyond a single sensory experience in the present).

And to answer your question, I wouldn't pay a penny to avoid the dust speck. Maybe I'm just particularly frugal, being a grad student at the moment.

Re: Rationality: From AI to Zombies

#39

Post by [deleted] »

DanAndersen wrote: post #34
philh wrote: post #30

Earlier quoted context omitted.

I think he does, although he doesn't say so explicitly in the original post. http://lesswrong.com/lw/kn/torture_vs_dust_specks/ At any rate, I choose torture, and I think that's the majority view at the LW meetups I attend, and I think we mostly take it for granted that Eliezer chooses torture.

Would you be able to help me understand your reasoning? From my perspective, "torture vs dust specks" doesn't seem to be a thing you can just "shut up and multiply" on, because the impact of torture vs the impact of a dust speck seem fundamentally different. To me the negative impact of pain is not just the sensory experience on a purely linear scale, but the effect it has on a person's life even afterward. It seems…

[deleted]

Re: Rationality: From AI to Zombies

#40

Post by maxerickson »

TeMPOraL wrote: post #18
danbmil99 wrote: post #14

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

The lesson I learned from Roko's Basilisk event that I think Eliezer and most of the LW community also endorses is that if you're not careful, applying some models too far can have those "extreme, nay dire consequences for certain thought experiments" - so it's best not to commit too far to those methods, because they may have some caveats, or you might just have applied them wrong.

Yet in this same thread here there is a discussion of torture and dust specks. As if the mystical power to prevent dust specks is close enough to reality to be interesting.

I wonder how many people that fed that question into their mentation have considered sabotaging a coal plant. Obviously, that's ethically more complicated than the thought experiment, but it follows the outline.

(If you believe that the substitutes for coal power are available and superior, then taking it offline should still be a net good, one that is distributed over a great many people, with the downside (jail) concentrated on the saboteur.)

Post reply on HN