Rationality: From AI to Zombies • Hacker News Board

Re: Rationality: From AI to Zombies

#111

Post by Strilanc » Fri, Mar 20, 2015, 9:48 PM UTC

nsxwolf wrote: post #62

convexfunction wrote: post #43

Earlier quoted context omitted.

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real ; that was why it was a big deal.

You don't what? What policy? Information hazards? What?! I feel like I just stumbled into a Scientology auditing session or something reading your comment.

"Information hazard" is just a term for "things you or others learn and then are worse off because of". Examples of information hazards:

- Instructions on how to make a gigaton bomb with processes and materials anyone can acquire and assemble privately.

- A reachable-by-a-country mechanism for blowing up the sun.

- An image designed to temporarily or permanently mess with your vision. Sharing [1] on reddit as a "cool optical illusion" would be mean.

- Probably whatever a computer spits out if you give it a scan of your brain and set it on the task of finding a paragraph/image/movie/whatever that maximizes dopamine production.

- Some people would consider a devastating argument against free will to be an information hazard, predicting that people would then do whatever they wanted because they "couldn't be blamed".

- If you're a hardcore atheist: religions. If you're a hardcore theist who thinks the devil is out to trick you: convincing arguments against belief.

1: http://en.wikipedia.org/wiki/McCollough_effect

Re: Rationality: From AI to Zombies

#112

Post by jtheory » Fri, Mar 20, 2015, 10:10 PM UTC

adamzerner wrote: post #9

jtheory wrote: post #3

Earlier quoted context omitted.

I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... Vague counterpoint -- this is an ebook collecting essays from lesswrong.com, which is a bit more subtle as names go.

> I don't think I could bring myself to be a "Bright" for that reason (see the-brights.net)... A name is enough of a reason on it's own?

Sure.

That is, my understanding of reality is the same, regardless. I think the goals of "The Brights" are laudable, generally. But I didn't register on their website, and wouldn't really want my photo and identify featured as a Bright, because the name just doesn't feel right to me.

How you identify yourself makes a difference to how you're judged, and hence the influence you can bring to bear on the world.

Or, you know, if I someday gain a bit more public recognition and clout, it may start to matter then how I self-identified back in 2010.

Re: Rationality: From AI to Zombies

#113

Post by jtheory » Fri, Mar 20, 2015, 10:12 PM UTC

maaku wrote: post #61

Strilanc wrote: post #52

Earlier quoted context omitted.

Given that this compilation was published on March 11th, and hpmor finished on March 14th, I don't think it's making the rounds now because of the timing of hpmor. Their finish/release dates are simply close together.

The release dates were purposefully close together.

I don't have inside knowledge, but yeah, it seems extremely likely -- the sequences have been available for years, after all.

Re: Rationality: From AI to Zombies

#114

Post by kbenson » Fri, Mar 20, 2015, 10:22 PM UTC

zyxley wrote: post #105

convexfunction wrote: post #43

Earlier quoted context omitted.

Eliezer agrees with you! Roko's Basilisk in particular is wrong. As a matter of policy though, you do not just publicly publish information hazards that might be real ; that was why it was a big deal.

> you do not just publicly publish information hazards that might be real But... for "might be real" to be valid at all, you have to buy into the Roko's basilisk concept in the first place. If you don't, there's nothing to get upset about.

That's like saying you shouldn't be mad at someone who attempted to murder you because they failed. There's a reason we have a crime called "attempted murder". You may find the wikipedia article on "Inchoate offense" interesting, specifically the section on Impossibility[1].

1: http://en.wikipedia.org/wiki/Inchoate_offense#Impossibility

Re: Rationality: From AI to Zombies

#115

Post by jtheory » Fri, Mar 20, 2015, 10:23 PM UTC

philh wrote: post #27

zyxley wrote: post #21

Earlier quoted context omitted.

> Which - keeping in mind that at the time I had absolutely no idea this would all blow up the way it did - caused me to yell quite loudly at Roko for violating ethics given his own premises, I mean really, WTF? You're going to get everyone who reads your article tortured so that you can argue against an AI proposal? In the twisted alternate reality of RationalWiki, this became proof that I believed in Roko's Basilis…

More like, "he believed that he'd accidentally invented an eternal torture machine and then he tried to turn it on , and I yelled at him even though the eternal torture machine didn't work."

Or maybe -- he posted something that the fact of his posting it revealed he actually intuitively knew it was bunk.

See "Belief in belief", which is right near the beginning of the first sequence....

http://lesswrong.com/lw/i4/belief_in_belief/

It should have been obvious to Roko that either it was a worthless idea (in which case he was safe to post it... but shouldn't, because it's worthless); the other case was that it was a serious, valid idea, in which case he obviously should not post it.

Re: Rationality: From AI to Zombies

#116

Post by zyxley » Fri, Mar 20, 2015, 10:32 PM UTC

ikeboy wrote: post #110

zyxley wrote: post #106

Earlier quoted context omitted.

But that's assuming that unpleasantness per individual is a continuous function always distinguishable from normal human experience, rather than something like a line eventually rising from the discontinuous murky soup that contains all the constant minor annoyances of being human (itchy nose, wedgies, the feeling of the back of your tongue against the roof of your mouth, one shoe being a little tighter than the othe…

Even if it's discontinuous, it still needs to grow incredibly slowly. The ratio of pain neurons that fire for a speck versus for torture over 50 years is nowhere near 3||3. You need a function that grows so slow that outweighs 3||3, which is damn near impossible for any plausible function.

I care about impact on people, not neuronal activity, and the point I'm making is that there's a minimum below which the real impact on a given person, considered in aggregate with the impact of everything else that's a part of being human, is effectively nil.

Re: Rationality: From AI to Zombies

#117

Post by zyxley » Fri, Mar 20, 2015, 10:36 PM UTC

Strilanc wrote: post #107

zyxley wrote: post #103

Earlier quoted context omitted.

> The basic reason it should follow is that not doing so makes you inconsistent and exploitable I still don't understand whatever framework you're trying to put forward here. How does preferring specks over torture in some absurd hypothetical somehow lead to the conclusion that I should start blinding people?

I'm not trying to put forth a framework, I'm trying to explain the side you're not on without straw-manning them. - If you prefer 100000000000000000000000000000000000000 specks to 1 torture - and you follow the axiom that A > B implies x% A > x% B, and value people equally - then you prefer 1 speck to a 0.000000000000000000000000000000000000001% chance of torture - and you will want to throw a speck in someone's eyes…

What I see here is proof that this is a bad model that produces insane results, rather than it proving that I've made a bad decision.

Re: Rationality: From AI to Zombies

#118

Post by jtheory » Fri, Mar 20, 2015, 10:39 PM UTC

Houshalter wrote: post #82

maxerickson wrote: post #74

Earlier quoted context omitted.

That's just what I mean, you end up examining your beliefs about what your gut reaction would be. Holding the knife in your hand and deciding what to do with it is when you find out your real gut reaction.

But I don't really know if we want to build a moral theory upon my gut reactions to things. My gut reactions are probably inconsistent. They are optimized by experience, culture, and evolution, not for things vastly outside the scope of common everyday situations. I know I'm not a 100% selfless altruist. I don't donate all my money to charity and I'm not going to risk going to prison to sabotage a power plant. But th…

Right -- gut morality is strongly based on small, local interactions.

If someone invites you over for dinner and they have a live calf there, nursing at its mother, and they're intending to slaughter it in front of you, it's not the same (to gut morality) as if they slaughtered it the day before, and really not the same as if they just bought some veal at the supermarket (i.e.: paid someone out of sight to do it).

Likewise; buying something that was produced in a way that involved significant human misery and/or death feels awful to gut morality if the sweatshop, diamond mine, or whatever is next door. On the other side of the world? Eh... well this one is $6 cheaper...

Why should it be so different? Well, gut morality isn't logical, at all. It's worth a bit of effort to do better.

Re: Rationality: From AI to Zombies

#119

Post by ikeboy » Fri, Mar 20, 2015, 10:42 PM UTC

zyxley wrote: post #116

ikeboy wrote: post #110

Earlier quoted context omitted.

Even if it's discontinuous, it still needs to grow incredibly slowly. The ratio of pain neurons that fire for a speck versus for torture over 50 years is nowhere near 3||3. You need a function that grows so slow that outweighs 3||3, which is damn near impossible for any plausible function.

I care about impact on people, not neuronal activity, and the point I'm making is that there's a minimum below which the real impact on a given person, considered in aggregate with the impact of everything else that's a part of being human, is effectively nil.

So at some point you need to say that "person being hurt X" is fine, but "person being hurt X+ one speck" is not.

Re: Rationality: From AI to Zombies

#120

Post by jtheory » Fri, Mar 20, 2015, 11:06 PM UTC

maxerickson wrote: post #29

danbmil99 wrote: post #14

Careful. Yudkowski and LW have a dark side. (google Roko's Basilisk) Rationality applied to ethics is a sticky wicket. You have to subscribe to some metric of "good" in order to weigh and compare outcomes. EY's definition of good is ruthlessly utilitarian, and that has some extreme, nay dire consequences for certain thought experiments. Consume this philosophy at your own risk.

If you take seriously the idea of an infohazard, you have a problem, because the idea that information can be hazardous is itself hazardous information (at a minimum, it's a prompt for us sloppy thinkers to start imagining what sort of information might be hazardous). Something as innocuous as "Think safe thoughts!" enjoys a similar property.

That doesn't make sense, though -- info hazards are not normally things you might just think of, or we'd be screwed either way; "whatever you do, don't think of a pink elephant", right?

Toss out silly ideas like the basilisk thing, and think up real info hazard possibilities.

E.g., let's say you knew what Snowden was up to, a few weeks before he figured out how to publish what he'd found and leave the US.

You'd have done a lot of harm to post that info publicly.

Suppose you find a laptop left in a taxi, with police records including personal details of rape victims in your city for the past 20 years. Post it to 4chan? Uh, no.

What about ongoing undercover investigations? Post it?

You accidentally shoulder-surf sensitive credentials -- bank manager's login, nuclear plant access code, whatever. Post them online?

Simple ways to make highly dangerous weapons, drugs, poisons, etc. can also be info hazards.

It's honestly not that hard to imagine real-life examples, without having the dangerous info in question or descending into government assassin-type scenarios.

It's just any data that, if you make it generally available, will predictably result in bad things happening.

In some cases -- like revealing important security flaws -- a public disclosure may be a last resort if less harmful methods have failed, but it's likely not an ethical first resort in those cases.