Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Most Terrifying Thought Experiment of All Time

slate.com

Post reply on HN
Print view View original

61–70 of 84 posts

Re: The Most Terrifying Thought Experiment of All Time

#61

Post by FeepingCreature »

teekert wrote: post #35
onion2k wrote: post #32

Earlier quoted context omitted.

This makes him Yudkowsky the perfect lab animal! Think of the things we can learn, the lives we can save by performing experiments (that most think are unethical) on him! Only if you have a way to stop the other people getting dust in their eyes. Otherwise you're merely increasing the amount of suffering rather than mitigating it by transferring it all to one person.

Of course, we'd have to be sure first. But, perhaps we can start right away with splitting his organs among people who are on the waiting lists, surely, just with his kidneys we can already increase the life of two persons more than the kidneys can increase his life alone. Imagine if we harvest all of his organs! What a guy and what a theory this moral utilitarianism! It's great! Think of the possibilities, they're e…

Honestly?

If there was an insurance where I could sign up and they'd optionally harvest my organs if they'd find a really lopsided situation like "save those other three people and you're a really good bodytype match", I'd sign up for that if it gave me better expected lifetime than regular approaches. - Just because it's taboo, doesn't mean it's bad. Just because you consider it absurd doesn't mean that others would.

Re: The Most Terrifying Thought Experiment of All Time

#63

Post by super_mario »

madaxe_again wrote: post #28

It seems that everyone deriding this as pointless may be unaware of the simulation argument. It hasn't been disproven, and it's more likely than not that at some point it will be proven , given the pace of technological advancement. When you subscribe to the idea that this could all be a sim, the whole thing results in mental contortions that are literally maddening. Baudrillard observed not dissimilar in Simulacrum…

The cost of simulating entire universe becomes increasingly expensive to the point you need a universe of energy to simulate a universe. Daniel Dennet in his "Consciousness Explained" makes a good argument why this is so.

Also there was some research done on the constraints on the universe as a numerical simulation

http://arxiv.org/abs/1210.1847

telling us that we can do experiments to detect if we are living in a simulation (but the simulators always have a freedom to "increase the resolution" of their simulation to thwart our efforts). The paper does not discuss if there are limits on how far this game can be played on either side.

Re: The Most Terrifying Thought Experiment of All Time

#64

Post by super_mario »

cousin_it wrote: post #58
paul_f wrote: post #17

Good grief that was a waste of time. This entire concept only works if time travel is possible and will be discovered in the future. It isn't, and it won't be.

How so? The future AI just needs to have access to the financial records of the organization that created it, so it can find and punish the folks who didn't donate enough. No time travel involved, no brain simulations, nothing. Actually, Roko's original version of the basilisk was a pretty nice AI that wanted to help people. A hundred people die every minute, how many of them could be saved with better technology? If…

If I die before the AI capable of simulating me (how is it ever going to "upload" me to its simulation is an open question that needs to break a lot of knows physical laws to happen) how can it torture me for eternity? Unless the torture is meant to happen on "simulated me", not meat space me. Or unless the hypothesis is we are already living in simulated universe, and there are good physical constraints on why this is not likely.

Re: The Most Terrifying Thought Experiment of All Time

#65

Post by cousin_it »

super_mario wrote: post #64
cousin_it wrote: post #58

Earlier quoted context omitted.

How so? The future AI just needs to have access to the financial records of the organization that created it, so it can find and punish the folks who didn't donate enough. No time travel involved, no brain simulations, nothing. Actually, Roko's original version of the basilisk was a pretty nice AI that wanted to help people. A hundred people die every minute, how many of them could be saved with better technology? If…

If I die before the AI capable of simulating me (how is it ever going to "upload" me to its simulation is an open question that needs to break a lot of knows physical laws to happen) how can it torture me for eternity? Unless the torture is meant to happen on "simulated me", not meat space me. Or unless the hypothesis is we are already living in simulated universe, and there are good physical constraints on why this…

That strikes me as dodging the question of AI extortion, which is kind of orthogonal to the question of simulations and eternal torture. Just imagine that the AI might appear within your lifetime, and will be strong enough to physically find you and punish you. It's easy enough to figure out after the fact that you didn't donate to the AI's creation, and come up with a very unpleasant punishment that doesn't require any advanced tech. That's the heart of Roko's basilisk argument, I think it's interesting even if you don't mention AI at all. It's basically a decision theory puzzle.

Re: The Most Terrifying Thought Experiment of All Time

#66

Post by teekert »

FeepingCreature wrote: post #61
teekert wrote: post #35

Earlier quoted context omitted.

Of course, we'd have to be sure first. But, perhaps we can start right away with splitting his organs among people who are on the waiting lists, surely, just with his kidneys we can already increase the life of two persons more than the kidneys can increase his life alone. Imagine if we harvest all of his organs! What a guy and what a theory this moral utilitarianism! It's great! Think of the possibilities, they're e…

Honestly? If there was an insurance where I could sign up and they'd optionally harvest my organs if they'd find a really lopsided situation like "save those other three people and you're a really good bodytype match", I'd sign up for that if it gave me better expected lifetime than regular approaches. - Just because it's taboo, doesn't mean it's bad. Just because you consider it absurd doesn't mean that others would…

Well the piece says "torture someone" not "torture someone who has agreed to this torture because he himself wants what's best for the world." If the latter was the case I'd be fine with it. I'd also be fine with your insurance scheme as long as it is not mandatory.

Re: The Most Terrifying Thought Experiment of All Time

#67

Post by madaxe_again »

super_mario wrote: post #63
madaxe_again wrote: post #28

It seems that everyone deriding this as pointless may be unaware of the simulation argument. It hasn't been disproven, and it's more likely than not that at some point it will be proven , given the pace of technological advancement. When you subscribe to the idea that this could all be a sim, the whole thing results in mental contortions that are literally maddening. Baudrillard observed not dissimilar in Simulacrum…

The cost of simulating entire universe becomes increasingly expensive to the point you need a universe of energy to simulate a universe. Daniel Dennet in his "Consciousness Explained" makes a good argument why this is so. Also there was some research done on the constraints on the universe as a numerical simulation http://arxiv.org/abs/1210.1847 telling us that we can do experiments to detect if we are living in a si…

You don't need to simulate the entire universe in order to accurately simulate a causal chain - just the visible universe. Which is a lot smaller.

Hell, you could just simulate an individual and what they observe. For all I know, I'm talking to the AI in which I, and solely I, reside right now. Woo, simulation-solipsism.

Re: The Most Terrifying Thought Experiment of All Time

#68

Post by sirodoht »

lxmorj wrote: post #18

So stupid. Roko's Basilisk and Roko's Infinite Blowjob Machine are equally probable. When I was a kid I watched a movie about a monster that came up from the basement when the kids in the house accidentally said an ancient Indian incantation. "BlahBlahBluey" or something I got scared that perhaps, the last thing I said "Good Night, Batman" or whatever, was in fact coincidentally an ancient curse incantation and I had…

> Roko's Basilisk and Roko's Infinite Blowjob Machine are equally probable.

I agree. So what would they be so afraid of? My assumption is that the LessWrong community tries to come up with the best system of thinking, so by proving (by presenting a situation which shows that the best system is flawed, since we may be slaves right now) means that there really is no hope.

As far as LessWrong in general, they share some controversial opinions but some of their concepts seem very interesting[1]. However, learning and especially thinking about these concepts does not appear to have any positive effect in everyday life.

[1] http://lesswrong.com/lw/k5/cached_thoughts/

Re: The Most Terrifying Thought Experiment of All Time

#69

Post by shinigami »

teekert wrote: post #35
onion2k wrote: post #32

Earlier quoted context omitted.

This makes him Yudkowsky the perfect lab animal! Think of the things we can learn, the lives we can save by performing experiments (that most think are unethical) on him! Only if you have a way to stop the other people getting dust in their eyes. Otherwise you're merely increasing the amount of suffering rather than mitigating it by transferring it all to one person.

Of course, we'd have to be sure first. But, perhaps we can start right away with splitting his organs among people who are on the waiting lists, surely, just with his kidneys we can already increase the life of two persons more than the kidneys can increase his life alone. Imagine if we harvest all of his organs! What a guy and what a theory this moral utilitarianism! It's great! Think of the possibilities, they're e…

If you kill Yudkowsky, then the evil AI will take over since he won't be there to save the world. Therefore harvesting his organs won't be worthwhile. (He actually believes this).

Re: The Most Terrifying Thought Experiment of All Time

#70

Post by davidgerard »

Tenoke wrote: post #29
iSnow wrote: post #20

How is this more than a souped-up version of Pascal's wager? You better believe in God now because if you don't and he exists, you'll rot in hell.

Just in case anyone cares what the difference is - most possible Gods are about equally likely (no reason to think that one is much more likely to exist than another), while possible AIs which have reached SuperIntelligence status (to the point where they can simulate people for a start) are more likely to be certain ways (for example, self-preservation is not a consideration for an immortal God, nor is research some…

As far as I can tell (and I've read LW since 2010), you're asserting that on no evidence.
Post reply on HN