The Most Terrifying Thought Experiment of All Time

RationalWiki editors (including me ... and Dmytry, which is part of why he can be an asshole on LW) have had email from people traumatised by the basilisk. It turns out you can't think your way out of things you did think your way into, too. This is why the RationalWiki article http://rationalwiki.org/wiki/Roko%27s_basilisk was created, because personal email doesn't scale.

I am giving the reasons why I think so, if you disagree, you can point out what is wrong in my line of thinking: AIs are more likely to care for self-preservation (in some way), because AIs that don't are less likely to exist (since they die earlier), while Gods could have other mechanisms for survival (such as being immortal right off the bat).

From the small sample of victims I know of, they tend to be aspergic, have OCD, have not so strong a sense of self, and buy into all the LW versions of the transhumanist memes. They realise intellectually that it's a silly idea - but something about the story pushes all their buttons, perfectly wrongly.

When I stsrted getting email from them (cos I edit RW, RW was pretty much the only place on the Internet that mentioned the basilisk at the time, and LW refused to talk about it so they couldn't voice their concerns there), and found others at RW had been getting email too, I decided I had to do something to help because email wouldn't scale but it would be inhuman not to try to help. Hence the RW article on the basilisk. No email since then, so it seems to do the job.

See the first talk page archive for initial discussion as we try to work out how to talk about this completely fractally stupid thing in a manner that's helpful: http://rationalwiki.org/wiki/Talk:Roko%27s_basilisk/Archive1

>Because Pascal's wager could apply to any possible variation of supreme being - including non-sensical ones and ones that will punish you if you DO believe in them.

Since we know exactly nothing about Roko's Basilisk, we know nothing about its behavior. I could propose that it concludes that for its survival it is best to cultivate a certain level of cooperation with humans. Based on that it might determine that those who did believe in it before any proof are gullible and irrational and might exterminate them in order to free resources for sceptical thinkers who cooperate in the face of proof.

Non-sensical is an ill-defined category if we are talking about something with higher intellectual capabilities than ourselves - a dog might consider a lot of human behavior non-sensical.

>Roko's Basilisk isn't really a deity, it's a powerful AI with the capability of simulating you and everybody else (or possibly just you).

Since this most likely violates thermodynamical principles (simulation DOES require energy, the simulation of everything requires infinite energy) I fail to see how it is not either impossible or a deity.

>Pascal's wager does concern itself with the possibility that you are currently in a simulation.

IDK, to me this is only a different take on solipsism which states that I can only be sure that my own mind exists, everything else - my body, you, the world around me - might be an imagination.

>When I stsrted getting email from them (cos I edit RW, RW was pretty much the only place on the Internet that mentioned the basilisk at the time, and LW refused to talk about it so they couldn't voice their concerns there), and found others at RW had been getting email too, I decided I had to do something to help because email wouldn't scale but it would be inhuman not to try to help. Hence the RW article on the basilisk. No email since then, so it seems to do the job.

Alright, first of all, don't pretend that RW as a whole was trying to help LW by writing about it. A bunch of the RW writers clearly held contempt for LW, when they started writing about it, which is shown in inside discussions[1], as well as in the article about LW[2]. Additionally, even though the wording has been cleared a little (LW posters are no longer described as 'butthurt' for example) over the years, the Wiki(s) still contain inaccurate information (although I'll have to make a whole post to point it all out). I am mentioning this, because this is where the initial RW writings about the basilisk were.

Second of all, you published the Basilisk article in 2013 - 3 whole years after the incident, when many people had forgotten about it, and new users hadn't even heard about it. Were you really trying to help, and not just make fun of LW (which seems to me to be the motivation of most people on that Talk page you linked)? You do realize, that it was after the RW article (with the help of the rest of XiXiDu's posts on the topic from the same time) which brought the topic back, and massively increased the publicity of the basilisk? Indeed, I suspect (but have no proof) that [a lot] more than half the people who have heard of the basilisk, would've never done so if it wasn't for your article (and slate wouldn't write about it).

At any rate, let's accept that at least your motivation was only to help.How big is this 'small sample of victims I know of', that you were trying to help? How many emails did you and Dymtry receive from people who were actually distressed? And, less importantly - how many did you receive in 2013?

You sure are making it sound like a significant part of LW believes in the basilisk, both in various revisions of the articles on RW, and in comments on the topic. At any rate, is the number larger than the number of crackpots who comment on HN, or the crackpots who read RW, or is it something like 4 people who got over it quickly?

1. http://rationalwiki.org/wiki/Talk:LessWrong/Archive1

2. http://web.archive.org/web/20110719151907/http://rationalwik...