Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

391–400 of 947 posts

Re: Why are there so many rationalist cults?

#391

> One is Black Lotus, a Burning Man camp led by alleged rapist Brent Dill, which developed a metaphysical system based on the tabletop roleplaying game Mage the Ascension. What the actual f. This is such an insane thing to read and understand what it means that i might need to go and sit in silence for the rest of the day. How did we get to this place with people going completely nuts like this?

>How did we get to this place with people going completely nuts like this?

God died and it's been rough going since then.

Re: Why are there so many rationalist cults?

#392

> Many of them also expect that, without heroic effort, AGI development will lead to human extinction. > These beliefs can make it difficult to care about much of anything else: what good is it to be a nurse or a notary or a novelist, if humanity is about to go extinct? Replace AGI causing extinction with the Rapture and you get a lot of US Christian fundamentalists. They often reject addressing problems in the envir…

The Rapture isn't doom for the people who believe in it though (except in the lost sense of the word), whereas the AI Apocalypse is, so I'd put it in a different category. And even in that category, I'd say that's a pretty small number of Christians, fundamentalist or no, who abandon earthly occupations for that reason.

Re: Why are there so many rationalist cults?

#393
post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

Perhaps part of being rational, as opposed to rationalist, is having a sense of when to override the conclusions of seemingly logical arguments.

In philosophy grad school, we described this as 'being reasonable' as opposed to 'being rational'.

That said, big-R Rationalism (the Lesswrong/Yudkowsky/Ziz social phenomenon) has very little in common with what we've standardly called 'rationalism'; trained philosophers tend to wince a little bit when we come into contact with these groups (who are nevertheless chockablock with fascinating personalities and compelling aesthetics.)

From my perspective (and I have only glancing contact,) these mostly seem to be _cults of consequentialism_, an epithet I'd also use for Effective Altruists.

Consequentialism has been making young people say and do daft things for hundreds of years -- Dostoevsky's _Crime and Punishment_ being the best character sketch I can think of.

While there are plenty of non-religious (and thus, small-r rationalist) alternatives to consequentialism, none of them seem to make it past the threshold in these communities.

The other codesmell these big-R rationalist groups have for me, and that which this article correctly flags, is their weaponization of psychology -- while I don't necessarily doubt the findings of sociology, psychology, etc, I wonder if they necessarily furnish useful tools for personal improvement. For example, memorizing a list of biases that people can potentially have is like numbering the stars in the sky; to me, it seems like this is a cargo-cultish transposition of the act of finding _fallacies in arguments_ into the domain of finding _faults in persons_.

And that's a relatively mild use of psychology. I simply can't imagine how annoying it would be to live in a household where everyone had memorized everything from connection theory to attachment theory to narrative therapy and routinely deployed hot takes on one another.

In actual philosophical discussion, back at the academy, psychologizing was considered 'below the belt', and would result in an intervention by the ref. Sometimes this was explicitly associated with something we called 'the Principle of Charity', which is that, out of an abundance of epistemic caution, you commit to always interpreting the motives and interests of your interlocutor in the kindest light possible, whether in 'steel manning' their arguments, or turning a strategically blind eye to bad behaviour in conversation.

The importance Principle of Charity is probably the most enduring lesson I took from my decade-long sojurn among the philosophers, and mutual psychological dissection is anathema to it.

Re: Why are there so many rationalist cults?

#394

> Many of them also expect that, without heroic effort, AGI development will lead to human extinction. > These beliefs can make it difficult to care about much of anything else: what good is it to be a nurse or a notary or a novelist, if humanity is about to go extinct? Replace AGI causing extinction with the Rapture and you get a lot of US Christian fundamentalists. They often reject addressing problems in the envir…

Replace AGI with Climate Change and you've got an entirely reasonable set of beliefs.

Re: Why are there so many rationalist cults?

#395

Earlier quoted context omitted.

tbf you have to have read it to know that! I can't help but think it's probably the "favourite book" of a lot of people who haven't finished it though, possibly to a greater extent than any other secular tome (at least LOTR's lightweight fans watched the movies!). I mean, if you've only read the blurb on the back it's the perfect book to signal your belief in free markets, conservative values and the American Dream:…

at least LOTR's lightweight fans watched the movies! Oh, there’s movies for lazy Rand fans, too. https://www.imdb.com/title/tt0480239/ More of a Fountainhead fan, are you? Do ya like Gary Cooper and Patricia Neal? https://www.imdb.com/title/tt0041386/?ref_=ext_shr_lnk

> Oh, there’s movies for lazy Rand fans, too.

tbf that comment was about 50% a joke about their poor performance at the box office :D

Re: Why are there so many rationalist cults?

#396
post #300

Earlier quoted context omitted.

To be fair, the description of the dragon incident is pretty mundane, and all he does is prove that the large reptile they had previously been feeding (& worshiping) could be killed: "Then Daniel took pitch, and fat, and hair, and did seethe them together, and made lumps thereof: this he put in the dragon's mouth, and so the dragon burst in sunder: and Daniel said, Lo, these are the gods ye worship."

I don't think it's mundane to cause something to "burst in sunder" by putting some pitch, fat, and hair in its mouth.

The story is pretty clearly meant to indicate that the Babylonians were worshiping an animal though. The theology of the book of Daniel emphasises that the Gods of the Babylonians don't exist, this story happens around the same time Daniel proves the priests had a secret passage they were using to get the food offered to Bel and eat it at night while pretending that Bel was eating it. Or when Daniel talks to King Belshazzar and says "You have praised the gods of silver and gold, of bronze, iron, wood, and stone, which do not see or hear or know, but the God in whose power is your very breath and to whom belong all your ways, you have not honored". This is not to argue for the historical accuracy of the stories, just that the point is that Daniel is acting as a debunker of the Babylonian beliefs in these stories while asserting the supremacy of the Israelite beliefs.

Re: Why are there so many rationalist cults?

#397
post #392

> Many of them also expect that, without heroic effort, AGI development will lead to human extinction. > These beliefs can make it difficult to care about much of anything else: what good is it to be a nurse or a notary or a novelist, if humanity is about to go extinct? Replace AGI causing extinction with the Rapture and you get a lot of US Christian fundamentalists. They often reject addressing problems in the envir…

The Rapture isn't doom for the people who believe in it though (except in the lost sense of the word), whereas the AI Apocalypse is, so I'd put it in a different category. And even in that category, I'd say that's a pretty small number of Christians, fundamentalist or no, who abandon earthly occupations for that reason.

Yes, I removed a parenthetical "(or euphoria loop for the Rapture believers who know they'll be saved)". But I removed it because not all who believe in the Rapture believe they will be saved (or have such high confidence) and, for them, it is a doom loop.

Both communities, though, end up reinforcing the belief amongst their members and tend towards increasing isolation from the rest of the world (leading to cultish behavior, if not forming a cult in the conventional sense), and a disregard for the here and now in favor of focusing on this impending world changing (destroying or saving) event.

Re: Why are there so many rationalist cults?

#398
A very interesting read.

My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons.

This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now.

The article seems to try very hard to find something positive to say about these groups, and comes up with:

“Rationalists came to correct views about the COVID-19 pandemic while many others were saying masks didn’t work and only hypochondriacs worried about covid; rationalists were some of the first people to warn about the threat of artificial intelligence.”

There’s nothing very unique about agreeing with the WHO, or thinking that building Skynet might be bad… (The rationalist Moses/Hubbard was 12 when that movie came out — the most impressionable age.) In the wider picture painted by the article, these presumed successes sound more like a case of a stopped clock being right twice a day.

Re: Why are there so many rationalist cults?

#399
post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

As a former mechanical engineer, I visualize this phenomenon like a "tolerance stackup". Effectively meaning that for each part you add to the chain, you accumulate error. If you're not damn careful, your assembly of parts (or conclusions) will fail to measure up to expectations.

I saw an article recently that talked about stringing likely inferences together but ending up with an unreliable outcome because enough 0.9 probabilities one after the other lead to an unlikely conclusion.

Edit: Couldn't find the article, but AI referenced Baysian "Chain of reasoning fallacy".

Re: Why are there so many rationalist cults?

#400

I get the impression that these people desperately want to study philosophy but for some reason can't be bothered to get formal training because it would be too humbling for them. I call it "small fishbowl syndrome," but maybe there's a better term for it.

My thoughts exactly! I'm a survivor of ten years in the academic philosophy trenches and it just sounds to me like what would happen if you left a planeload of undergraduates on a _Survivor_ island with an infinite supply of pizza pockets and adderall
Post reply on HN