Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

681–690 of 947 posts

Re: Why are there so many rationalist cults?

#681
post #117

Earlier quoted context omitted.

To be honest I don't understand that objection. If you strip it from all its culty sociological effects, one of the original ideas of rationalism was to try to use logical reasoning and statistical techniques to explicitly avoid the pitfalls of known cognitive biases. Given that foundational tenet, "rationalism" seems like an extremely appropriate moniker. I fully accept that the rationalist community may have morphe…

I'm going to start a group called "Mentally Healthy People". We use data, logical thinking, and informal peer review. If you disagree with us, our first question will be "what's wrong with mental health?"

But, to be frank, "Mentally Healthy People" fully acknowledge and accept their emotions, and indeed understand that emotions are the fundamental way that natural selection implements motivation.

Calling yourself "rationalist" doesn't inherently mean that you think you're better than everyone else, or somehow infallible. To me it just means your specific approach to problem solving.

Re: Why are there so many rationalist cults?

#682

I think I found the problem! The rationalist community was drawn together by AI researcher Eliezer Yudkowsky’s blog post series The Sequences, a set of essays about how to think more rationally I actually don't mind Yudkowski as an individual - I think he is almost always wrong and undeservedly arrogant, but mostly sincere. Yet treating him as an AI researcher and serious philosopher (as opposed to a sci-fi essayist…

How has he fared in the fields of philosophy and AI research in terms of peer review, is there some kind of roundup or survey around about this?

EY and MIRI as a whole have largely failed to produce anything which even reaches the point of being peer reviewable. He does not have any formal education and is uninterested in learning how to navigate academia.

Re: Why are there so many rationalist cults?

#683

This article is beautifully written, and it's full of proper original research. I'm sad that most comments so far are knee-jerk "lol rationalists" type responses. I haven't seen any comment yet that isn't already addressed in much more colour and nuance in the article itself.

I think it's perfectly fine to read these articles, think "definitely a cult" and ignore whether they believe in spaceships, or demons, or AGI. The key takeaway from the article is that if you have a group leader who cuts you off from other people, that's a red flag – not really a novel, or unique, or situational insight.

> The key takeaway from the article is that if you have a group leader who cuts you off from other people, that's a red flag – not really a novel, or unique, or situational insight

Well yes and no. The reason why I think the insight is so interesting is that these groups were formed, almost definitionally for the purpose of avoiding such "obvious" mistakes. The name of the group is literally the "Rationalists"!

I find that funny, ironic, and saying something important about this philosophy, in that it implies that the rest of society wasn't so "irrational" after all.

As a more extreme and silly example, imagine there was a group called "Cults suck, and we are not a cult!", that was created for the very purpose of fighting cults, and yet, ironically, became a cult into and of itself. That would be insightful and funny.

Re: Why are there so many rationalist cults?

#684

Earlier quoted context omitted.

To be honest I don't understand that objection. If you strip it from all its culty sociological effects, one of the original ideas of rationalism was to try to use logical reasoning and statistical techniques to explicitly avoid the pitfalls of known cognitive biases. Given that foundational tenet, "rationalism" seems like an extremely appropriate moniker. I fully accept that the rationalist community may have morphe…

That it refers to a different but confusingly related concept in philosophy is a real downside of the name.

That point seems fair enough to me, as I'm not familiar with the specifics and history of the related concept in philosophy. But that seems different than the objection that calling yourself "rationalist" somehow implies you think that you have the "1 true answer" to the world's problems.

Re: Why are there so many rationalist cults?

#685

I get the impression that these people desperately want to study philosophy but for some reason can't be bothered to get formal training because it would be too humbling for them. I call it "small fishbowl syndrome," but maybe there's a better term for it.

Why would they need formal training? Can't they just read Plato, Socrates, etc, and classical lit like Dostoevsky, Camus, Kafka etc? That would be far better than whatever they're doing now.

Philosophy postgrad here, my take is: yeah, sorta, but it's hard to build your own curriculum without expertise, and it's hard to engage with subject matter fully without social discussion of, and guidance through texts.

It's the same as saying "why learn maths at university, it's cheaper just to buy and read the textbooks/papers?". That's kind of true, but I don't think that's effective for most people.

Re: Why are there so many rationalist cults?

#686
It is so strange that this article would hijack the term "rationalist" to mean this extraordinarily specific set of people "drawn together by AI researcher Eliezer Yudkowsky’s blog post series The Sequences, a set of essays about how to think more rationally".

As a counter example (with many many more people) is the Indian Rationalist Association (https://en.wikipedia.org/wiki/Indian_Rationalist_Association) to "promote scientific skepticism and critique supernatural claims". This isn't a cult of any kind, even if the members broadly agree about what it means to be rational with the set above.

Re: Why are there so many rationalist cults?

#687

Earlier quoted context omitted.

It makes a lot of sense when you realize that for many of the “leaders” in this community like Yudkowsky, their understanding of science (what it is, how it works, and its potential) comes entirely from reading science fiction and playing video games. Sad because Eli’s dad was actually a real and well-credentialed researcher at Bell Labs. Too bad he let his son quit school at an early age to be an autodidact.

I'm not at all a rationalist or a defender, but big yud has an epistemology that takes the form of the rationalist sacred text mentioned in the article (the sequences). A lot of it is well thought out, and probably can't be discarded as just coming from science fiction and video games. Yud has a great 4 hour talk with Stephen Wolfram where he holds his own.

Holding one’s own against Stephen Wolfram isn’t exactly the endorsement it might seem.

Re: Why are there so many rationalist cults?

#689
post #231

It's amphetamine. All of these people are constantly tweaking. They're annoying people to begin with, but they're all constantly yakked up and won't stop babbling. It's really obvious, I don't know why it isn't highlighted more in all these post Ziz articles.

This is one of the only comments here mentioning their drugs. These guys are juiced to the gills (on a combination of legal + prescription + illegal drugs) and doing weird shit because of it. The author even mentions the example of the polycule taking MDMA in a blackout room.

It makes me wonder whether everyone on this forum is just so loaded on antidepressants and adhd meds that they don't even find it unusual.

Re: Why are there so many rationalist cults?

#690
This is a very interesting article. It's surprising though to see it not use the term "certainty" at all. (It only uses "certain" in a couple instances of like "a certain X" and one use of "certainly" for generic emphasis.)

Most of what the article says makes sense, but it seems to sidestep the issue that a major feature distinguishing the "good" rationalists from the "bad" is that the bad ones are willing to take very extreme actions in support of their beliefs. This is not coincidentally something that distinguishes good believers in various religions or philosophies from bad believers (e.g., people who say God told them to kill people). This is also lurking in the background of discussion of those who "muddled through" or "did the best they could". The difference is not so much in the beliefs as in the willingness to act on them, and that willingness is in turn largely driven by certainty.

I think it's plausible there is a special dimension to rationalism that may exacerbate this, namely a tendency of rationalists to feel especially "proud" of their beliefs because of their meta-belief that they derived their beliefs rationally. Just like an amateur painter may give themselves extra brownie points because no one taught them how to paint, my impression of rationalists is that they sometimes give themselves an extra pat on the back for "pulling themselves up by their bootstraps" in the sense of not relying on faith or similar "crutches" to determine the best course of action. This can paradoxically increase their certainty in their beliefs when actually it's often a warning that those beliefs may be inadequately tested against reality.

I always find it a bit odd that people who profess to be rationalists can propose or perform various extreme acts, because it seems to me that one of the strongest and most useful rational beliefs is that your knowledge is incomplete and your beliefs are almost surely not as well-grounded as you think they are. (Certainly no less an exponent of reason than Socrates was well aware of this.) This on its own seems sufficient to me to override some of the most absurd "rationalist" conclusions (like that you should at all costs become rich or fix Brent Dill's depression). It's all the more so when you combine it with some pretty common-sense forecasts of what might happen if you're wrong. (As in, if you devote your life to curing Brent Dill's depression on the theory that he will then save the world, and he turns out to be just an ordinary guy or worse, you wasted your life curing one person's depression when you yourself could have done more good with your own abilities, just by volunteering at a soup kitchen or something.) It's never made sense to me that self-described rationalists could seriously consider some of these possible courses of action in this light.

Sort of related is the claim at the end that rationalists "want to do things differently from the society around them". It's unclear why this would be a rational desire. It might be rational in a sense to say you want to avoid being influenced by the society around you, but that's different from affirmatively wanting to differ from it. This again suggests a sort of "psychological greed" to reach a level of certainty that allows you to confidently, radically diverge from society, rather than accepting that you may never reach a level of certainty that allows you to make such deviations on a truly rational basis.

It's also interesting to me that the article focuses a lot not on rationalist belief per se, but on the logistics and practices of rationalist communities. This in itself seems like a warning that the rationality of rationalism is not all it's cracked up to be. It's sort of like, you can try to think as logically as possible, but if you hit yourself in the head with a hammer every day you're likely going to make mistakes anyway. And some of the "high demand" practices mentioned seem like slightly less severe psychological versions of that.

Post reply on HN