Why are there so many rationalist cults?

To play devils advocate, you could be seen as trying to subjugate the worlds health to your own economic well-being, and far fewer people are concerned with your tax bracket than there are people on earth. In a pure democracy, I'm fairly certain the planets well being would be deemed more important than the economy of whatever nation you live in.

> advocating for focusing on one... is primarily following a religion

Maybe, but they could also just be doing the risk calculus a bit differently. If you are a many step thinker the long term fecundity of our species might feel more important than any level of short term financial motivation.

The key takeaway from the article is that if you have a group leader who cuts you off from other people, that's a red flag – not really a novel, or unique, or situational insight.

Some meta-commentary first... How would one go about testing if this is true? If true, then such "promises" are not written down -- they are implied. So one would need to ask at least two questions: 1. Did the author intend to make these implicit promises? 2. What portion of readers perceive them as such?

> ... There is an art of thinking better ...

First, this isn't _implicit_ in the Sequences; it is stated directly. In any case, the quote does not constitute a promise: so far, it is a claim. And yes, rationalists do think there are better and worse ways of thinking, in the sense of "what are more effective ways of thinking that will help me accomplish my goals?"

> ..., and we’ve figured it out.

Codswallop. This is not a message of the rationality movement -- quite the opposite. We share what we've learned and why we believe it to be true, but we don't claim we've figured it all out. It is better to remain curious.

> If you learn it, you can solve all your problems...

Bollocks. This is not claimed implicitly or explicitly. Besides, some problems are intractable.

> ... become brilliant and hardworking and successful and happy ...

Rubbish.

> ..., and be one of the small elite shaping not only society but the entire future of humanity.

Bunk.

For those who haven't read it, I'll offer a relevant extended quote from Yudkowsky's 2009 "Go Forth and Create the Art!" [1], the last post of the Sequences:

## Excerpt from Go Forth and Create the Art

But those small pieces of rationality that I've set out... I hope... just maybe...

I suspect—you could even call it a guess—that there is a barrier to getting started, in this matter of rationality. Where by default, in the beginning, you don't have enough to build on. Indeed so little that you don't have a clue that more exists, that there is an Art to be found. And if you do begin to sense that more is possible—then you may just instantaneously go wrong. As David Stove observes—I'm not going to link it, because it deserves its own post—most "great thinkers" in philosophy, e.g. Hegel, are properly objects of pity. That's what happens by default to anyone who sets out to develop the art of thinking; they develop fake answers.

When you try to develop part of the human art of thinking... then you are doing something not too dissimilar to what I was doing over in Artificial Intelligence. You will be tempted by fake explanations of the mind, fake accounts of causality, mysterious holy words, and the amazing idea that solves everything.

It's not that the particular, epistemic, fake-detecting methods that I use, are so good for every particular problem; but they seem like they might be helpful for discriminating good and bad systems of thinking.

I hope that someone who learns the part of the Art that I've set down here, will not instantaneously and automatically go wrong, if they start asking themselves, "How should people think, in order to solve new problem X that I'm working on?" They will not immediately run away; they will not just make stuff up at random; they may be moved to consult the literature in experimental psychology; they will not automatically go into an affective death spiral around their Brilliant Idea; they will have some idea of what distinguishes a fake explanation from a real one. They will get a saving throw.

It's this sort of barrier, perhaps, which prevents people from beginning to develop an art of rationality, if they are not already rational.

And so instead they... go off and invent Freudian psychoanalysis. Or a new religion. Or something. That's what happens by default, when people start thinking about thinking.

I hope that the part of the Art I have set down, as incomplete as it may be, can surpass that preliminary barrier—give people a base to build on; give them an idea that an Art exists, and somewhat of how it ought to be developed; and give them at least a saving throw before they instantaneously go astray.

That's my dream—that this highly specialized-seeming art of answering confused questions, may be some of what is needed, in the very beginning, to go and complete the rest.

[1]: https://www.lesswrong.com/posts/aFEsqd6ofwnkNqaXo/go-forth-a...

When you're uncertain about a topic, you can explore it by writing a lot about said topic. Ideally, when you've finished exploring and studying a topic, you should be able to write a much more condensed / synthesized version.