Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

361–370 of 947 posts

Re: Why are there so many rationalist cults?

#361

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

There are certain things I am sure of even though I derived them on my own.

But I constantly battle tested them against other smart people’s views, and just after I ran out of people to bring me new rational objections did I become sure.

Now I can battle test them against LLMs.

On a lesser level of confidence, I have also found a lot of times the people who disagreed with what I thought had to be the case, later came to regret it because their strategies ended up in failure and they told me they regretted not taking my recommendation. But that is on an individual level. I have gotten pretty good at seeing systemic problems, architecting systemic solutions, and realizing what it would take to get them adopted to at least a critical mass. Usually, they fly in the face of what happens normally in society. People don’t see how their strategies and lives are shaped by the technology and social norms around them.

Here, I will share three examples:

Public Health: https://www.laweekly.com/restoring-healthy-communities/

Economic and Governmental: https://magarshak.com/blog/?p=362

Wars & Destruction: https://magarshak.com/blog/?p=424

For that last one, I am often proven somewhat wrong by right-wing war hawks, because my left-leaning anti-war stance is about avoiding inflicting large scale misery on populations, but the war hawks go through with it anyway and wind up defeating their geopolitical enemies and gaining ground as the conflict fades into history.

Re: Why are there so many rationalist cults?

#362
post #112

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

> I immediately become suspicious of anyone who is very certain of something, especially if they derived it on their own. Are you certain about this?

Well you could be a critical rationalist and do away with the notion of "certainty" or any sort of justification or privileged source of knowledge (including "rationality").

Re: Why are there so many rationalist cults?

#364

I find it ironic that the question is asked unempirically. Where is the data stating there are many more than before? Start there, then go down the rabbit hole. Otherwise, you're concluding on something that may not be true, and trying to rationalize the answer, just as a cultist does.

Oh come on.

Anyone who's ever seen the sky knows it's blue. Anyone who's spent much time around rationalism knows the premise of this article is real. It would make zero sense to ban talking about about a serious and obvious problem in their community until some double blind peer reviewed data can be gathered.

It would be what they call an "isolated demand for rigor".

Re: Why are there so many rationalist cults?

#366

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

I find Yudowsky-style rationalists morbidly fascinating in the same way as Scientologists and other cults. Probably because they seem to genuinely believe they're living in a sci-fi story. I read a lot of their stuff, probably too much, even though I find it mostly ridiculous. The biggest nonsense axiom I see in the AI-cult rationalist world is recursive self-improvement. It's the classic reason superintelligence tak…

> - What if there are several paths to different kinds of intelligence with their own local maxima, in which the AI can easily get stuck after optimizing itself into the wrong type of intelligence?

I think what's more plausible is that there is general intelligence, and humans have that, and it's general in the same sense that Turing machines are general, meaning that there is no "higher form" of intelligence that has strictly greater capability. Computation speed, memory capacity, etc. can obviously increase, but those are available to biological general intelligences just like they would be available to electronic general intelligences.

Re: Why are there so many rationalist cults?

#367
post #299
post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

Yet I think most people err in the other direction. They 'know' the basics of health, of discipline, of charity, but have a hard time following through. 'Take a simple idea, and take it seriously': a favorite aphorism of Charlie Munger. Most of the good things in my life have come from trying to follow through the real implications of a theoretical belief.

And “always invert”! A related mungerism.

Re: Why are there so many rationalist cults?

#368
post #231

It's amphetamine. All of these people are constantly tweaking. They're annoying people to begin with, but they're all constantly yakked up and won't stop babbling. It's really obvious, I don't know why it isn't highlighted more in all these post Ziz articles.

Who's writing them?

Re: Why are there so many rationalist cults?

#369

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

There should be an extremist cult of people who are certain only that uncertainty is the only certain thing

https://realworldrisk.com/

Re: Why are there so many rationalist cults?

#370
post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

> I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. I really like your way of putting it. It’s a fundamental fallacy to assume certainty when trying to pre…

Deep Space 9 had an episode dealing with something similar. Superintelligent beings determine that a situation is hopeless and act accordingly. The normal beings take issue with the actions of the Superintelligents. The normal beings turn out to be right.
Post reply on HN