Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

Post reply on HN
Print view View original

271–280 of 947 posts

Re: Why are there so many rationalist cults?

#271

Post by MajimasEyepatch »

jl6 wrote: post #187
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

I feel this way about some of the more extreme effective altruists. There is no room for uncertainty or recognition of the way that errors compound.

- "We should focus our charitable endeavors on the problems that are most impactful, like eradicating preventable diseases in poor countries." Cool, I'm on board.

- "I should do the job that makes the absolute most amount of money possible, like starting a crypto exchange, so that I can use my vast wealth in the most effective way." Maybe? If you like crypto, go for it, I guess, but I don't think that's the only way to live, and I'm not frankly willing to trust the infallibility and incorruptibility of these so-called geniuses.

- "There are many billions more people who will be born in the future than those people who are alive today. Therefore, we should focus on long-term problems over short-term ones because the long-term ones will affect far more people." Long-term problems are obviously important, but the further we get into the future, the less certain we can be about our projections. We're not even good at seeing five years into the future. We should have very little faith in some billionaire tech bro insisting that their projections about the 22nd century are correct (especially when those projections just so happen to show that the best thing you can do in the present is buy the products that said tech bro is selling).

Re: Why are there so many rationalist cults?

#272

Post by bobson381 »

pancakemouse wrote: post #232
bobson381 wrote: post #96

Earlier quoted context omitted.

There should be an extremist cult of people who are certain only that uncertainty is the only certain thing

My favourite bumper sticker, "Militant Agnostic. I don't know, and neither do you."

I heard about this the other day! I think I need one.

Re: Why are there so many rationalist cults?

#274

Post by derektank »

inasio wrote: post #263
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

Saw once a discussion that people should not have kids as it's by far the highest increase in your carbon footprint in your lifetime (>10x than going vegan, etc) be driven all the way to advocating genocide as a way of carbon footprint minimization

Setting aside the reductio ad absurdum of genocide, this is an unfortunately common viewpoint. People really need to take into account the chances their child might wind up working on science or technology which reduces global CO2 emissions or even captures CO2. This reasoning can be applied to all sorts of naive "more people bad" arguments. I can't imagine where the world would be if Norman Borlaug's parents had decided to never have kids out of concern for global food insecurity.

Re: Why are there so many rationalist cults?

#275

Post by ehmrb »

BeFlatXIII wrote: post #259
jl6 wrote: post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

> Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some intellectual humility. Non-rationalists are forced to use their physical senses more often because they can't follow the chain of logic as far. This is to their advantage. Empiricism > rationalism.

[dead]

Re: Why are there so many rationalist cults?

#276

Post by patrickmay »

TrackerFF wrote: post #33
mlinhares wrote: post #4

> One is Black Lotus, a Burning Man camp led by alleged rapist Brent Dill, which developed a metaphysical system based on the tabletop roleplaying game Mage the Ascension. What the actual f. This is such an insane thing to read and understand what it means that i might need to go and sit in silence for the rest of the day. How did we get to this place with people going completely nuts like this?

Cult leaders tend to be narcissists. Narcissists tend to believe that they are always right, no mater what the topic is, or how knowledgeable they are. This makes them speak with confidence and conviction. Some people are very drawn to confident people. If the cult leader has other mental health issues, it can/will seep into their rhetoric. Combine that with unwavering support from loyal followers that will take ever…

If what you say is true, we're very lucky no one like that with a massive following has ever gotten into politics in the United States. It would be an ongoing disaster!

Re: Why are there so many rationalist cults?

#277

Post by marcosdumay »

JKCalhoun wrote: post #238
ar-nelson wrote: post #200

Earlier quoted context omitted.

I find Yudowsky-style rationalists morbidly fascinating in the same way as Scientologists and other cults. Probably because they seem to genuinely believe they're living in a sci-fi story. I read a lot of their stuff, probably too much, even though I find it mostly ridiculous. The biggest nonsense axiom I see in the AI-cult rationalist world is recursive self-improvement. It's the classic reason superintelligence tak…

An interesting point you make there — one would assume that if recursive self-improvement were a thing, Nature would have already lead humans into that "hall of mirrors".

Well, arguably that's exactly where we are, but machines can evolve faster.

And that's an entire new angle that the cultists are ignoring... because superintelligence may just not be very valuable.

And we don't need superintelligence for smart machines to be a problem anyway. We don't need even AGI. IMO, there's no reason to focus on that.

Re: Why are there so many rationalist cults?

#278

Post by staunton »

rglover wrote: post #116
mlinhares wrote: post #4

> One is Black Lotus, a Burning Man camp led by alleged rapist Brent Dill, which developed a metaphysical system based on the tabletop roleplaying game Mage the Ascension. What the actual f. This is such an insane thing to read and understand what it means that i might need to go and sit in silence for the rest of the day. How did we get to this place with people going completely nuts like this?

I came to comments first. Thank you for sharing this quote. Gave me a solid chuckle. I think people are going nuts because we've drifted from the dock of a stable civilization. Institutions are a mess. Economy is a mess. Combine all of that together with the advent of social media making the creation of echo chambers (and the inevitable narcissism of "leaders" in those echo chambers) effortless and ~15 years later, w…

People have been going nuts all throughout recorded history, that's really nothing new.

The only scary thing is that they have ever more power to change the world and influence others without being forced to grapple with that responsibility...

Re: Why are there so many rationalist cults?

#279

Post by dcminter »

lordnacho wrote: post #211
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

> I immediately become suspicious of anyone who is very certain of something Me too, in almost every area of life. There's a reason it's called a conman: they are tricking your natural sense that confidence is connected to correctness. But also, even when it isn't about conning you, how do people become certain of something? They ignored the evidence against whatever they are certain of. People who actually know what…

In the term "conman" the confidence in question is that of the mark, not the perpetrator.
Post reply on HN