Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

Post reply on HN
Print view View original

241–250 of 947 posts

Re: Why are there so many rationalist cults?

#241

Post by kergonath »

danaris wrote: post #197
jl6 wrote: post #187

Earlier quoted context omitted.

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

> I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Yeah, this is a pattern I've seen a lot of recently—especially in discussions about LLMs and the supposed inevitability of AGI (and the Singularity). This is a go…

Another annoying one is the simulation theory group. They know just enough about Physics to build sophisticated mental constructs without understanding how flimsy the foundations are or how their logical steps are actually unproven hypotheses.

Re: Why are there so many rationalist cults?

#242

Post by JKCalhoun »

ambicapter wrote: post #207
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

One of the only idioms that I don't mind living my life by is, "Follow the truth-seeker, but beware those who've found it".

Interesting. I can't say I've done much following though — not that I am aware of anyway. Maybe I just had no leaders growing up.

Re: Why are there so many rationalist cults?

#243

Post by michaeldoron »

o11c wrote: post #191
saghm wrote: post #132

Earlier quoted context omitted.

Without speaking for religions I'm not familiar with, I grew up Catholic, and one of the most important Catholic beliefs is that during Mass, the bread (i.e. "communion" wafers) and wine quite literally transform into the body and blood of Jesus, and that eating and drinking it is a necessary ritual to get into heaven[1], which was a source of controversy even back as far as the Protestant Reformation, with some sect…

The "bears" story reads a lot more sensibly if you translated it correctly as "a gang of thugs tries to bully Elisha into killing himself." Still reliant on the supernatural, but what do you expect from such a book?

Where do you see that in the text? I am looking at the Hebrew script, and the text only reads that as Elisha went up a path, young lads left the city and mocked him by saying "get up baldy", and he turned to them and cursed them to be killed by two she bears. I don't think saying "get up baldy" to a guy walking up a hill constitutes bullying him into killing himself.

Re: Why are there so many rationalist cults?

#244

Post by akomtu »

It's a religion of an overdeveloped mind that hides from everything it cannot understand. It's an anti-religion, in a sense, that puts your mind on the pedestal.

Note the common pattern in major religions: they tell you that thoughts and emotions obscure the light of intuition, like clouds obscure sunlight. Rationalism is the opposite: it denies the very idea of intuition, or anything above the sphere of thoughts, and tells to create as many thoughts as possible.

Rationalists deny anything spiritual, good or evil, because they don't have evidence to think otherwise. They remain in this state of neutral nihilism until someone bigger than them sneaks into their ranks and casually introduces them to evil with some undeniable evidence. Their minds quickly pass the denial-anger-acceptance stages and being faithful to their rationalist doctrine they update their beliefs with what they know. From that point they are a cult. That's the story of Scientology, which has too many many parallels with Rationalism.

Re: Why are there so many rationalist cults?

#245

Post by randcraw »

nathan_compton wrote: post #12

Thinking too hard about anything will drive you insane but I think the real issue here is that rationalists simply over-estimate both the power of rational thought and their ability to do it. If you think of people who tend to make that kind of mistake you can see how you get a lot of crazy groups. I guess I'm a radical skeptic, secular humanist, utilitarianish sort of guy, but I'm not dumb enough to think throwing a…

The second-most common cognitive mistake we make has to be the failure to validate what we think we know -- is it actually true? The crux of being right isn't reasoning. It's avoiding dumb blunders based on falsehoods, both honest and dishonest. In today's political and media climate, I'd say dishonest falsehoods are a far greater cause for being wrong than irrationality.

Re: Why are there so many rationalist cults?

#246

Post by skrebbel »

This article is beautifully written, and it's full of proper original research. I'm sad that most comments so far are knee-jerk "lol rationalists" type responses. I haven't seen any comment yet that isn't already addressed in much more colour and nuance in the article itself.

Re: Why are there so many rationalist cults?

#247

Post by mitthrowaway2 »

JKCalhoun wrote: post #235

> Many of them also expect that, without heroic effort, AGI development will lead to human extinction. Odd to me. Not biological warfare? Global warming? All-out nuclear war? I guess The Terminator was a formative experience for them. (For me perhaps it was The Andromeda Strain .)

These aren't mutually exclusive. Even in The Terminator, Skynet's method of choice is nuclear war. Yudkowsky frequency expressses concern that a malevolent AI might synthesize a bioweapon. I personally worry that destroying the ozone layer might be an easy opening volley. Either way, I don't want a really smart computer spending its time figuring out plans to end the human species, because I think there are too many ways to be successful.

Re: Why are there so many rationalist cults?

#248

Post by jacquesm »

afpx wrote: post #195
glenstein wrote: post #56

Earlier quoted context omitted.

I personally (for better or worse) became familiar with Ayn Rand as a teenager, and I think Objectivism as a kind of extended Ayn Rand social circle and set of organizations has faced the charge of cultish-ness, and that dates back to, I want to say, the 70s and 80s at least. I know Rand wrote much earlier than that, but I think the social and organizational dynamics unfolded rather late in her career.

Her books were very popular with the gifted kids I hung out with in the late 80s. Cool kids would carry around hardback copies of Atlas Shrugged, impressive by the sheer physical size and art deco cover. How did that trend begin?

By setting up the misfits in a revenge of the nerds scenario?

Ira Levin did a much better job of it and showed what it would lead to but his 'This Perfect Day' did not - predictably - get the same kind of reception as Atlas Shrugged did.

Re: Why are there so many rationalist cults?

#249

Post by jacquesm »

prepend wrote: post #229
afpx wrote: post #195

Earlier quoted context omitted.

Her books were very popular with the gifted kids I hung out with in the late 80s. Cool kids would carry around hardback copies of Atlas Shrugged, impressive by the sheer physical size and art deco cover. How did that trend begin?

People reading the book and being into it and telling other people. It’s also a hard book to read so it may be smart kids trying to signal being smart.

The only thing that makes it hard to read is the incessant soap-boxing by random characters. I have a rule that if I start a book I finish it but that one had me tempted.

Re: Why are there so many rationalist cults?

#250

Post by tim333 »

ar-nelson wrote: post #200
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

I find Yudowsky-style rationalists morbidly fascinating in the same way as Scientologists and other cults. Probably because they seem to genuinely believe they're living in a sci-fi story. I read a lot of their stuff, probably too much, even though I find it mostly ridiculous. The biggest nonsense axiom I see in the AI-cult rationalist world is recursive self-improvement. It's the classic reason superintelligence tak…

I've pondered recursive self-improvement. I'm fairly sure it will be a thing - we're at a point already where people could try telling Claude or some such to have a go, even if not quite at a point it would work. But I imagine take off would be very gradual. It would be constrained by available computing resources and probably only comparably good to current human researchers and so still take ages to get anywhere.
Post reply on HN