Why are there so many rationalist cults? • Hacker News Board

Re: Why are there so many rationalist cults?

#691

Post by morpheos137 » Wed, Aug 13, 2025, 5:44 AM UTC

1attice wrote: post #393

analog31 wrote: post #262

Earlier quoted context omitted.

Perhaps part of being rational, as opposed to rationalist, is having a sense of when to override the conclusions of seemingly logical arguments.

In philosophy grad school, we described this as 'being reasonable' as opposed to 'being rational'. That said, big-R Rationalism (the Lesswrong/Yudkowsky/Ziz social phenomenon) has very little in common with what we've standardly called 'rationalism'; trained philosophers tend to wince a little bit when we come into contact with these groups (who are nevertheless chockablock with fascinating personalities and compelli…

I am interested in your journey from philosophy to coding.

Re: Why are there so many rationalist cults?

#692

Post by elliotto » Wed, Aug 13, 2025, 5:45 AM UTC

eschaton wrote: post #687

elliotto wrote: post #680

Earlier quoted context omitted.

I'm not at all a rationalist or a defender, but big yud has an epistemology that takes the form of the rationalist sacred text mentioned in the article (the sequences). A lot of it is well thought out, and probably can't be discarded as just coming from science fiction and video games. Yud has a great 4 hour talk with Stephen Wolfram where he holds his own.

Holding one’s own against Stephen Wolfram isn’t exactly the endorsement it might seem.

I'm interested in this perspective, I haven't come across much criticism of Wolfram but I haven't really looked for it much either. Is it because of his theory of everything ruliad stuff?

I really enjoy his blog posts and his work on automata seems to be well respected. I've felt he presents a solid epistemology.

Re: Why are there so many rationalist cults?

#693

Post by seanhunter » Wed, Aug 13, 2025, 5:51 AM UTC

skybrian wrote: post #541

pavlov wrote: post #398

A very interesting read. My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons. This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now. The article see…

The WHO didn't declare a global pandemic until March 11, 2020 [1]. That's a little slow and some rationalists were earlier than that. (Other people too.) After reading a warning from a rationalist blog, I posted a lot about COVID news to another forum and others there gave me credit for giving the heads-up that it was a Big Deal and not just another thing in the news. (Not sure it made all that much difference, thoug…

Do you think that the consequences of the WHO declaring a pandemic and some rationalist blog warning about covid are the same? Clearly the WHO has to be more cautious. I have no doubt there were people at the WHO who felt a global pandemic was likely at least as early as you and the person writing the rationalist blog.

Re: Why are there so many rationalist cults?

#694

Post by s1artibartfast » Wed, Aug 13, 2025, 6:05 AM UTC

guerrilla wrote: post #417

> One way that thinking for yourself goes wrong is that you realize your society is wrong about something, don’t realize that you can’t outperform it, and wind up even wronger. I've been there myself. > And without the steadying influence of some kind of external goal you either achieve or don’t achieve, your beliefs can get arbitrarily disconnected from reality — which is very dangerous if you’re going to act on the…

>I think this and the entire previous two paragraphs preceding it are excellent arguments for philosophical pragmatism and empiricism. It's strange to me that the community would not have already converged on that after all their obsessions with decision theory

They did! One of the great ironies inside the community is that they are and openly admit to being empiricists. They reject most of the French/European rationalist cannon.

>Why are rationalists not able to write less?

The answer is a lot more boring. They like to write and they like to think. They also think by writing. It is written as much for themselves as for anyone else, probably more.

Re: Why are there so many rationalist cults?

#695

Post by com2kid » Wed, Aug 13, 2025, 6:10 AM UTC

prepend wrote: post #230

JTbane wrote: post #20

Earlier quoted context omitted.

I don't know how you can call yourself a "rationalist" and base your worldview on a fantasy game.

I mean, is it a really good game? I’ve never played, but now I’m kind of interesting.

I ran a long running campaign, it is pretty fun. The game books were obviously written by artists and no mathematician was involved, some of the rules are very broken (may have been fixed in later revisions).

The magic system is very fun and gives players complete freedom to come up with spells on the fly. The tl;dr is there aren't pre-made spells, you have spheres you have learned, and you can combine those spheres of magic however you want. So if someone has matter and life, reaching into someone's chest and pulling out their still beating heart would be a perfectly fine thing for a brand new character to be able to do. (Of course magic has downsides, reality doesn't like being bent and it will snap back with violent force is coerced too hard!)

The books are laid out horribly, there isn't a single set of tables to refer to, you have to post it note bookmark everything. Picking up and playing the rules are really simple, the number of dots you have in attributes + skill is how many d10 dice you get to roll for a check. 8+ is a success, and you can reroll 10s. 90% of the game is as simple as that, but then there are like 5 pages of rules for grappling including basically a complete breakdown of wrestling moves and gaining position, but feel free to just ignore those.

Re: Why are there so many rationalist cults?

#696

Post by solid_fuel » Wed, Aug 13, 2025, 6:14 AM UTC

Terr_ wrote: post #376

JKCalhoun wrote: post #238

Earlier quoted context omitted.

An interesting point you make there — one would assume that if recursive self-improvement were a thing, Nature would have already lead humans into that "hall of mirrors".

I often like to point out that Earth was already consumed by Grey Goo, and today we are hive-minds in titanic mobile megastructure-swarms of trillions of the most complex nanobots in existence (that we know of), inheritors of tactics and capabilities from a zillion years of physical and algorithmic warfare. As we imagine the ascension of AI/robots, it may seem like we're being humble about ourselves... But I think it…

To take it a little further - if you stretch the conventional definition of intelligence a bit - we already assemble ourselves into a kind of collective intelligence.

Nations, corporations, clubs, communes -- any functional group of humans is capable of observing, manipulating, and understanding our environment in ways no individual human is capable of. When we dream of hive minds and super-intelligent AI it almost feels like we are giving up on collaboration.

Re: Why are there so many rationalist cults?

#697

Post by vintermann » Wed, Aug 13, 2025, 6:14 AM UTC

throw4847285 wrote: post #635

Dylan16807 wrote: post #596

Earlier quoted context omitted.

Listening to other viewpoints is hard. Restating is a good tool to improve listening and understanding. I don't agree with this criticism at all, since that "prodigious intellect" bit isn't inherent to the term.

I was being snarky, but I think steelmanning does have one major flaw. By restating the argument in terms that are most convincing to you, you may already be warping the conclusions of your interlocutor to fit what you want them to be saying. Charity is, "I will assume this person is intelligent and overlook any mistakes in order to try and understand what they are actually communicating." Steelmanning is "I can make…

One of the most annoying habits of Rationalists, and something that annoyed me with plenty of people online before Yudkowsky's brand was even a thing, is the assumption that they're much smarter than almost everyone else. If that is your true core belief, the one that will never be shaken, then of course you're not going to waste time trying to understand the nuances of the arguments of some pious medieval peasant.

Re: Why are there so many rationalist cults?

#698

Post by com2kid » Wed, Aug 13, 2025, 6:16 AM UTC

TimorousBestie wrote: post #11

mlinhares wrote: post #4

> One is Black Lotus, a Burning Man camp led by alleged rapist Brent Dill, which developed a metaphysical system based on the tabletop roleplaying game Mage the Ascension. What the actual f. This is such an insane thing to read and understand what it means that i might need to go and sit in silence for the rest of the day. How did we get to this place with people going completely nuts like this?

Mage: The Ascension is basically a delusions of grandeur simulator, so I can see how an already unstable personality might get attached to it and become more unstable.

The magic system is amazing though, best I've played in any game. Easy to use, role play heavy, and it lets players go wild with ideas, but still reins in their crazier impulses.

Mage: The Awakening is a minor rules revision to the magic system, but the lore is super boring in comparison IMHO. It is too wishy washy.

Ascension has tons of cool source material, and White Wolf ended up tying all their properties together into one grand finale story line. That said it is all very 90s cringe in retrospect, but if you are willing to embrace the 90s retro feel, it is still fun.

Awakening's lore never drew me in, the grand battle just isn't there. So many shades of grey is is damn near technicolor.

Re: Why are there so many rationalist cults?

#699

Post by godelski » Wed, Aug 13, 2025, 6:23 AM UTC

Dylan16807 wrote: post #598

godelski wrote: post #469

Earlier quoted context omitted.

Definitely. Real world adds lots of complexities and nuances, but I was just trying to make the point that it matters how those inferences compound. That we can't just conclude that compounding inferences decreases likelihood

Well they were talking about a chain , A->B, B->C, C->D. You're talking about multiple pieces of evidence for the same statement. Your tests don't depend on any of the previous tests also being right.

Be careful with your description there, are you sure it doesn't apply to the Bayesian example (which was... illustrative...? And not supposed to be every possible example?)? We calculated f(f(f(x))), so I wouldn't say that this "doesn't depend on the previous 'test'". Take your chain, we can represent it with h(g(f(x))) (or (f∘g∘h)(x)). That clearly fits your case for when f=g=h. Don't lose sight of the abstractions.

Re: Why are there so many rationalist cults?

#700

Post by solid_fuel » Wed, Aug 13, 2025, 6:24 AM UTC

doubleunplussed wrote: post #476

tempfile wrote: post #293

Earlier quoted context omitted.

I honestly am not trying to be rude when I say this, but this is exactly the sort of speculation I find problematic and that I think most people in this thread are complaining about. Being able to tell Claude to have a go has no relation at all to whether it may ever succeed, and you don't actually address any of the legitimate concerns the comment you're replying to points out. There really isn't anything in this co…

On the other hand, I'm baffled to encounter recursive self-improvement being discussed as something not only weird to expect, but as damning evidence of sloppy thinking by those who speculate about it. We have an existence proof for intelligence that can improve AI: humans. If AI ever gets to human-level intelligence, it would be quite strange if it couldn't improve itself. Are people really that sceptical that AI wi…

> We have an existence proof for intelligence that can improve AI: humans.

I don't understand what you mean by this. The human brain has not meaningfully changed, biologically, in the past 40,000 years.

We, collectively, have built a larger base of knowledge and learned to cooperate effectively enough to make large changes to our environment. But that is not the same thing as recursive self-improvement. No one has been editing our genes or performing brain surgery on children to increase our intelligence or change the fundamental way it works.

Modern brains don't work "better" than those of ancient humans, we just have more knowledge and resources to work with. If you took a modern human child and raised them in the middle ages, they would behave like everyone else in the culture that raised them. They would not suddenly discover electricity and calculus just because they were born in 2025 instead of 950.

----

And, if you are talking specifically about the ability to build better AI, we haven't matched human intelligence yet and there is no indication that the current LLM-heavy approach will ever get there.