Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

841–850 of 947 posts

Re: Why are there so many rationalist cults?

#841
post #398

A very interesting read. My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons. This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now. The article see…

> Rationalists came to correct views about the COVID-19 pandemic while many others were saying masks didn’t work I wonder what views about covid-19 are correct. On masks, I remember the mainstream messaging went through the stages that were masks don't work, some masks work, all masks work, double masking works, to finally masks don't work (or some masks work; I can't remember where we ended up).

Putting just about anything in front of your face will help prevent spreading illness to some extent, this is why we teach children to "vampire cough". Masks were always effective to some degree. The CDC lied to the public by initially telling them not to use masks because they wanted to keep the supply for healthcare workers and they were afraid that the pubic would buy them all up first. It was a very very stupid thing to do and it undermined people's trust in the CDC and confused people about masks. After that masks became politicized and the whole topic became a minefield.

Re: Why are there so many rationalist cults?

#842

Earlier quoted context omitted.

So in your example you can apply just one test result at a time, in any order. And the more pieces of evidence you apply, the stronger your argument gets. f = "The test(s) say the patient is a vampire, with a .01 false positive rate." f∘f∘f = "The test(s) say the patient is a vampire, with a .000001 false positive rate." In the chain example f or g or h on its own is useless. Only f∘g∘h is relevant. And f∘g∘h is a lo…

Again, my only argument was that you can have both situations occur. We could still construct a f∘g∘h to increase probability if we want. I'm not saying it cannot go down, I'm saying there's no absolute rule you can follow.

I don't think you can make a chain of logic f∘g∘h where the probability of the combined function is higher than the probability of f or g or h on their own.

Chain of logic meaning that only the last function updates the probability you care about, and the preceeding functions give you intermediate information that is only useful to feed into the next function.

It is an absolute rule you can follow, as long as you're applying it the way it was intended, to a specific organization of functions. It's not any kind of combining, it's A->B->C->D combining. As opposed to multiple pieces that each independently imply D.

Just because you can use ∘ in both situations doesn't make them the same. Whether x∘y∘z is chaining depends on what x and y and z do. If all of them update the same probability, that's not chaining. If removing any of them would leave you with no information about your target probability, then it's chaining.

TL;DR: ∘ doesn't tell you if something is a chain, you're conflating chains with non-chains, the rule is useful when it comes to chains

Re: Why are there so many rationalist cults?

#843
post #727
post #715

Earlier quoted context omitted.

> The "they" you are describing is a large body of disparate people spread around the world. [Citation needed] I sincerely doubt anything but a tiny insignificant minority consider themselves part of the "rationalist community".

"Large" is very vague. The leaderboard shows (50 of) 166385 registered accounts* on https://www.lesswrong.com/leaderboard This is simultaneously a large body and an insignificant minority. * How many are junk accounts? IDK. But I do know it's international, because I live in Berlin, Germany, and socialise regularly.

> But I do know it's international, because I live in Berlin, Germany, and socialise regularly.

This made me chuckle. Thanks!

Re: Why are there so many rationalist cults?

#844

Earlier quoted context omitted.

> to finally masks don't work (or some masks work; I can't remember where we ended up). Most masks 'work', for some value of 'work', but efficacy differs (which, to be clear, was ~always known; there was a very short period when some authorities insisted that covid was primarily transmitted by touch, but you're talking weeks at most). In particular I think what confused people was that the standard blue surgical mask…

> In particular I think what confused people was that the standard blue surgical masks are somewhat effective at stopping an infected person from passing on covid (and various other things), but not hugely effective at preventing the wearer from contracting covid Yes, exactly. If we look at guidelines about influenza, we will see them say that "surgical masks are not considered adequate respiratory protection for air…

And any protection is better than no protection, even if it only reduces your chance by 30% or whatever - it's unlikely to make things worse.

Re: Why are there so many rationalist cults?

#845
post #715

Earlier quoted context omitted.

> The "they" you are describing is a large body of disparate people spread around the world. [Citation needed] I sincerely doubt anything but a tiny insignificant minority consider themselves part of the "rationalist community".

I realized a few years ago that there's an important difference between someone who believes women should have equal rights and a feminist. Similarly, there's a difference between someone who believes men should have equal rights and a men's rights advocate. I often sympathize with the first group. I often disagree with the latter. This same distinction applies to rationality: there's a huge difference between someon…

> This same distinction applies to rationality: there's a huge difference between someone who strives to be rational and someone who belongs to a "rationalist community".

I consider myself a member of both these groups, and I fully agree. Many people are in the rationality community only because they like the vibes, not because they actually try to practice the art. It probably can't be avoided -- the community is open to newcomers and has no formal membership.

Re: Why are there so many rationalist cults?

#846
post #706

Earlier quoted context omitted.

The article specifically defines the rationalists it’s talking about: “The rationalist community was drawn together by AI researcher Eliezer Yudkowsky’s blog post series The Sequences, a set of essays about how to think more rationally.” Is this really a large body of disparate people spread around the world? I suspect not.

Define "large" : https://www.astralcodexten.com/p/fall-meetups-everywhere-cal... (One of the largest subgroups AFAIK ?) (They still seem to be leaning heavily USA-based, and in particular California-based.)

Maybe a better link would be to the previous wave of meetups, because some places join the event at the last moment:

https://www.astralcodexten.com/p/meetups-everywhere-spring-2...

Re: Why are there so many rationalist cults?

#847
post #662

I have a lot of experience with rationalists. What I will say is: 1) If you have a criticism about them or their stupid name or how "'all I know is that I know nothing' how smug of them to say they're truly wise," rest assured they have been self flagellating over these criticisms 100x longer than you've been aware of their group. That doesn't mean they succeeded at addressing the criticisms, of course, but I can tel…

A lot of terrible people are self-aware, well-read and ultimately concerned with doing good. All of the catastrophes of the 20th century were led by men that fit this description: Stalin, Mao, Hitler. Perhaps this is a bit hyperbolic, but the troubling belief that the Rationalists have in common with these evil men is the ironclad conviction that self-awareness, being well-read, and being concerned with good, somehow…

IMO we can’t define hubris anymore. The concept has become foreign.

Re: Why are there so many rationalist cults?

#848

Earlier quoted context omitted.

> For the "AI will be [human to ant difference] more intelligent than us" argument to follow it would require us to be pretty fucking dumb, and in that case we're pretty fucking dumb and it'd be silly to think we can make these types of predictions with reasonable accuracy (also true in the unbounded case!). ...which is why we should be careful not to rush full-speed ahead and develop AI before we can predict how it…

> you are assuming that intelligence will necessarily and inherently lead to (good) morality Please read before responding. I said no such thing. I even said there are bad smart people. I only argued that a person's goodness is orthogonal to their intelligence. But I absolutely did not make an assumption that intelligence equates to good. I said it was irrelevant...

Idk, you certainly seemed to be implying that especially in your earlier comment. I would agree that it is orthogonal, I would think most rationalists would, too.

Re: Why are there so many rationalist cults?

#849

Earlier quoted context omitted.

I apologize for the tone of my comment, but this is how I read your arguments (I was a little drunk at the time): 1. future AI cannot be infinitely intelligent, therefore AI is safe But even with our level of intelligence, if we get serious we can eliminate all humans. 2. some smart ppl I know are peaceful Do you think Putin is dumb? 3. smart ppl have different preferences than other ppl therefore AI is safe Ironical…

> I was a little drunk at the time Honestly it still sounds like you are. You've still misread my comment and think I said there can't be bad smart people. I made no such argument, I argued that intelligence isn't related to goodness.

If that was what you meant to say though, you've gotta admit that opening a paragraph with "The other weird assumption I hear is about how it'll just kill us all", and then spending the rest of the paragraph giving examples of the peacefulness of smart people, is not the most effective strategy of communicating that.

Re: Why are there so many rationalist cults?

#850
post #398

A very interesting read. My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons. This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now. The article see…

As someone who started reading without knowing about rationalists, I actually came out without knowing much more. Lots of context is assumed I guess. Some main figures and rituals are mentioned but I still don’t know how the activities and communities arise from the purported origin. How do we go from “let’s rationally analyze how we think and get rid of bias” to creating a crypto, or being hype focused on AI, or sum…

LessWrong was originally a personal blog of Eliezer Yudkowsky. It was an inspiration for what later became the "rationality community". These days, LessWrong is a community blog. The original articles were published as a book, freely available at: https://www.readthesequences.com/ If you read it, you can see what the community was originally about; but it is long.

Some frequent topics debated on LessWrong are AI safety, human rationality, effective altruism. But it has no strict boundaries; some people even post about their hobbies or family life. Debating politics is discouraged, but not banned. The website is mostly moderated by its users, by voting on articles and comments. The voting is relatively strict, and can be scary for many newcomers. (Maybe it is not strategic to say this, but most comments on Hacker News would probably be downvoted on LessWrong for insufficient quality.)

Members of the community, the readers of the website, are all over the planet. (Just what you would expect from readers of an internet forum.) But in some cities there are enough of them so they can organize an offline meetup once in a while. And if a very few cities, there are so many of them, that they are practically a permanent offline community; most notably in the Bay Area.

I don't live in the Bay Area. To describe how the community functions in my part of the world: we meet about once in a month, sometimes less frequently, and we discuss various nerdy stuff. (Apologies if this is insufficiently impressive. From my perspective, the quality of those discussions is much higher than I have seen anywhere else, but I guess there is no way to provide this experience second-hand.) There is a spirit of self-improvement; we encourage each other to think logically and try to improve our lives.

Oh, and how does the bad part connect to it?

Unfortunately, although the community is about trying to think better, for some reason it also seems very attractive for people who are looking for someone to tell them how to think. (I mean, we do tell them how to think, but in a very abstract way: check the evidence, remember your cognitive biases, et cetera.) They are a perfect material for a cult.

The rationality community itself is not a cult. Too much disagreement and criticism of our own celebrities for that! There is also no formal membership; anyone is free to come and go. Sometimes a wannabe cult leader joins the community, takes a few vulnerable people aside, and starts a small cult. Two out of three examples in the article, it was a group of about five people -- when you have hundreds of members in a city, you won't notice when five of them start attending your meetups less frequently, and then disappear completely. And one day... you read about them in the newspapers.

> How do we go from “let’s rationally analyze how we think and get rid of bias” to creating a crypto, or being hype focused on AI, or summoning demons? Why did they raise this idea of matching confrontation always with escalation?

Rationality and AI have always been the focus of the community. Buying cryptos was considered common sense back then when Bitcoin was cheap; but I haven't heard talking about cryptos in the rationality community recently.

On the other hand, believing in demons, and the idea that you should always escalate... those are specific ideas of the leaders of the small cults, definitely not shared by the rest of the community.

Notice how the first things the wannabe cult leaders do is isolate their followers even from the rest of the rationality community. They are quite aware that what they are doing would be considered wrong by the rest of the community.

The question is, how can the community prevent this? If your meetings are open for everyone, how can you prevent one newcomer from privately contacting a few other newcomers, meeting them in private, and brainwashing them? I don't have a good answer for that.

Post reply on HN