Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My falling-out with the rationalist community

lcamtuf.substack.com

Post reply on HN
Print view View original

71–80 of 175 posts

Re: My falling-out with the rationalist community

#71

Post by zahlman »

twic wrote: post #12
vovavili wrote: post #2

This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.

It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less. [1] https://lydialaurenson.substack.com/p/the-inside-story-of-le... [2] https://asteriskmag.com/issues/…

Asking "why are there so many rationalist cults?" (as the URL does) in this context is kind of like asking "why are there so many mathematical cranks?", and then holding forth that this somehow demonstrates a fundamental problem with mathematics.

Re: My falling-out with the rationalist community

#72

Post by cindyllm »

zahlman wrote: post #71
twic wrote: post #12

Earlier quoted context omitted.

It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less. [1] https://lydialaurenson.substack.com/p/the-inside-story-of-le... [2] https://asteriskmag.com/issues/…

Asking "why are there so many rationalist cults?" (as the URL does) in this context is kind of like asking "why are there so many mathematical cranks?", and then holding forth that this somehow demonstrates a fundamental problem with mathematics.

[dead]

Re: My falling-out with the rationalist community

#73

Post by zahlman »

cogman10 wrote: post #68
zahlman wrote: post #61

Earlier quoted context omitted.

Okay, but those pieces don't establish your claim. You would need a Roko's basilisk-like concept that only 5% of the community agree with, and a "group meetup" or "conference" where "they all" (i.e.: also the 95% ) "proclaim that their beliefs (i.e.: including the basilisk thing ) are true". If I observed such a gathering knowing that 5% of the people there have sincerely come to a particular conclusion based on the…

Which claim exactly did I make which you are refuting? Please actually quote it. > I would be extremely surprised to see everyone going around "larping belief". And I don't think you can actually show such a thing occurring. I never claimed I could or that I observed such a thing. The 95/5% didn't come from me, it came from the person I responded to. My point was that - It's impossible to distinguish who's a true bel…

> Which claim exactly did I make which you are refuting? Please actually quote it.

When you said:

> How about the 95% stop having gatherings where they all proclaim that their beliefs are true even if they don't seriously believe them?

you implicitly claimed that "the 95%" indeed have such gatherings.

As far as I can tell, the gatherings that they have are not of that nature.

> My point was that It's impossible to distinguish who's a true believer and who isn't.

And my claim is that it's actually trivial to tell who's a true believer and who isn't, because the concept of "true believer" does not pertain to "rationality" (by construction!) but to fringe beliefs, and people who are doing rationality correctly (which I expect to be the large majority) simply don't profess the fringe beliefs (because doing so is inherently antithetical to rationality). Therefore, the fringe 5% are the 5% of people actually saying the fringe thing, and the 95% don't actually say the fringe thing. Trivial.

For example, you won't catch Scott Alexander demanding "donate to MIRI or the basilisk might reach back in time and torture you".

> That the meetups of the rationalists re-enforce beliefs, particularly in true believers. Also that these beliefs of the rationalists can be pretty whacky and based on a jenga tower of priors.

And you have done nothing to establish this, and I believe you're incorrect.

Re: My falling-out with the rationalist community

#74

Post by joegibbs »

twic wrote: post #12
vovavili wrote: post #2

This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.

It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less. [1] https://lydialaurenson.substack.com/p/the-inside-story-of-le... [2] https://asteriskmag.com/issues/…

The problem is the whole idea of working from first principles to try to discover the actual truth, rather than just going along with other ideas that other people have created through history. The benefit of going along with other peoples' ideas is that they've been tested out for hundreds or thousands of years and been found to not be that destructive.

If you start reasoning from first principles you can very easily make an incorrect assumption somewhere down the line that snowballs until you're talking about shrimp welfare, or making decisions based on the potential sextillion humans that exist in 1,000,000 AD, or planning to convert the entire universe into unthinking organic matter that can only experience bliss.

Re: My falling-out with the rationalist community

#75

Post by hollerith »

twic wrote: post #12
vovavili wrote: post #2

This seems like an awfully uncharitable and cherry-picked treatment of a community of well-intentioned and occasionally eccentric disagreeable online nerds. You're effectively just noticing that a community that actively aims to overcome bias is not free from bias itself, but no one within that community would ever deny that.

It's not just that they aren't free of bias. They appear to incubate truly wacky stuff - Leverage Research [1], the Zizians, all the AI/x-risk stuff. I am not the only person to have noticed this [2]. There's something about their approach that appears to actually make people more susceptible to nonsense, not less. [1] https://lydialaurenson.substack.com/p/the-inside-story-of-le... [2] https://asteriskmag.com/issues/…

>appears to actually make people more susceptible to nonsense

I think it is more that teenagers and very young adults and people high in the trait of "openness to new experiences" are more susceptible to nonsense and more attracted to new intellectual movements, of which the rationalist community is one.

Also, I expect a rationalist to be lower in conscientiousness than average (because the conscientious people are too busy to devote a lot of time to reading long intellectual texts) and low conscientiousness makes violence and other transgressive behavior more likely.

Re: My falling-out with the rationalist community

#76

Post by perching_aix »

zahlman wrote: post #29
nradov wrote: post #8

Rationalism is just a secular religion, complete with all the usual arguments and schisms over minor points of dogma. These are not people worth taking seriously.

There isn't actually such a thing as "rationalism". This is a label applied from outside to those who are trying to apply the concepts of rationality, or those who try to get into rationality but don't actually understand it. Yes, there are schisms. No, there is not dogma. There are arguments people don't want to rehash, and there are fundamental axioms of reasoning . The entire point of the exercise is to establish…

We probably understand this word very differently, because as far as I'm concerned, rationality (rational behavior) is essentially an exercise in behaving optimally, one that is thus strictly relative to some particular context. It is not just "facts and logic", and it is most certainly not universal, since universal optimality doesn't exist: https://en.wikipedia.org/wiki/No_free_lunch_theorem

> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith

Taken at face value, that'd mean bridging the analog hole (https://en.wikipedia.org/wiki/Analog_hole), which is fundamentally impossible. Logic proves consistency, not truth, despite the many related terminologies suggesting otherwise. This is further skipping over how people segment the world different, even relative to themselves over time, which thus forms a coordination problem.

It reminds me to how HTTP is stateless. Yeah, it's stateless - all the state keeping has simply been laundered a layer above. It may sound tantalizing to position certain arguments as simply rational (i.e. correct/optimal), but all that ultimately does is sidestep the actual coordination part, which never really went away. There is still an underlying segmentation model being asserted, it is just now coated in self-righteousness, an illusion of rigor, drawing up a false dichotomy of "either you segment the world like we do, and then you're correct, or you're incorrect, and your segmentation is too".

See this rhetorical landmine in action in a sibling thread: https://news.ycombinator.com/item?id=48986530

Even if I go out of my way and interpret the word such that it merely means "acting on reason" (as opposed to some sort of nonreason), that doesn't mean much for the same reasons. You can have any arbitrary reason to act or think any particular way; possessing an explainable rationale on its own, even if internally coherent, is not necessarily one way or another.

Re: My falling-out with the rationalist community

#77

Post by zahlman »

bigbadfeline wrote: post #67
zahlman wrote: post #58

Earlier quoted context omitted.

> You do realize you sound exactly like a cult member right? If you say so. That doesn't change the fact that P(A|B) = P(B|A)P(A)/P(B).

> That doesn't change the fact that P(A|B) = P(B|A)P(A)/P(B). It didn't take long before a strawman got pulled out of nowhere. That's so typical, if one disagrees with the wild conjectures of the rationalists, which they call "axioms", they blame you for rejecting the Pythagorean Theorem or some such. Dishonesty is the defining feature of that "movement"

> It didn't take long before a strawman got pulled out of nowhere.

It's not a strawman. It's a central example of what I had in mind when I used the word "axiom".

> if one disagrees with the wild conjectures of the rationalists, which they call "axioms"

They don't refer to wild conjectures as axioms.

> they blame you for rejecting the Pythagorean Theorem or some such.

There is no "blame" going on here. Someone said I sounded like a cult member (which incidentally is insulting and inappropriate on HN; but I thought I would try to steer the discussion back on course) and I correctly explained why that perception is irrelevant. When I said

> The entire point of the exercise is to establish that certain things are true, and that this is not because of faith, and that disbelieving them cannot make them false.

That is, in fact, inarguably true. Bayes' Law is an example of that exact sort of thing. If you don't believe that conditional probabilities can be worked out that way, that won't change the fact that conditional probabilities can in fact be worked out that way; and it won't change the fact that in the long run I will be able to exploit your fallacy in games of chance.

The other poster gave no example of supposedly false or unsound axioms, so I cannot be strawmanning by failing to respond to them. My assertion is that such false or unsound axioms are not, in fact, in play. The burden of proof lies with those claiming their existence.

Re: My falling-out with the rationalist community

#78

Post by joegibbs »

FeepingCreature wrote: post #17
cogman10 wrote: post #11

Earlier quoted context omitted.

I think the issue with the community is they take their eccentric beliefs as being gospel truths which can lead to truly unhinged and irrational beliefs and actions (see: the Zizians). That's what can make the rationalist movement feel more like a religious cult and less like a bunch of eccentrics. As an outsider looking in, it seems like the entire movement is filled with just accepting priors with very little evide…

Consider a community filled with 100 people; 95 of them who don't take their eccentric beliefs as gospel truths and 5 who do. Those 5 will inevitably become the public face of the community when they make the news. However there is very little that the 95 could have done to stop them.

Doesn't it say something if 5% of a group are wacko lunatics if, for instance, a random sample of the rest of the population has only 0.5% wackos?

Re: My falling-out with the rationalist community

#80

Post by justonepost2 »

zahlman wrote: post #36
g-b-r wrote: post #28

Earlier quoted context omitted.

It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us. That, along with constantly calling yourself rational and being well-intentioned, easily leads to cultish beliefs to be pursued despite how wacky they seem at their face, because it was your superior brains that led you to them. If we add the often good (or enormous) financial resources, it makes for a seriously d…

> It's not the bias the problem, it's the conviction of being better and "less wrong" than the rest of us. There is not any such conviction. This is just your interpretation of a domain name, which is not intended that way. > That, along with constantly calling yourself rational This is explicitly not what you are supposed to do, and the point is explicitly not to create an identity around it.

https://xcancel.com/tenobrus/status/2023473440364392850#m
Post reply on HN