Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why are there so many rationalist cults?

asteriskmag.com

Post reply on HN
Print view View original

881–890 of 947 posts

Re: Why are there so many rationalist cults?

#881

Post by low_tech_love »

Viliam1234 wrote: post #852
low_tech_love wrote: post #705

Earlier quoted context omitted.

The view from the inside, written by a person who is waist deep into the movement, is the only fair look into the phenomenon?

In theory, there should be a middle way between "waist deep into the movement" and "my research consists of collecting rumors on the internet, and then calling one or two people to give me a quote". In practice, I don't remember reading an article on the rationality community written from such position. Most articles are based on other articles, which are based on yet other articles... ultimately based on someone's o…

Every reporting is always based on the reporter, so you’re never going to escape the need to just make your own conclusions and decide for yourself whether you think that the piece makes sense or not. There is no conspiracy against rationalists, they’re being reported on with the same methods as everything else; whether you trust journalism in general or not is up to you.

Re: Why are there so many rationalist cults?

#882

Post by eru »

jl6 wrote: post #187
JohnMakin wrote: post #85

One of a few issues I have with groups like these, is that they often confidently and aggressively spew a set of beliefs that on their face logically follow from one another, until you realize they are built on a set of axioms that are either entirely untested or outright nonsense. This is common everywhere, but I feel especially pronounced in communities like this. It also involves quite a bit of navel gazing that m…

I don’t think it’s just (or even particularly) bad axioms, I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence. Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some…

> Not that non-rationalists are any better at reasoning, but non-rationalists do at least benefit from some intellectual humility.

I have observed no such correlation of intellectual humility.

Re: Why are there so many rationalist cults?

#883

Post by Dylan16807 »

godelski wrote: post #875
Dylan16807 wrote: post #872

Earlier quoted context omitted.

I'll try to keep this simple. > I'm not disagreeing with you. You understand that, right? We disagree about whether context can make a difference, right? > The parent was talking about stringing together inferences. My argument was how you string them together matters . That's all. I said "context matters." > TLDR: We can't determine if likelihood increases or decreases without additional context The situations you d…

> I'm mostly sure we're only disagreeing about the definition of stringing/chaining? Correct. > No, you're being way too loose. Okay, instead of just making claims and for me to trust you, go point to something concrete. I've even tried to google, but despite my years of study in statistics, metric theory, and even mathematical logic I'm at a loss in finding your definition. I'm aware of the Chain Rule of Probability…

> Okay, instead of just making claims and for me to trust you, go point to something concrete. I've even tried to google, but despite my years of study in statistics, metric theory, and even mathematical logic I'm at a loss in finding your definition.

Let's look again at what we're talking about:

>>> I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity that follows inevitably from its antecedents, but actually each step is a little bit leaky, leading to runaway growth in false confidence.

>> As a former mechanical engineer, I visualize this phenomenon like a "tolerance stackup". Effectively meaning that for each part you add to the chain, you accumulate error.

> I saw an article recently that talked about stringing likely inferences together but ending up with an unreliable outcome because enough 0.9 probabilities one after the other lead to an unlikely conclusion.

> Edit: Couldn't find the article, but AI referenced Baysian "Chain of reasoning fallacy".

The only term in there you could google is "tolerance stackup". The rest is people making ad-hoc descriptions of things. Except for "Chain of reasoning fallacy" which is a fake term. So I'm not surprised you didn't find anything in google, and I can't provide you anything from google. There is nothing "concrete" to ask for when it comes to some guy's ad-hoc description, you just have to read it and do your best.

And everything I said was referring back to those posts, primarily the last one by robocat. I was not introducing anything new when I used the terms "string" and "chain". I was not referring to any scientific definitions. I was only talking about the concept described by those three posts.

Looking back at those posts, I will confidently state that the concept they were talking about does not include markov chains. You're not having a stroke, it's just a coincidence that the word "chain" can be used to mean multiple things.

Re: Why are there so many rationalist cults?

#885

Post by wolvesechoes »

dfabulich wrote: post #82

The whole game of Rationalism is that you should ignore gut intuitions and cultural norms that you can't justify with rational arguments. Well, it turns out that intuition and long-lived cultural norms often have rational justifications, but individuals may not know what they are, and norms/intuitions provide useful antibodies against narcissist would-be cult leaders. Can you find the "rational" justification not to…

> The whole game of Rationalism is that you should ignore gut intuitions and cultural norms that you can't justify with rational arguments.

This is why it is so naive - gut intuitions and cultural norms pretty much dictate what does it mean for the argument to be rational.

Re: Why are there so many rationalist cults?

#886

Post by godelski »

Dylan16807 wrote: post #883
godelski wrote: post #875

Earlier quoted context omitted.

> I'm mostly sure we're only disagreeing about the definition of stringing/chaining? Correct. > No, you're being way too loose. Okay, instead of just making claims and for me to trust you, go point to something concrete. I've even tried to google, but despite my years of study in statistics, metric theory, and even mathematical logic I'm at a loss in finding your definition. I'm aware of the Chain Rule of Probability…

> Okay, instead of just making claims and for me to trust you, go point to something concrete. I've even tried to google, but despite my years of study in statistics, metric theory, and even mathematical logic I'm at a loss in finding your definition. Let's look again at what we're talking about: >>> I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity t…

I googled YOUR terms. And if you read my messages you'd notice that I'm not a novice when it comes to math. Hell, you should have gotten that from my very first comment. I was never questioning if I had a stroke, I was questioning your literacy.

  > I was not referring to any scientific definitions.
Yet, you confidently argued against ones that were stated.

If you're going to speak out your ass, at least have the decency to let everyone know first.

Re: Why are there so many rationalist cults?

#887

Post by bondarchuk »

godelski wrote: post #866
bondarchuk wrote: post #849

Earlier quoted context omitted.

If that was what you meant to say though, you've gotta admit that opening a paragraph with "The other weird assumption I hear is about how it'll just kill us all" , and then spending the rest of the paragraph giving examples of the peacefulness of smart people, is not the most effective strategy of communicating that.

You were the one who interpreted "Here's examples of smart peaceful people" as "smart == peaceful". I was never attempting to make such a claim and did say that. The whole thread is about bad assumptions and bad logic. > is not the most effective strategy of communicating that. The difficulty of talking on the internet is you can't know your audience and your audience is everybody. Yes, this should make us more aware…

I think the big difference between our views is that you are taking the rationalist argument to be "from intelligence follows malice, therefore it will want to kill us all" whereas I take it to be "from intelligence follows great capability and no morality, therefore it may or may not kill us uncaringly in pursuit of other goals".

Re: Why are there so many rationalist cults?

#888

Post by Dylan16807 »

godelski wrote: post #886
Dylan16807 wrote: post #883

Earlier quoted context omitted.

> Okay, instead of just making claims and for me to trust you, go point to something concrete. I've even tried to google, but despite my years of study in statistics, metric theory, and even mathematical logic I'm at a loss in finding your definition. Let's look again at what we're talking about: >>> I think it’s that people tend to build up “logical” conclusions where they think each step is a watertight necessity t…

I googled YOUR terms. And if you read my messages you'd notice that I'm not a novice when it comes to math. Hell, you should have gotten that from my very first comment. I was never questioning if I had a stroke, I was questioning your literacy. > I was not referring to any scientific definitions. Yet, you confidently argued against ones that were stated. If you're going to speak out your ass, at least have the decen…

They were never my terms. They were the terms from the people that were having a nice conversation before you interrupted.

You told them they were wrong, that it could go either way.

That's not true.

What they were talking about cannot go either way.

You were never talking about the same thing as them. I gave you the benefit of the doubt by thinking you were trying to talk about the same thing as them. Apparently I shouldn't have.

You can't win this on definitions. They were talking about a thing without using formal definitions, and you replied to them with your own unrelated talk, as if it was what they meant. No. You don't get to change what they meant.

That's why I argued against your definition. Your definition is lovely in some other conversation. Your definition is not what they meant, and cannot override what they meant.

Re: Why are there so many rationalist cults?

#889

Post by godelski »

joe_the_user wrote: post #869
godelski wrote: post #702

Earlier quoted context omitted.

> you talking of axiomatization of mathematical or related systems. Why do you think these are so different? Math is just a language in which we are able to formalize abstraction. Sure, it is pedantic as fuck, but that doesn't make it "not real world". If you want to talk about the bus always being late you just do this distributionally. Probabilities are our formalization around uncertainty. We're talking about "rat…

Why do you think these are so different? Because "reality" doesn't have "atomic", certain, etc operations? Also, it's notable that since most reasonings about the real world are approximate, the law of excluded middle is much less likely to apply. If you want to talk about the bus always being late you just do this distributionally. Probabilities are our formalization around uncertainty. Ah, but you can't be certain…

  > Because "reality" doesn't have "atomic", certain, etc operations?
That's not a requirement. The axioms are for our modeling, not reality.

  > but you can't be certain that you're dealing with a given distribution, not outside the quantum realm.
I guess I'll never understand why non-physicists want to talk so confidently about physics. Especially quantum mechanics[0]. You can get through Griffiths with mostly algebra and some calculus. Group theory is a big plus, but not necessary. I also suggest having a stiff drink on hand. Sometimes you'll need to just shut up and do the math. Don't worry, it'll only be more confusing years later if you get to Messiah.

[0] https://xkcd.com/451/

Re: Why are there so many rationalist cults?

#890

Post by godelski »

bondarchuk wrote: post #887
godelski wrote: post #866

Earlier quoted context omitted.

You were the one who interpreted "Here's examples of smart peaceful people" as "smart == peaceful". I was never attempting to make such a claim and did say that. The whole thread is about bad assumptions and bad logic. > is not the most effective strategy of communicating that. The difficulty of talking on the internet is you can't know your audience and your audience is everybody. Yes, this should make us more aware…

I think the big difference between our views is that you are taking the rationalist argument to be "from intelligence follows malice, therefore it will want to kill us all" whereas I take it to be "from intelligence follows great capability and no morality, therefore it may or may not kill us uncaringly in pursuit of other goals".

  > you are taking the rationalist argument to be
I think they say P(doom) is high number[0]. Or in other words, AGI is likely to kill us. I interpret this as "if we make a really intelligent machine it is very likely to kill us all." My interpretation is mainly biased on them saying "if we build a really intelligent machine, it is very likely to kill us all."

Yud literally wrote a book titled "If Anyone Builds It, Everyone Dies."[1] There's not much room for ambiguity here...

[0] Yud is on the record saying at least 95% https://pauseai.info/pdoom He also said anyone with a higher P(doom) than him is crazy so I think that says a lot...

[1] https://ifanyonebuildsit.com/

Post reply on HN