Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Guess I'm a rationalist now

scottaaronson.blog

Post reply on HN
Print view View original

521–530 of 1001 posts

Re: Guess I'm a rationalist now

#521

Post by oh_my_goodness »

LargeWu wrote: post #438
jhbadger wrote: post #414

Earlier quoted context omitted.

As Adam Becker shows in his book, EAs started out being reasonable "give to charity as much as you can, and research which charities do the most good" but have gotten into absurdities like "it is more important to fund rockets than help starving people or prevent malaria because maybe an asteroid will hit the Earth, killing everyone, starving or not".

It's also not a very big leap from "My purpose is to do whatever is the greatest good" to "It doesn't matter if I hurt people as long as the overall net result is good (by some arbitrary standard)"

I think this is the key comment so far.

Re: Guess I'm a rationalist now

#522

Post by TimTheTinker »

nyeah wrote: post #490
TeMPOraL wrote: post #450

Earlier quoted context omitted.

You mean people projecting this problem onto them with zero evidence beyond their own preconceptions and an occasional stereotype they read online? HN should be better than this.

Let's read the thread: "There has to be (or ought to be) a name for this kind of epistemological fallacy, where in pursuit of truth, the pursuit of logical sophistication and soundness between starting assumptions (or first principles) and conclusions becomes functionally way more important than carefully evaluating and thoughtfully choosing the right starting assumptions (and being willing to change them when they a…

Name calling may be helpful in some contexts, but I doubt it's persuasive to the people in question.

Re: Guess I'm a rationalist now

#523

Post by Aurornis »

Certhas wrote: post #275
mitthrowaway2 wrote: post #189

Earlier quoted context omitted.

> They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considered all angles to it, maybe I'm wrong - but here it is". Aren't these the people who started the trend of writing things like " epistemic status: mostly speculation " on their blog posts? And writing essays about the dangers of overconfidence? And measuring how often their predictions turn out wrong? And maintaining webpages tit…

I think this is a valid point. But to some degree both can be true. I often felt when reading some of these type of texts: Wait a second, there is a wealth of thinking on these topics out there; You are not at all situating all your elaborate thinking in a broader context. And there absolutely is willingness to be challenged, and (maybe less so) a willingness to be wrong. But there also is an arrogance that "we are t…

> But there also is an arrogance that "we are the ones thinking about this rationally, and we will figure this out". As if people hadn't been thinking and discussing and (verbally and literally) fighting over all sorts of adjacent and similar topics in philosophy and sociology and anthropology and ... clubs and seminars forever

This is a feature, not a bug, for writers who hold an opinion on something and want to rationalize it.

So many of the rationalist posts I've read through the years come from someone who has an opinion or gut feeling about something, but they want it to be seen as something more rigorous. The "first principles" writing style is a license to throw out the existing research on the topic, including contradictory evidence, and construct an all new scaffold around their opinion that makes it look more valid.

I use the "SlimeTimeMoldTime - A Chemical Hunger" blog series as an example because it was so widely shared and endorsed in the rationalist community: https://slimemoldtimemold.com/2021/07/07/a-chemical-hunger-p... It even received a financial grant from Scott Alexander of Astral Codex Ten

Actual experts were discrediting the series from the first blog post and explaining all of the author's errors, but the community soldiered on with it anyway, eventually making the belief that lithium in the water supply was causing the obesity epidemic into a meme within the rationalist community. There's no evidence supporting this and countless take-downs of how the author misinterpreted or cherry-picked data, but because it was written with the rationalist style and given the implicit blessing of a rationalist figurehead it was adopted as ground truth by many for years. People have been waking up to issues with the series for a while now, but at the time it was remarkable how quickly the idea spread as if it was a true, novel discovery.

Re: Guess I'm a rationalist now

#524

Post by const_cast »

lukas099 wrote: post #504
Aurornis wrote: post #474

Earlier quoted context omitted.

> They are a community—they have an in-group, and if you are not one of them you are by-definition in the out-group. The rationalist community is most definitely not exclusive. You can join it by declaring yourself to be a rationalist, posting blogs with "epistemic status" taglines, and calling yourself a rationalist. The criticisms are not because it's a cool club that won't let people in. > They have unusual opinio…

> The rationalist community is most definitely not exclusive. I agree, and didn't intend to express otherwise. It's not an exclusive community, but it is a community, and if you aren't in it you are in the out-group. > The problem isn't differing opinions, the problem is that they disregard actual expertise and try ham-fisted attempts at "first principals" evaluations of a subject while ignoring contradictory evidenc…

> Maybe I don't give people enough credit and you do, but I don't think most people care how you arrived at an opinion; they merely care about whether you're in their opinion-tribe or not.

Yes, most people don't care how you arrived at an opinion, they rather care about the practical impact of said opinion. IMO this is largely a good thing.

You can logically push yourself to just about any opinion, even absolutely horrific ones. Everyone has implicit biases and everyone is going to start at a different starting point. The problem with string of logic for real-world phenomena is that you HAVE to make assumptions. Like, thousands of them. Because real-world phenomena are complex and your model is simple. Which assumptions you choose to make and in which directions are completely unknown, even to you, the one making said assumptions.

Ultimately most people aren't going to sit here and try to psychoanalyze why you made the assumptions you made and if you were abused in childhood or deduce which country you grew up in or whatever. It's too much work and it's pointless - you yourself don't know, so how would we know?

So, instead, we just look at the end opinion. If it's crazy, people are just going to call you crazy. Which I think is fair.

Re: Guess I'm a rationalist now

#525

Post by Johanx64 »

troyastorino wrote: post #276
jandrese wrote: post #237

Earlier quoted context omitted.

They remind me of the "Effective Altruism" crowd who get completely wound up in these hypothetical logical thought exercises and end up coming to insane conclusions that they feel trapped in because they got there using pure logic. Not realizing that their initial conditions were highly artificial so any conclusion they reach is only of academic value. There is a term for this. "Getting stuck up your own butt." It wo…

The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current peopl…

>People in these communities are generally quite smart, and it’s seductive to reason in a purely logical, deductive way. There is real value in thinking rigorously and in making sure you’re not beholden to commonly held beliefs. But, like you said, reality is complex, and it’s really hard to pick initial premises that capture everything relevant. The insane conclusions they get to could be avoided by re-checking & revising premises, especially when the argument is going in a direction that clashes with history, real-world experience, or basic common sense.

They don't even do this.

If you're reasoning in purely logical and deductive way - it's blatantly obvious that living beings experience way more pain and suffering, than pleasure and joy. If you do the math, humanity getting wiped out in effect is the best thing that could happen.

Which is why accelerationism ignoring all the AGI risks is correct strategy presuming the AGI will either wipe us out (good outcome) or provide technologies that improve the human condition and reduce suffering (good outcome).

Logical and deductive reasoning based on completely baseless and obviously incorrect premises is flat out idiotic.

You can't deprive non-existent people out of anything.

And if you do, I hope you're ready for purely logical, deductive follow up - every droplet of sperm is sacred and should be used to impregnate.

Re: Guess I'm a rationalist now

#526

Post by baxtr »

kragen wrote: post #145
baxtr wrote: post #99

Earlier quoted context omitted.

My main problem with the movement is their emphasis on Bayesianism in conjunction with an almost total neglect of Popperian epistemology. In my opinion, there can’t be a meaningful distinction made between rational and irrational without Popper. Popper injects an epistemic humility that Bayesianism, taken alone, can miss. I think that aligns well with your observation.

Hmm, what epistemological propositions of Popper's do you think they're missing? To the extent that I understand the issues, they're building on Popper's epistemology, but by virtue of having a more rigorous formulation of the issues, they resolve some of the apparent contradictions in his views. Most of Popper's key points are elaborated on at length in blog posts on LessWrong. Perhaps they got something wrong? Or o…

> Popper’s falsificationism – this is the old philosophy that the Bayesian revolution is currently dethroning.

https://www.yudkowsky.net/rational/bayes

These are the kind of statements I’m referring to. Happy to be falsified btw :) that’s how we learn.

Also note that Popper never called his theory falsificationism.

Re: Guess I'm a rationalist now

#527

Post by jay_kyburz »

NoMoreNicksLeft wrote: post #318
danans wrote: post #301

Earlier quoted context omitted.

> We don't even have anything resembling real AI at the moment. Generative models are probably some blind alley. I think that the OP's point was that it doesn't matter whether it's "real AI" or not. Even if it's just a glorified auto-correct system, it's one that has the clear potential to overturn our information/communication systems and our assumptions about individuals' economic value.

If that has the potential to ruin economies, then the economic rot is so much more profound than anyone (me included) ever realized.

I think when the GP says "our assumptions about individuals' economic value." they mean half the workforce becoming unemployed because the auto corrector can do it cheaper.

That's going to be a swift kick to your economy, no matter how strong.

Re: Guess I'm a rationalist now

#528

Post by Johanx64 »

im3w1l wrote: post #390
troyastorino wrote: post #276

Earlier quoted context omitted.

The overlap between the Effective Altruism community and the Rationalist community is extremely high. They’re largely the same people. Effective Altruism gained a lot of early attention on LessWrong, and the pessimistic focus on AI existential risk largely stems from an EA desire to avoid “temporal-discounting” bias. The reasoning is something like: if you accept that future people count just as much as current peopl…

I think most everyone can agree with this: Being 100% rigorous and rational, reasoning from first principles and completely discarding received wisdom is a great trait in a philosopher but a terrible trait in a policymaker. Because for the former, exploring ideas for the benefit of future generations is more important than whether they ultimately reach the right conclusion or not.

> Being 100% rigorous and rational, reasoning from first principles

It really annoys me when people say that those religious cultists do that.

They derive their bullshit from faulty, poorly thought out premises.

If you fuck up the very firsts calculations of the algorithm, it doesn't matter how rigorous all the subsequent steps are. There results are going to be all wrong.

Re: Guess I'm a rationalist now

#529

Post by const_cast »

FeepingCreature wrote: post #336
mock-possum wrote: post #262

Earlier quoted context omitted.

…but that just sounds like scientific racism. Rather than judging an individual on their actual intelligence, these kinds of statistical trends allow you to justify judging an individual based on their race, because you feel you can credibly claim that race is an acceptable proxy for their genome, is an acceptable proxy for their intelligence. Or for their trustworthiness, or creativity, or sexuality, or dutifulness,…

Well, the rational thing is obviously to be scared of what ideas sound like. > Rather than judging an individual on their actual intelligence Actual intelligence is hard to know! However, lots of factors allow you to make a rapid initial estimate of their actual intelligence, which you can then refine as required. (When the factors include apparent genetic heritage, this is called "racism" and society doesn't like it…

> Well, the rational thing is obviously to be scared of what ideas sound like.

Yes, actually. If an idea sounds like it can be used to commit crimes against humanity, you should pause. You should reassess said idea multiple times. You should be skeptical. You shouldn't ignore that feeling.

What a lot of people are missing is intent - the human element. Why were these studies conducted? Who conducted them?

If someone insane conducts a study then yes - that is absolutely grounds to be skeptical of said study. It's perfectly rationale. If extremely racist people produce studies which just so happen to be racist, we should take a step back and go "hmm".

Being right or being correct is one thing, but it's not absolutely valuable. The end-result and how "bad" it is also matters, and often times it matters more. And, elephant in the room, nobody actually knows if they're right. Making logical conclusions isn't so, because you are forced to make thousands of assumptions.

You might be right, you might not be. Let's all have some humility.

Re: Guess I'm a rationalist now

#530

Post by JKCalhoun »

contrarian1234 wrote: post #9

The article made me think deeper about what rubs me the wrong way about the whole movement I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considere…

> lack of acknowledgment that maybe we don't have a full grasp of the implications of AI

And why single out AI anyway? Because it's sexy maybe? Because if I had to place bets on the collapse of humanity it would look more like the British series "The Survivors" (1975–1977) than "Terminator".

Post reply on HN