Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Guess I'm a rationalist now

scottaaronson.blog

Post reply on HN
Print view View original

751–760 of 1001 posts

Re: Guess I'm a rationalist now

#751

Post by imtringued »

astrange wrote: post #453
nradov wrote: post #437

Earlier quoted context omitted.

In what sense are people in those communities "quite smart"? Stupid is as stupid does. There are plenty of people who get good grades and score highly on standardized tests, but are in fact nothing but pontificating blowhards and useless wankers.

They're members of a religion which says that if you do math in your head the right way you'll be correct about everything, and so they think they're correct about everything. They also secondarily believe everyone has an IQ which is their DBZ power level; they believe anything they see that has math in it, and IQ is math, so they believe anything they see about IQ. So if you avoid trying to find out your own IQ you…

According to rationalists, humans don't work together, so you can't add up their individual intelligence to get more intelligence. Meanwhile building a single giant super AI is technologically feasible, so they weigh the intelligence of a single person vs all AIs operating as a collective hivemind.

Re: Guess I'm a rationalist now

#752

Post by jstanley »

biofox wrote: post #335
sixo wrote: post #139

Earlier quoted context omitted.

To the Stem-enlightened mind, the classical understanding and pedagogy of such ideas is underwhelming, vague, and riddled with language-game problems, compared to the precision a mathematically-rooted idea has. They're rederiving all this stuff not out of obstinacy, but because they prefer it. I don't really identify with rationalism per se, but I'm with them on this--the humanities are over-cooked and a humanity edu…

If you contextualise the outmoded ideas as part of the Great Conversation [1], and the story of how we reached our current understanding, rather than objective statements of fact, then they becomes a lot more valuable and worthy of study. [1] https://en.wikipedia.org/wiki/Great_Conversation

But isn't the content of LessWrong part of the Great Conversation too?

Re: Guess I'm a rationalist now

#753

Post by imtringued »

DowsingSpoon wrote: post #660
ineptech wrote: post #610

Earlier quoted context omitted.

What's the seductive-but-wrong part of EA? As far as I can tell, the vast majority of the opposition to it boils down to "Maybe you shouldn't donate money to pet shelters when people are dying of preventable diseases" vs "But donating money to pet shelters feels better!"

It might be the parts that lead a person to commit large scale fraud with the idea that the good they can do with the stolen money outweighs all the negatives. Or, at least, that’s the popular idea of what happened to Sam Bankman-Fried. I have no idea what was actually going through that man’s mind. In any case, EA smells strongly of “the ends justify the means” which most popular moral philosophies reject with stron…

I've noticed this with a lot of radical streamers on both sides. They don't care about principles, they care about "winning" by any means necessary.

Winning at things that align with your principle is a principle. If you don't care about principles, you don't care about what you're winning at, thereby making every victory hollow and meaningless. That is how you turn into a loser at everything you do.

Re: Guess I'm a rationalist now

#754

Post by yawnxyz »

comp_throw7 wrote: post #737
yawnxyz wrote: post #455

Earlier quoted context omitted.

Having just moved to the Bay Area, I've met a few AI "safety researchers" who seems to come from this EA / Rationalist camp, and they all behave more like preachers than thinkers / academics / researchers. I don't think any "Rationalists" I ever met would actually consider concepts like scientific method...

> I don't think any "Rationalists" I ever met would actually consider concepts like scientific method... In that case I don't think you've met any of the people under discussion.

maybe, but I Stanford scientists are a different breed of rationalists, and not Rationalists

Re: Guess I'm a rationalist now

#755

Post by imtringued »

bena wrote: post #575
aspenmayer wrote: post #543

Earlier quoted context omitted.

> Technically "long-termism" should lead them straight to nihilism. Because, eventually, everything will end. One way or another. The odds are just 1. At some point, there are no more future humans. The number of humans are zero. Also, due to the nature of the infinite, any finite thing is essentially a rounding error and not worth concerning oneself with. The current sums invested and donated in altruist causes are…

I think it's essentially a grift. An excuse to do nothing while looking like you care and reaping rewards. If they wanted to help, they should be focused on the now. Global poverty, climate change, despotic world leaders. They should be aligning themselves against such things. But instead what we see is essentially not that. Effective altruism is a lot like the Democratic People's Republic of Korea, a bit of a misnom…

They have aligned themselves in favor of global poverty, climate change and despotic world leaders.

A lot of them argue that poor countries essentially don't matter. Climate change is not an extinction event and there should an authoritarian world government to prevent nuclear conflict to minimize the risk of nuclear extinction.

>In his dissertation On the Overwhelming Importance of Shaping the Far Future (2013), supposedly “one of the best texts on existential risks,”[9] Nicholas Beckstead meditates on the “ripple effects” a human life might have for future generations and concludes “that saving a life in a rich country is substantially more important than saving a life in a poor country” due to the higher level of innovation and economic productivity attained in these countries.[10]

https://umbau.hfg-karlsruhe.de/posts/philosophy-against-the-...

Re: Guess I'm a rationalist now

#756

Post by gjm11 »

FeepingCreature wrote: post #622
gjm11 wrote: post #518

Earlier quoted context omitted.

I don't think "rationality" is a good name for the movement, for the same reason as I wish "effective altruism" had picked a different name: it conflates the goal with the achievement of the goal . A rationalist (in the Yudkowsky sense) is someone who is trying to be rational , in a particular way. But "rationality" means actually being rational. I don't think it's actually true that rationalists-in-this-sense common…

Yeah, that's why they famously say they're an "aspiring rationalist." But I don't think there's anything wrong with setting a target, even if it's unreachable.

No, absolutely nothing wrong with setting ambitious and unreachable targets. Reach for the stars and you might at least get to the moon, etc. But if you say "I'm an effective altruist" then you are claiming to actually be effective and if you say "our movement is Rationality" then you are claiming to be paradigmatic examples of actually being rational, and I think that's a bad idea because it should be possible to say "I'm part of such-and-such a movement" without making that kind of claim.

Re: Guess I'm a rationalist now

#757

Post by tsimionescu »

kragen wrote: post #187
mathattack wrote: post #182

Logic is an awesome tool that took us from Greek philosophers to the gates on our computers. The challenge with pure rationalism is checking the first principles that the thinking comes from. Logic can lead you astray if the principles are wrong, or you miss the complexity along the way. On the missing first principles, look at Aristotle. One of the history's greatest logicians came to many false conclusions. On miss…

The ‘rationalist’ group being discussed here aren't Cartesian rationalists, who dismissed empiricism; rather, they're Bayesian empiricists. Bayesian probability turns out to be precisely the unique extension of Boolean logic to continuous real probability that Aristotle (nominally an empiricist!) was lacking. (I think they call themselves “rationalists” because of the ideal of a “rational Bayesian agent” in economics…

Bayesian inference is very, very often used in the types of philosophical/speculative discussions that Rationalists like instead of actual empirical study. It's a very convinient framework for speculating wildly while still maintaining a level of in-principle rationality, since, of course, you [claim that] you will update your priors if someone happens to actually study the phenomenon in question.

The reality is that reasoning breaks down almost immediately if probabilities are not almost perfectly known (to the level that we know them in, say, quantum mechanics, or poker). So applying Bayesian reasoning to something like the number of intelligent species in the galaxy ("Drake's equation"), or the relative intelligence of AI ("the Singularity") or any such subject allows you to draw any conclusion you actually wanted to draw all along, and then find premises you like to reach there.

Re: Guess I'm a rationalist now

#758

Post by thrance »

int_19h wrote: post #699
bilbo0s wrote: post #197

Earlier quoted context omitted.

They won't be allowed to use it unless they serve the capitalists who own it. It's not social media. It's a model the capitalists train and own. Best the rest of us will have access to are open source ones. It's like the difference between trying to go into court backed by google searches as opposed to Lexis/Nexis. You're gonna have a bad day with the judge. Here's hoping the open source stuff gets trained on quality…

Looking at Grok especially, it doesn't feel like a given that you can have a true SOTA model that is properly brainwashed.

To be fair, Musk is probably the least subtle megabillionaire of them all, and that reflects in the odd behavior of his silicon child. I don't doubt in the competency of the likes of Thiel to build their techo-monarchy.

Re: Guess I'm a rationalist now

#759

Post by gjm11 »

alephnerd wrote: post #635
gjm11 wrote: post #540

Earlier quoted context omitted.

My understanding of the Yudkowskian argument for AI x-risk is that a key step is along the lines of "an AI much smarter than us will find ways to get what it wants even if we want something else -- even though we can't predict now what those ways will be, just as chimpanzees could not have predicted how humans would outcompete them and just as you could not predict exactly how Magnus Carlsen will crush you if you pla…

> I take it this is what you have in mind when you say that whenever you ask for an "attack path" you keep getting a deus ex machina. But it seems to me like a pretty weak basis for calling Yudkowsky's position on this a religion. Agree to disagree. > So far as I can tell this is completely untrue I was under the impression EY attended Yeshivat Sha'alvim (the USC of Yeshivas - rigorous and well regarded, but a "warme…

Happy to agree-to-disagree but I will push on it just one bit more.

Let's take the chess analogy. I take it you agree that I would very reliably lose if I played Magnus Carlsen at chess; he's got more than 1000 Elo points on me. But I couldn't tell you the "attack path" he would use. I mean, I could say vague things like "probably he will spot tactical errors I make and win material, and in the unlikely event that I don't make any he will just make better moves than me and gradually improve his position until mine collapses", but that's the equivalent of things like "the AI will get some of the things it wants by being superhumanly persuasive" or "the AI will be able to figure out scientific/engineering things much better than us and that will give it an advantage" which Yudkowsky can also say. I won't be able to tell you in advance what mistakes I will make or where my pawn structure will be weak or whatever.

Does this mean that, for you, if I cared enough about my inevitable defeat at Carlsen's hands that expectation would be religious?

To me it seems obvious that it wouldn't, and that if Yudkowsky's (or other rationalists') position on AI is religious then it can't be just because one important argument they make has a step in it where they can't fill out all the details. I am pretty sure you have other things in mind too that you haven't made so explicit.

(The specific other things I've heard people cite as reasons why rationalism is really a religion also, individually and collectively, seem very unconvincing to me. But if you throw 'em all in then we are in what seems to me like more reasonable agree-to-disagree territory.)

> That said, EY is absolutely from a Daatim or Chabad household

I think holding that against him, as you seem to be doing, is contemptible. If his ideas are wrong, they're fair game, but insinuating that we should be suspicious of his ideas because of the religion of his family, which he has rejected? Please, no. That goes nowhere good.

Re: Guess I'm a rationalist now

#760

Post by GoblinSlayer »

lisper wrote: post #499
dasil003 wrote: post #338

Earlier quoted context omitted.

Intelligence and rational thought is useful, but like any strategy it has its tradeoffs and limitations. No amount of intelligence can overcome the chaos of long time horizons, especially when we're talking about human civilization. IMHO it's reasonable to pick a long-term problem/risk and focus on solving it. But it's pure hubris to think rationality will give you anything approaching high confidence of what the big…

I think one can levy a much more specific critique of rationalism: rationalism is in some sense self-defeating. If you are rational you will necessarily conclude that the fundamental dynamic that drives the (interesting parts of) the universe is Darwinian evolution, which is not rational. It blindly selects for reproductive fitness at the expense of all else. If you are a gene, you can probably produce more offspring…

Darwinism isn't a weakness of rationality, teleology has fine tuning problem, while darwinism is minimally fine tuned to work from scratch, which can be said to be optimal. Also darwinism doesn't select for reproductive fitness, it's only a proxy goal; true goal is survival, so you can produce more offspring only in a way compatible with true goal.
Post reply on HN