Live data from Hacker News

Guess I'm a rationalist now

scottaaronson.blog

911–920 of 1001 posts

Re: Guess I'm a rationalist now

#911

Earlier quoted context omitted.

Well, sure, but mathematics is the domain for which this holds maybe the most true out of any. It's less true for fields which are not as old. I'm not sure if this counterpoint generalizes entirely to the original critique, since certainly LessWrongers aren't usually posting about or discussing math as if they've discovered it-- usually substantially more niche topics.

... philosophy is if anything older than what we call mathematics.

I suppose you're right about that, so I can't make the argument go through by saying "mathematics" vs "philosophy". Maybe what I should say instead is that as some dialectics advance/technologies develop, subfields of both such things sprout up and have a lot of low-hanging fruit to pick, and in these cases, the new work will be descended from but not too-essentially informed by the prior work.

Like mathematical logic (in the intersection of math and philosophy) didn't have that many true predecessors and was developed very far by maybe only 5-10 individuals cumulatively, or information theory was basically established by Claude Shannon and maybe two other guys, or various aspects of convex optimization or Fourier analysis were only developed in the 80s or so, it stands to reason that the AI-related applications of various aspects of philosophy are ripe to be developed now. (By contrast, we don't see, as much, people on LW trying to redo linear algebra from the ground up, nor more "mature" aspects of philosophy.)

(If anything, I think it's more feasible than ever before, also, for a bunch of relative amateurs to non-professionally make real intellectual contributions, like noticeably moreso than 100 or even 20 years ago. That's what increasing the baseline levels of education/wealth/exposure to information was intended to achieve, on some level, isn't it?)

Re: Guess I'm a rationalist now

#912
post #227

Earlier quoted context omitted.

My impression is that many rationalists enjoy believing that they update their beliefs, but in practice they're human and just as attached to preconceived notions as anyone else. But if you go around telling everyone that updating is your super-power, you're going to be a lot less humble about your own failures to do so. If you want to see how human and tribal rationalists are, go criticize the movement as an outside…

Yes, I've never met anyone who stated they have "strong opinions, weakly held" who wasn't A) some kind of arsehole and B) lying.

Well, yeah, I think it's a pretty socially unaware thing to say about yourself out loud, so that's a pretty strong filter there.

It's rather different for a community to say that's a standard they aspire to, which is a lot less ridiculously grandstanding of a position IMO.

Re: Guess I'm a rationalist now

#913
post #87

The article made me think deeper about what rubs me the wrong way about the whole movement I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considere…

The rationalist discussions rarely consider what should be the baseline assumption of what if one or more of the logical assumptions or associations are wrong. They also tend to not systematically plan to validate. And in many domains - what could hold true for one moment can easily shift.

I am really not sure where you get any of these ideas. For each of your critiques, there are not only discussions, but taxonomies of compendiums of discussions about the topics at hand on LessWrong, which can easily be found by Googling any keyword or phrase in your comment.

On "considering what should be the baseline assumption":

https://www.lesswrong.com/w/epistemology

https://www.lesswrong.com/w/priors, particularly https://www.lesswrong.com/posts/hNqte2p48nqKux3wS/trapped-pr...

On the idea that "rationalists think that they can just apply rationality infinitely to everything":

https://www.lesswrong.com/w/bounded-rationality

On the critique that rationalists are blind to the fact that "reason isn't the only thing that's important", generously reworded as "reason has to be grounded in a set of human values", some of the most philosophically coherent stuff I see on the internet is from LW:

https://www.lesswrong.com/w/metaethics-sequence

https://www.lesswrong.com/w/human-values

On "systematically plan to validate":

https://www.lesswrong.com/w/rationality-verification

https://www.lesswrong.com/w/making-beliefs-pay-rent

On "what could hold true for one moment could easily shift":

https://www.lesswrong.com/w/black-swans

https://www.lesswrong.com/w/distributional-shifts

https://www.lesswrong.com/w/forecasting-and-prediction

Re: Guess I'm a rationalist now

#914
post #99

The article made me think deeper about what rubs me the wrong way about the whole movement I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considere…

My main problem with the movement is their emphasis on Bayesianism in conjunction with an almost total neglect of Popperian epistemology. In my opinion, there can’t be a meaningful distinction made between rational and irrational without Popper. Popper injects an epistemic humility that Bayesianism, taken alone, can miss. I think that aligns well with your observation.

Here's a collection of debates about that topic:

https://www.lesswrong.com/posts/85mfawamKdxzzaPeK/any-good-c...

I personally don't have that much of an interest in this topic, so I can't critique them for quality myself, but they may at least be of relevance to you.

Re: Guess I'm a rationalist now

#915
post #602
post #408

Earlier quoted context omitted.

Calling yourself rationalists: frames everyone else as irrational. It reminds me Kier Starmers Labour, calling themselves "the adults in the room". Its a cheap framing trick, belying an emptiness on the people using it.

I agree 100%, and that's my main issue with them. To build a group with its identity centered around "we form our opinions with logical inquiry from first principles" implies that you think that everyone else is doing something else. In reality, we just end up with a lot of worldviews and arguments that seem suspiciously like they are nothing more than people advocating for their own interests using some sophistry th…

I am sure that there are some people who exhibit the behaviors you're describing, but I really don't think the group as a whole is disinterested in prior work or discussion of philosophy in general:

https://www.lesswrong.com/w/epistemology

https://www.lesswrong.com/w/priors

https://www.lesswrong.com/posts/2x67s6u8oAitNKF73/ (a post noting that the foundational problems in mech interp are grounded in philosophical questions about representation ~150 years old)

https://www.lesswrong.com/w/consciousness (the page on consciousness first citing the MIT and Stanford encyclopedias, then providing a timeline from Democritus, through Descartes, Hobbes,... all the way to Nagel, Chalmers, Tegmark).

There is also sort of a meme of interest in Thomas Kuhn: https://www.lesswrong.com/posts/HcjL8ydHxPezj6wrt/book-revie...

See also these attempts to refer and collate prior literature: https://www.lesswrong.com/posts/qc7P2NwfxQMC3hdgm/rationalis...

https://www.lesswrong.com/posts/xg3hXCYQPJkwHyik2/the-best-t...

https://www.lesswrong.com/posts/SXJGSPeQWbACveJhs/the-best-t...

https://www.lesswrong.com/posts/HLJMyd4ncE3kvjwhe/the-best-r...

https://www.lesswrong.com/posts/bMmD5qNFKRqKBJnKw/rigorous-p...

Now, one may disagree with the particular choices or philosophical positions taken, but it's pretty hard to say these people are ignorant or not trying to be informed about what prior thinkers have done, especially compared to any particular reference culture, except maybe academics.

As for the thing about Aella, I feel she's not as much of a thought leader as you've surmised, and I think doesn't claim to be. My personal view is that she does some interesting semi-rigorous surveying that is unlikely to be done elsewhere. She's not a scientist/statistician or a total revolutionary but her stuff is not devoid of informational value either. Some of her claims are hedged adequately, some of them are hedged a bit inadequately. You might have encountered some particularly (irrationally?) ardent fans.

Re: Guess I'm a rationalist now

#916

Earlier quoted context omitted.

> So I forgot about the slightly condescending, lecturing tone that comes out when you disagree with rationalist figureheads. :) How was it condescending or lecturing? > You could simply ask "Can you provide examples" instead of the "If you ____ then I suggest ____" form. Why is that not equally condescending or lecturing?

> Why is that not equally condescending or lecturing? It just isn't! An inability to accurately identify what comes off as condescending is kind of the point...

That sounds an awful lot like other people making stuff up to oppress me by sticking a "condescending" label to me without me having any way to contest it.

Re: Guess I'm a rationalist now

#917

Earlier quoted context omitted.

> But they treat those issues as if they'll work out in the wash. there's something pathologically, virally sick about wealth accumulation's primary function being the further accumulation of wealth. for a movement rooted in "rationalism," EA seems pretty irrationally focused on excusing antisocial behavior.

> wealth accumulation's primary function being the further accumulation of wealth Then we are lucky that EA promotes giving more to charity as the primary function of accumulation of wealth. > EA seems pretty irrationally focused on excusing antisocial behavior. Is a guy getting a well-paid job at Microsoft and donating half of his salary to African charities really your best example of antisocial behavior?

What it promotes and what happens are two different things.

“The purpose of a system is what it does.”

Re: Guess I'm a rationalist now

#918

Earlier quoted context omitted.

Looking at Grok especially, it doesn't feel like a given that you can have a true SOTA model that is properly brainwashed.

To be fair, Musk is probably the least subtle megabillionaire of them all, and that reflects in the odd behavior of his silicon child. I don't doubt in the competency of the likes of Thiel to build their techo-monarchy.

The problem for them is that it might be a rather fundamental limitation. We already know that RLHF makes models dumber. It is entirely possible that, in order to make the model buy fully into what those people are peddling, the amount of forceful training required would crater the model's overall performance.

Re: Guess I'm a rationalist now

#919
post #187

Earlier quoted context omitted.

The ‘rationalist’ group being discussed here aren't Cartesian rationalists, who dismissed empiricism; rather, they're Bayesian empiricists. Bayesian probability turns out to be precisely the unique extension of Boolean logic to continuous real probability that Aristotle (nominally an empiricist!) was lacking. (I think they call themselves “rationalists” because of the ideal of a “rational Bayesian agent” in economics…

Bayesian inference is very, very often used in the types of philosophical/speculative discussions that Rationalists like instead of actual empirical study. It's a very convinient framework for speculating wildly while still maintaining a level of in-principle rationality, since, of course, you [claim that] you will update your priors if someone happens to actually study the phenomenon in question. The reality is that…

Beautifully put.

Re: Guess I'm a rationalist now

#920

The article made me think deeper about what rubs me the wrong way about the whole movement I think there is some inherent tension btwn being "rational" about things and trying to reason about things from first principle.. And the general absolutist tone of the community. The people involved all seem very... Full of themselves ? They don't really ever show a sense of "hey, I've got a thought, maybe I haven't considere…

A good question to ask: what differentiates X from everything else? So, how is "rationalism" different from everything else? What warrants this distinction? It can't be the use of reason or some refuge for rational discussion. I don't think I have to explain why that would be a ridiculous position to take.

> It can't be the use of reason [...] I don't think I have to explain why that would be a ridiculous position to take.

I think it would be better if you did, because otherwise you leave me guessing what your argument is, and if I guess wrong then I just wasted a lot of words for no good reason.

My guess is that you meant some combination of: "everyone is using their reason" and "everyone believes that their approach is the reasonable one" or "look at all those awesome scientists and philosophers, how many smart words they wrote and how many awesome inventions they made". Which is all true. But I wish I knew which one of these is closest to your objection, because they are not the same; the reason that everyone uses is obviously not the same degree or kind as the reason the awesome scientists use.

In my opinion, the thing that makes the rationality community different from everything else is a combination of things. None of these things, taken individually, is exclusive to the rationality community. Actually, most of the ideas, perhaps all of them, are taken from books written by someone else. The unique thing was (1) putting all these things together, (2) publicly, and (3) trying to take them seriously. As opposed to: only caring about one of these things and ignoring the rest, or teaching them only to selected people e.g. at school, or just discussing things without actually attempting to change your life because of them.

Here are some of the things:

* Probabilistic thinking. The idea that there is no absolute certainty, only probabilities. (Although some probabilities can, for all practical purposes, be very high or low.) That people should try to express their probabilities in numbers, and then calibrate themselves to get those numbers right. Getting probabilities right means that if you keep a log of all things that you have assigned to e.g. a 30% probability, then statistically about 30% of them should turn out to be true in long term. This was specifically taught at the rationality minicamps; they even made an app for that. Some math related to this.

* The consistency of reality. As opposed to e.g. the idea that science and religion are "separate magisteria" and each follows different laws of logic and probability. Nope. The laws of logic you use in your lab are exactly the same as the laws of logic you should use outside the lab. Just like gravity does not stop applying when your job is over, neither do the laws of evidence. Science is not a special world that plays by special rules; the atoms you study in the lab are the same atoms that the entire world is made of.

* People are "predictably irrational" (to use the popular phrase). Yes... and now that you know, you should do something about that. Even if you can't be perfect, that doesn't mean there is no low-hanging fruit to pick. Learn about the most frequent types of mistakes humans make, try to notice how it feels inside when you are doing that, and either try to avoid such situations, or try to recognize the feeling when it happens, or at least use some reflection to notice it afterwards. Together try to create an environment where those mistakes are easier to avoid.

* Notice the many ways how words can fail to reflect reality. For example, if you use the same word for two different things, it prevents you from noticing their differences. (But it is a natural mistake to make if those things are indeed similar, and if in the past they were interchangeable and just recently stopped being so.) How people incorrectly assume that everything that can be understood has a simple explanation, because that was kinda true in the jungle where our ancestors evolved. Etc.

* Reality matters. If your model of the world does not match reality, it is your model that is wrong. If you learn that fairies do not exist, that does not mean that the world suddenly became less magical -- it was always the same as now, only you didn't know it.

* Notice how you could do things better. Then actually do it.

* Ethics. Consequentialism and its problems. Good reasons to avoid temptations that seem convincing at the moment.

...and many more, sorry this already took too much time to write. If you are curious, https://www.readthesequences.com/ -- but the point is, this is not just about reading an interesting text, but actually changing yourself. And of course, there are many people who read the texts and don't change themselves. That cannot be prevented, especially when the text is freely available online. The rationality community actually had many workshops where people could practice these skills, so the opportunity was there.

Again, none of these things is scarce individually. It's the combination. For example, people who do math + care about improving the world = effective altruism. Trying to counter biases + caring about truth = steelmaning your opponents. Probabilistic thinking + reality matters = even if there is only 30% chance that a superhuman AI might kill us all, we better try hard to reduce that chance. The synergy of all of that... that is the goal that the rationality community is moving towards.

Post reply on HN