711–720 of 947 posts
This article is beautifully written, and it's full of proper original research. I'm sad that most comments so far are knee-jerk "lol rationalists" type responses. I haven't seen any comment yet that isn't already addressed in much more colour and nuance in the article itself.
I get the impression that these people desperately want to study philosophy but for some reason can't be bothered to get formal training because it would be too humbling for them. I call it "small fishbowl syndrome," but maybe there's a better term for it.
Earlier quoted context omitted.
I honestly am not trying to be rude when I say this, but this is exactly the sort of speculation I find problematic and that I think most people in this thread are complaining about. Being able to tell Claude to have a go has no relation at all to whether it may ever succeed, and you don't actually address any of the legitimate concerns the comment you're replying to points out. There really isn't anything in this co…
On the other hand, I'm baffled to encounter recursive self-improvement being discussed as something not only weird to expect, but as damning evidence of sloppy thinking by those who speculate about it. We have an existence proof for intelligence that can improve AI: humans. If AI ever gets to human-level intelligence, it would be quite strange if it couldn't improve itself. Are people really that sceptical that AI wi…
> If AI ever gets to human-level intelligence
This picture of intelligence as a numerical scale that you just go up or down, with ants at the bottom and humans/AI at the top, is very very shaky. AI is vulnerable to this problem, because we do not have a definition of intelligence. We can attempt to match up capabilities LLMs seem to have with capabilities humans have, and if the capability is well-defined we may even be able to reason about how stable it is relative to how LLMs work.
For "reasoning" we categorically do not have this. There is not even any evidence that LLMs will continue increasing as techniques improve, except in the tautological sense that if LLMs don't appear to resemble humans more closely we will call the technique a failure. IIRC there was a recent paper about giving LLMs more opportunity processing time, and this reduced performance. Same with adding extraneous details, sometimes that reduces performance too. What if eventually everything you try reduces performance? Totally unaddressed.
> It that an insane belief worthy of being a primary example of a community not thinking clearly?
I really need to stress this: thinking clearly is about the reasoning, not the conclusion. Given the available evidence, no legitimate argument has been presented that implies the conclusion. This does not mean the conclusion is wrong! But just putting your finger in the air and saying "the wind feels right, we'll probably have AGI tomorrow" is how you get bubbles and winters.
A very interesting read. My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons. This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now. The article see…
You're falling into some sort of fallacy; maybe a better rationalist than I could name it. The "they" you are describing is a large body of disparate people spread around the world. We're reading an article that focuses on a few dysfunctional subgroups. They are interesting because they are so dysfunctional and rare. Or put it this way: Name one -ism that _doesn't_ have sub/splinter groups that kill people. Even Paci…
A very interesting read. My idea of these self-proclaimed rationalists was fifteen years out of date. I thought they’re people who write wordy fan fiction, but turns out they’ve reached the point of having subgroups that kill people and exorcise demons. This must be how people who had read one Hubbard pulp novel in the 1950s felt decades later when they find out he’s running a full-blown religion now. The article see…
You're falling into some sort of fallacy; maybe a better rationalist than I could name it. The "they" you are describing is a large body of disparate people spread around the world. We're reading an article that focuses on a few dysfunctional subgroups. They are interesting because they are so dysfunctional and rare. Or put it this way: Name one -ism that _doesn't_ have sub/splinter groups that kill people. Even Paci…
[Citation needed]
I sincerely doubt anything but a tiny insignificant minority consider themselves part of the "rationalist community".
Earlier quoted context omitted.
fine, the jargon of a "social group" of science is a red flag? sure, theres lots of nasty side effects of how academia is run, rewarded, etc.. but thats not because of precision of language employed. do you want scientists recycling the same words and overloading ever more meanings onto ever more ambiguous words?
I don’t think we disagree. I’m not taking issue with scientists having jargon, which I agree is good and necessary (though I think the less analytical academic disciplines, not being rooted in fact, have come to bear many similarities to state-backed religions; and I think they use jargon accordingly). I’m pointing out that I specifically intended to exclude professionals by scoping my statement to “social groups”. P…
I think your argumentation is a generalization that's close to a rationalist fallacy we're discussing:
> a social group with a lot of invented lingo is a red flag that you can see before you get isolated from your loved ones.
Groups of artists do this all the time for the sake of agency over their intentions. They borrow terminology from economics, psychology, computer science etc., but exclude economists, psychologists and computer scientists all the time. I had one choreographer talk to me about his performances as if they were "Protocols". People are free to use any vocabulary to describe their observed dynamics, expressions or phenomena.
As far as red flag moments go, the intent to use a certain terminology still prevails any choice of terminology itself.
Earlier quoted context omitted.
You're falling into some sort of fallacy; maybe a better rationalist than I could name it. The "they" you are describing is a large body of disparate people spread around the world. We're reading an article that focuses on a few dysfunctional subgroups. They are interesting because they are so dysfunctional and rare. Or put it this way: Name one -ism that _doesn't_ have sub/splinter groups that kill people. Even Paci…
The article specifically defines the rationalists it’s talking about: “The rationalist community was drawn together by AI researcher Eliezer Yudkowsky’s blog post series The Sequences, a set of essays about how to think more rationally.” Is this really a large body of disparate people spread around the world? I suspect not.
So I suspect yes, it's a large body of loosely coupled people.
Earlier quoted context omitted.
The reason why people can't be bothered to get formal training is that modern philosophy doesn't seem that useful. It was a while ago, but take the infamous story of the 2006 rape case in Duke University. If you check out coverage of that case, you get the impression every member of faculty that joined in the hysteria was from some humanities department, including philosophy. And quite a few of them refused to change…
Couldn't you take this same line of reasoning and apply it to the rationalist group from the article who killed a bunch of people, and conclude that you shouldn't become a rationalist because you probably kill people?
Earlier quoted context omitted.
The WHO didn't declare a global pandemic until March 11, 2020 [1]. That's a little slow and some rationalists were earlier than that. (Other people too.) After reading a warning from a rationalist blog, I posted a lot about COVID news to another forum and others there gave me credit for giving the heads-up that it was a Big Deal and not just another thing in the news. (Not sure it made all that much difference, thoug…
Do you think that the consequences of the WHO declaring a pandemic and some rationalist blog warning about covid are the same? Clearly the WHO has to be more cautious. I have no doubt there were people at the WHO who felt a global pandemic was likely at least as early as you and the person writing the rationalist blog.
Are the WHO personnel rational and competent? I would like to believe so. But that isn't a given - the amount of nonsense I had to fight in institutions considered as pinnacles of rationality is just depressing. Regardless, WHO was encumbered by international policitics. Their rationality would have made no difference. That is why the opinion of rational outsiders matter - especially of those with domain expertise.
The signs of an uncontained contagion were evident by the middle of December 2020, well before the WHO declared the pandemic in March 2021. They could have asked everyone to start preparing around then. Instead, there were alarming news coming out of Wuhan and endless debates on TV about the appeasement of the Chinese administration by WHO - things that started ringing the alarm bells for us. We started preparing by at least the middle of January. WHO chose to wait again till everything was obvious and a declaration was inevitable. People were dying by the thousands everyday and the lockdowns had already started by then. Their rubberstamp wasn't necessary to confirm what everyone knew already. That was one instance where waiting for the WHO wasn't a prudent choice.
WHO is a critical institution to the entire world. Their timing can mean the difference between life and death for millions everywhere. These sorts of failing shouldn't be excused and swept under the rug so easily.