Earlier quoted context omitted.
As someone with a background in AI and who has been playing around with neural nets for decades at this point, it's been genuinely amazing watching extremely intelligent people make confident predictions about AI capabilities and progress, then be so completely wrong. There's a kind of theory of mind for AI (specifically neural nets) which I now realise I seem to have which is very hard to explain to people who haven…
> extremely intelligent people make confident predictions about AI capabilities and progress, then be so completely wrong. The ability for the human mind to rationalize conclusions to maintain denial in the face of a very scary future is immense. Genuinely grappling with the implication of where we're headed is usually very crushing. It's not easy to engage with the possibility, and very intelligent people will use t…
As someone currently prepping for various AI doom scenarios and who has been dealing with AI-related nightmares for years this is very relateable.
Although, I don't personally think it's this. In my experience the opposite is more true – the majority of high probability doomers seem rather laid back about considering what they believe will happen to the people they love in a few years. Equally I don't get the sense those who don't have such extreme predictions are worried at all. If anything there's not enough emotion.
In my opinion people just don't reason well when it comes to exponentials and are ignorant about things they don't have good mental models of. At least I know I struggle with this.