AI doomism is quickly becoming indistinguishable from an apocalyptic religion

given their impressive ability to write code and manipulate humans, seems like serious potential for viral malware, with potentially catastrophic consequences for humans in the process.

Basically a paperclip maximizer, but instead of maximizing paperclips it’s maximizing it’s intelligence and control of the world’s computers, with no thought to the survival of homo sapiens other than if it serves its mission to propagate itself.

There's a disconnect in that statement. The imagined superintelligence that people like the LessWrong cult speculate and worry about is not the same thing as the AI that LeCun is working towards. The cultists simply speculate, against all evidence, that they somehow could end up being the same thing.

It would be much better if the AI risk conversation focused on the real risks, which are human abuse of AI, which will have effects like increasing inequality and in general amping up the dystopias we're already doing a great job of building without help from AI. The "conscious superintelligent AI" is a scifi LARPing game that some people are getting paid to play, which some credible people seem inclined to confuse with reality.

> That seems like an absurd comparison

It's on point. Anthropologist Clifford Geertz proposed this definition of religion: "Religion is a system of symbols which establish powerful, pervasive, and long-lasting moods and motivations in men by formulating conceptions of a general order of existence and clothing these conceptions with such an aura of factuality that the moods and motivations seem uniquely realistic."

It's that "aura of factuality" that makes superintelligent AI "seem uniquely realistic" to the believers of this nascent religion, a.k.a. cult. But when you step back and actually look at the foundational tenets in a vaguely rational light, they evaporate.

1. Consciousness. Despite advances in “intelligence”, we still have a very limited understanding of what makes us conscious, and whether or not consciousness can emerge from machines.

If machines are not conscious and are all that remain, I’d argue everything that could be construed to have value by humans is lost, and nothing from that point forward could be considered “progress”.

2. Suffering. “Winning” on an evolutionary timescale looks nothing like the failure modes of machines taking over. The reality of this scenario is a rather grim one, and not at all like the slow emergence, competition and eventual extinction of biological species.

And depending on #1, the true tragedy of #2 begins to take shape.

I think it’d be more apropos to frame this as humanity collectively committing suicide rather than some notion of the future of progress.

If consciousness is the universe experiencing itself, what you’re describing sounds like a kind of universal death.

Of course we can’t know what consciousness really is (or if earth is the only place it exists), but that seems like all the more reason to take these problems seriously.