Earlier quoted context omitted.
I don't think it's very reasonable to think "AI" is more of an existential threat than global warming and nuclear weapons. In fact I'd say that's a ridiculous claim. The only way I can see AI causing total extinction is a Terminator-like scenario where an AI completely takes over, self-sustains by manufacturing killer robots and running power plants etc. It's literally science fiction. CharGPT is cool and all, really…
> I don't think it's very reasonable to think "AI" is more of an existential threat than global warming and nuclear weapons. In fact I'd say that's a ridiculous claim. AI x-risk is effectively a superset of global warming, nuclear war, engineering bioweapons, grey goo scenario, lethal geoengineering, and pretty much anything else that isn't just Earth winning the cosmic extinction lottery (asteroids, gamma ray bursts…
I also don't agree that helping humans make scientific progress is a danger. We already have the tools to wipe ourselves out, adding more of them doesn't really change much. It might well help us discover ways to improve things, and whatever we discover it's up to us how we use it.
We don't know what the future holds. GPT-4 may be close to the limit of current possibility. It is not given that we will discover significant improvements. Even if we do discover significant theoretical improvements it is not given that it will be feasible with current nor near-future hardware.
I can agree that there exists hypothetical potential for danger, but to put that hypothetical risk higher than real threats is in my view exaggeration.