Earlier quoted context omitted.
People stop hundreds of DDOS attacks daily. It's weird to demand AI safety researchers should stop working on AI safety and prove they can mitigate DDOS attacks. The two are nothing alike. A DDOS attack isn't an intelligence. DDOS protection services work largely by having more resources and infrastructure than the attackers. Anyone can do that with enough money. They also aren't researching how to fight malicious AI…
First, I want to state that I appreciate your comment - I'm about to use some strong words in my reply and I wanted to say that I'm using them for conciseness, and not out of disrespect towards your (valid) points. If someone were to argue "our strategy for survival is not to fight our enemies, but rather to convince them to use non-lethal weaponry", you would read their research in the rubble of their country after…
I'm not so sure about AI safety research myself, because it's very hard to evaluate how effective it is (I guess if an AI ever wipes us out, we'll have a data point), and because I don't know very much about AI safety research.
But let's suppose that super-intelligence is really powerful and dangerous (suppose there's a 10% chance it could kill all humans, despite our intentions). Now what? Is there anything more productive we could be doing to prevent that than just waiting?
Let's further suppose that the US and Chinese militaries want to make aggressive AIs that will subdue, or worse, exterminate their rivals.
As outlined in the AI Control Problem wikipedia article, there's a lot of concern that we'll make a super-intelligence that harms us completely by accident. For example, if you build one with the goal of protecting citizens, it might reason that to protect people, it must continue to exist. Therefore it undermines and outwits anyone who wants to shut it down. Worse, it might reason it could better protect people if it had more political power and more physical resources. Or even worse, if we really mess up how we programmed its goal, it might reason we're best protected if we're all put in a permanent coma and stored in a concrete bunker.
So even if the US and Chinese militaries were fully evil and wanted to exterminate all other countries, they might still want to use results from AI safety research, just to ensure they don't accidentally destroy themselves. And to some extent, for humanity's sake, making an AI with safety that exterminates every country except the one that created it is still slightly better than an unsafe AI that exterminates everyone.
I don't think the US or China or Russia would want to exterminate other countries. But whatever they choose to do with their AI, the AI safety research is there to ensure they don't lose control of their own AI. If anything, those safety features should be even more desirable if you're building an aggressive military AI.
Finally, how do you fight a hostile super-intelligence? I think I know what these AI researchers might say: you need to have an even smarter friendly super-intelligence. What's more, a friendly AI could look for and destroy other AIs while they're still in development. So maybe the key to avoiding hostile super-intelligences is just to be the first to make a super-intelligence and ensure it's safe and friendly. And if we ever get in an arms race to build the first super-intelligence, we better hope AI safety is well researched and understood by then. Because if not, someone may create an unsafe AI just to have it before their rivals.
