It is not a matter of morality. Suppose you have 100 billion robots, each programmed
not to press a world-ending button. Some of them
will press the button. It's a matter of statistics and the moral framework of the other robots doesn't even enter into the equation. That's exactly the situation we're in.
Moral progress has always trailed behind our capabilities, and if I may say something heretical here, maybe it's also been driven by technological progress.
From the moment on where the first knife was used to attack a fellow human being, our capability to do harm has always been greater than our moral inhibition to do so. And it still is like this, by the way, with the humble knife. It seems we've never mastered the art of handling it responsibly, people are still being stabbed to death.
However, I think it's also important to point out that we're dealing with outliers. Knife murder is not something a person typically does during the average day, nor is it being a bomber, or a Bond-type villain researching world-ending microbes. At the same time though it's important to recognize that the potential of these outliers to do harm to large numbers of people is only going to grow. Concocting the next deadly plague that might kill millions is now within the grasp of determined single crazy people. It's kind of astonishing how little the media has caught on to this profound game changer that has been decades in the making.
We live in an age where a single person could very well wage war against a huge group of (possibly defenseless) people. That's why the morality argument here is so weak: even supposing an ethically perfect society, we'll never have the statistical certainty to exclude destructive outliers.
I don't have a solution for this dilemma, of course, but it's not as "simple" as merely complaining about the slow pace of our collective morality; or, more insidiously, calling for technological regression as a measure to uphold the common good. The growing disproportional power individual crazy people are wielding makes a reasonably good explanation for the causes behind the Fermi paradox, so I don't think we'll get past this any time soon - at least not until reasonable technological protection is available that doesn't at the same time enslave the civilization it's supposed to protect.
In closing, I think there's hope. The cold war was a first testing ground for the whole effect, and the fate of the world rested repeatedly on the moral actions of individuals. I'm still somewhat incredulous that it all worked out in the end. By the same token, even though inflicting mass casualties is now definitely within the reach of single people, there's a distinct absence of world-ending catastrophes, and more importantly: there's even an absence of people who actually tried to do it.
We might be OK, we might not. But it's not down to large-group morality somehow playing catch-up with technology. The call to artificially inhibit technology "until we're ready" seems a downright unethical proposition to me. It could be many, many generations until things like religious delusions finally die out, but you can be absolutely certain that will never happen in a world that is artificially tech-restricted. I thoroughly believe the only way forward is, you know, the way forward.