Earlier quoted context omitted.
Openness in the context of AI is not straightforward. The open source folks read it one way, and the alignment people read it another. It is entirely possible a program that spits out the complete code for a nuclear targeting system should not be released in the wild.
Nuclear codes, assuming they are using modern cryptography would not be spat out by any AI, unless they were leaked publicly. Bigger concern would be the construction of a bomb, which, still, takes a lot of hard to hide resources. I'm more worried about other kinds of weapons, but at the same time I really don't like the idea of censoring the science of nature from people. I think the only long term option is to beef…
The average postgraduate in physics can design a nuclear bomb. That ship sailed in the 1960s. Anyone who uses that as an argument wants a censorship regime that the medieval catholic church would find excessive.