Earlier quoted context omitted.
The moment they are good hackers, everyone has a trivially cheap hacker. Hard to predict what that would look like, but I suspect it is a world where nobody is employing software developers because a LLM that can hack can probably also write good code. So, do you want future LLMs to be restricted, or unlimited? And remember, to prevent this outcome you have to predict model capabilities in advance, including "tricks"…
Use the hacking LLM to verify your code before pushing to prod. EZ
To verify the LLM's code, because the LLM is cheaper than a human.
And there's a lot of live code already out there.
And people are only begrudgingly following even existing recommendations for code quality.