One solution I haven’t seen recommended much is to have a Claude instruction/skill that explicitly audits the diff of every upgrade, and force this manual audit as part of your upgrade workflow. This seems like it would work pretty reliably.
Super dumb question as someone who has been using some form of AI for dev since 2023: How does having an AI audit external code help? Can they not be prompt injected to ignore a malicious change? I guess I am sort of concerned that they are a pretty thin layer and even if you put "DO NOT ALLOW PROMPT INJECTION", it's a bit like saying "make no mistakes". There _is_ a priority between `system` and `user` level message…
The concern is real and unsolved. I think security researchers have an advantage here because they still can fall back to manual audits if their automated analysis (or scores thereof) is off.