NPM's release cooldown is security theater

https://www.npr.org/sections/memmos/2016/06/09/605796769/che...

I'm confused by this take. In general, LLMs don't replace the expertise required to use an LLM.

Knowing the right input to give the LLM is not enough, nor does it ever end. The LLMs themselves need to be continuously updated and/or specialized for use case, and the "right input" is a moving target.

Keeping both ends of this aligned is the exact same expertise we already had before LLMs. These implementation details would only impact the credibility of those researchers who don't adapt to using this new tool. Who would stumble over something as trivial as this? LLMs aren't even the most important tool in the toolbox anyway.

> But if it turns out they can’t serve as the cooldown vanguard, then we have great evidence that they shouldn’t be trusted at all.

If they cannot serve as the cooldown vanguard, we are indeed going to realize they are not trustable, but by the time the damage has already been done. Therefore, if companies want to prevent the damage as much as possible, I believe they should do the scan by themselves.

Account takeovers, deployment exploits whatever the root cause, given a few days it's incredibly likely maintainers will notice they've released an unexpected update and fix/sound an alarm. A lot of existing cases like this people realise pretty fast, but the packages might still be live for a few hours as NPM catches up. Cooldown entirely removes that 'updated at the wrong time' issue, an no-one even needs to actually look at the code at all.

For inactive or intentionally malicious maintainers, yes it relies on vetting. But it's hardly a mythical 'canary' - there are already a lot of companies scanning every popular npm package, auditing new ones, or just generally analysing threats. I can think of at least 4 such companies without even trying, I'm sure there's loads more.

And they'll continue because it's not warm hearted so much as promoting/testing their security scanning services and tools so they can get the kudos from being the first to spot X etc. Which often does hit the front of HN or reddit, but they don't need the massive cyber loudspeaker to be effective. Flagging the issue to maintainers, reporting to NPM security - with a cooldown in place the package can be voided before anyone really gets the chance to install it. This is literally how a lot of things are spotted already, just with that 'updated at the wrong time' issue catching unlucky people out.

Does it solve everything? No. Will it reduce the amount of problems that make it to end users? Obviously yes.

And for all the author's 'no one will do this', 'waiting for Godot' cries - what do they recommend? Running your own audits, using LLMs, static analysis and testing it yourself. Basically being the veter they claim no-one will be! Unless they don't intend to sound the alarm themselves, they are literally the canary they're looking for.

Cooldowns aren't magic, but they're not security theater either.