Npm Audit: broken by design?

It doesn't solve everything, but we use npm-audit-resolver[0] and it's.. workable. It presents you vulnerabilities, offers to fix (upgrade the nested dependency) if a version exists that meets all the constraints, and gives you an option to ignore for a week/month/forever if no fix exists. Those decisions (including fixes, which is a bit silly) get recorded in a JSON file in source control. For each group of ignores we add a link to the relevant Github issue where it's been reported, so if the ignore time we chose expires we can quickly go and see what the status is.

There are still problems:

- The decisions file gets unweildy, mainly because every time it fixes something it writes to the file. You probably only care about ignores. It's also append only, though you could manually clear it down sometimes.

- It always defaults to fixing at the deepest level, which is.. not ideal for NPM. On my machine (a not very old Macbook Pro) NPM simply can't update a dependency 20 layers deep in the tree, ie `npm update nested-dependency-from-hell --depth 20` will eventually time out and won't fix anything. So you have to manually crawl up tree yourself and find the thing that can be updated - or just ignore it until the thing right at the top of the tree gets updated.

I'm not surprised to see Dan posting this though. I agree with everything he said, so I don't mean this as an attack, but a lot of the time the thing at the top of the tree we're waiting for an update on is create-react-app. It must be incredibly annoying how many Github issues get opened on that repo every time there's a new NPM advisory on some 20-dependencies-deep parser it uses for something or other.

I do like the suggested fix that a maintainer can use their knowledge of the specific usage to say the vulnerability doesn't apply. Often in these threads there's a perfectly good explanation of why it isn't a real issue, and then people come back with "Okay but can you please update it anyway because I'm forced to audit and my security team/CI are yelling at me".

[0] https://www.npmjs.com/package/npm-audit-resolver

This kind of nonsense really goes back to the broken CVE process. https://opensourcesecurity.io/2021/03/30/its-time-to-fix-cve...

Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast...

Edit: LWN mention https://lwn.net/Articles/801157/

Indeed, npm is not aware of the context of the vulnerabilities. That does not invalidate them, however, and mean they should be hidden. I've worked in offensive security for quite long enough (been a dev for 10+ years before) to tell you that your context, or the most common one, isn't all that exist and someone's going use it the way it makes the app vulnerable. Based on the article's example, someone's going to build an app that builds an app.

A vulnerability is a vulnerability, whether it applies to your context or not. A metaphor might be: "The passenger door is broken on my car, but I'm the only one to use it". Seriously, who, in their right mind, is going to argue that the door isn't broken?

- If your dependencies have security vulnerabilities, apply the updates.

- If you cannot update because there's no fix available, let your org or you assess the risk and go from there.

- If you cannot update because it breaks your app, {find a replacement, fix it yourself, let your org or you assess the risk and from from there}.

A sensible org has a process that freezes releases until known security issues are fixed. Freezes can also be opposed by devs and are evaluated on a case-by-case basis (because sometimes they are not relevant to the product, or someone steps up to take the blame for incidents and the org agrees).

We might not like it because it disturbs the "flow", but it's just part of the engineering process. More to the point though, why not take this opportunity to teach newcomers how to code properly, pick well-engineered and -written programs, and handle this vulnerability management process altogether? In any case, I hope newcoming-dev is not going to push to prod anytime soon. ;)

edit: formatting

Yep, some vulnerable package isn't even in the compile output. How a dev server that only binds to 127.0.0.1 a serious DOS problem? Who on the earth will want to DOS that?

I don't think anyone should be blindly upgrading anything. Doing so brings about its set of problems and in many cases makes things worse. This is particularly true in the case of Create React App where just upgrading dependencies will often break the application.

The last Create React App I worked on (~3 months ago) had over 500 "vulnerabilities" reported by npm/yarn audit. Most of the reported vulnerabilities were obviously junk. As the author noted, there's no need to report vulnerabilities in the same dependency in every path through the dependency graph. The noise made it very difficult to sift through the output for anything useful. Even then, I have my doubts about how applicable the results are because with tree shaking of an SPA, it's quite possible the vulnerable part of a dependency is never even used.