Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Npm Audit: broken by design?

overreacted.io

Post reply on HN
Print view View original

11–20 of 268 posts

Re: Npm Audit: broken by design?

#11

Post by cphoover »

Vulnerabilities are just code paths that can behave in unexpected ways and be abused.... I'm not sure the author's point, that development configuration could not hide malicious code? Why not?

All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it.

I think it's totally worthwhile to figure out which tools rely on insecure dependencies.

Also looks like you can specify npm to ignore dev dependencies:

> Any packages in the tree that do not have a version field in their package.json file will be ignored. If any --omit options are specified (either via the --omit config, or one of the shorthands such as --production, --only=dev, and so on), then packages will be omitted from the submitted payload as appropriate.

Re: Npm Audit: broken by design?

#12

Post by z5h »

Seems to me that if you don't have a pure functional language with tree-shaking, and only including existing (non-generated) code, you won't know if any (3rd party) code in particular will be called in compiled output, and you'll just have to guess and err on the side of paranoia Is there a fix?

Re: Npm Audit: broken by design?

#13

Post by mbesto »

> this “vulnerability” is absurd in this context

Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities.

If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense.

I do agree with the author's note about providing a better way to provide feedback on severity reviews.

npm audit is better than no npm audit...telling people it's broken by design is going to discourage them from using it completely. smh.

Re: Npm Audit: broken by design?

#14

Post by danabramov »

cphoover wrote: post #11

Vulnerabilities are just code paths that can behave in unexpected ways and be abused.... I'm not sure the author's point, that development configuration could not hide malicious code? Why not? All NPM does is scan to the dependency graph for vulnerability reports, it doesn't make any assessment of your consuming application's use-case. If you don't find this useful that is fine, don't use it. I think it's totally wor…

>I'm not sure the author's point, that development configuration could not hide malicious code? Why not?

Quite the opposite! Quoting the article:

As any security professional will tell you, development dependencies actually are an attack vector, and perhaps one of the most dangerous ones because it’s so hard to detect and the code runs with high trust assumptions. This is why the situation is so bad in particular: any real issue gets buried below dozens of non-issues that npm audit is training people and maintainers to ignore. It’s only a matter of time until this happens.

My point is that in the sea of non-issues, real issues are easy to miss and ignore.

>If you don't find this useful that is fine, don't use it.

You can't "not use it" because it's literally the default behavior built into `npm install` now. Of course there are ways to opt out, but this doesn't alleviate the confusion.

Re: Npm Audit: broken by design?

#15

Post by SahAssar »

quintex wrote: post #6

As someone who only had just gotten into front end programming after years of backend work, npm has been a nightmare. I haven't experienced the same level of frustration with other package managers (pip, cargo, go mod, etc) as I have with npm. Is yarn the better option? What is our path forward?

I think the problem is not really NPM it's the way that mainstream frontend development requires very wide and deep dependencies. If you stick to using smaller frameworks and libraries without heavy build tools then it's not much of a problem.

Re: Npm Audit: broken by design?

#16

Post by raziel2p »

mbesto wrote: post #13

> this “vulnerability” is absurd in this context Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities. If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense. I do agree with the author's note about…

Exactly. Why does it matter if it's absurd in this context? Just upgrade it to be on the safe side. Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea, even more so when you consider that the line between static and dynamic have gone blurry long ago.

Re: Npm Audit: broken by design?

#17

Post by _jal »

Macha wrote: post #2

This is a general problem with many security scanning tools, and when a security team is empowered to give deadlines to fix any issue they report, leads to much frustration and poor relations in teams. Imagine if you had 3 days to fix the regex DoS issue shown there, screw your release freeze and your current sprint plans, and you have the real working environment in some companies. I've also heard reports of people…

Among other things, it illustrates the insufficiency of a single numeric value for assessing 'badness', which in turn masks a management issue.

CVEs try to supplement with flags for remotely exploitable, etc. but it still intentionally leaves a lot of space for interpretation, which is necessary for any normal enterprise.

The problem comes in when analysts (or their managers) interpret inflexibly, without appropriate technical context, or without understanding business impact and tradeoffs.

If you look at the workflow, it is hard to close the loop from engineering or IT back to security. We need a set of controls for secops departments' output relevance and departmental interoperation.

Re: Npm Audit: broken by design?

#19

Post by nonameiguess »

It isn't broken by design. I sympathize and understand it's annoying to have to comb through false positives and mark them as such, but until a level of AI we're nowhere near being near exists, you can't automate this process, so the only alternative is to ignore all vulnerabilities.

Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are sensitive by default because the cost of lots of false positives is annoyed developers who have to slow down their delivery cadence. The cost of false negatives is anything from compromising user data to losing your company to bringing down the power grid of a major city depending on what the application is.

Which of those is a higher cost? npm can't possibly know the answer to that, so it has to default to assuming security actually matters to you. If it doesn't, your local policies can be more lax, but don't expect the tool to change for you.

Re: Npm Audit: broken by design?

#20

Post by dale_glass »

It's a tricky problem to solve.

Ideally you'd want to show only relevant alerts, but... how? You'd need to know which kind of errors are relevant for a particular project, but that'd require solving the halting problem. This is made much worse by that it's JS.

Some libraries have an enormous complexity and attack surface. Take a database interface -- there probably is a vulnerability in some obscure corner the typical person may not even know exists.

I think though at the very least some improvement could be made by better priorization and categorization. DoS by exploiting a regex parser isn't that big of a deal if your project is just getting started, but an exploit allowing arbitrary code execution would still be.

Post reply on HN