Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Npm Audit: broken by design?

overreacted.io

Post reply on HN
Print view View original

51–60 of 268 posts

Re: Npm Audit: broken by design?

#51

Post by cratermoon »

bogota wrote: post #27
nonameiguess wrote: post #19

It isn't broken by design. I sympathize and understand it's annoying to have to comb through false positives and mark them as such, but until a level of AI we're nowhere near being near exists, you can't automate this process, so the only alternative is to ignore all vulnerabilities. Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are s…

That is not the cost of false positives. The cost of false positives is ignoring it completely. I have had jobs where we just block the security scanners because they won’t listen to any feedback about why what they are scanning was intentionally setup for the purpose of testing vulnerabilities on an internal only network. Additionally at other jobs security tickets just start to get ignored because they send too man…

I agree about the false positive problem. Boy who cried wolf and all. I've also worked with security vendors who offer to run "free" vulnerability scans for you, and to absolutely nobody's surprise, they find vulnerabilities that just happen to be the ones that they can fix, if you buy what they are selling.

Still, your example is problematic. Beware the "internal-only network". Such a thing has mostly lost meaning today, and it was never much more than a picket fence anyway. "All devices must be capable of maintaining their security policy on an un-trusted network." https://collaboration.opengroup.org/jericho/commandments_v1....

Re: Npm Audit: broken by design?

#52

Post by three14 »

nonameiguess wrote: post #19

It isn't broken by design. I sympathize and understand it's annoying to have to comb through false positives and mark them as such, but until a level of AI we're nowhere near being near exists, you can't automate this process, so the only alternative is to ignore all vulnerabilities. Complaining about this is misunderstanding the asymmetrical costs of different types of statistical error. Vulnerability scanners are s…

The alternative is hiding most of the reports by default. It's a firehose, and almost all of it is guaranteed to be useless. For example, denial of service in a devDependency shouldn't be shown unless the user specifically requests it. Denial of service shouldn't be marked "high" importance, even though occasionally it might be quite important to some user.

Re: Npm Audit: broken by design?

#53

Post by paulddraper »

esens wrote: post #29

I found that much of the underlying cause is those mass reporting regex denial of services as being high severity bugs. So many people are reporting these in tons of different projects: https://github.com/search?q=regex+denial+of+service&type=iss... Anyhow it is just annoying and they broke NPM Audit based on these reports. It is good to fix all possible bugs, but many of these are not anywhere close to the level of…

Add prototype pollution and you've covered 90% of all "vulnerabilities"

Re: Npm Audit: broken by design?

#54

Post by akoumjian »

The interaction is not the worse thing about npm audit. The security model of the tool has a big hole depending on how you use it: https://mulch.dev/blog/CVE-2020-5252-python-safety-vuln/

In essence, if you are scanning an environment that is already compromised, `npm audit` results can't be relied upon if you are running it in the same environment. It should be self-evident but I'm sure plenty of people use the tool this way.

Re: Npm Audit: broken by design?

#55

Post by cratermoon »

zinekeller wrote: post #44
cratermoon wrote: post #34

Earlier quoted context omitted.

This kind of nonsense really goes back to the broken CVE process. https://opensourcesecurity.io/2021/03/30/its-time-to-fix-cve... Linux kernel maintainer Greg Kroah-Hartman has a similar opinion. https://github.com/gregkh/presentation-cve-is-dead/blob/mast... Edit: LWN mention https://lwn.net/Articles/801157/

The view of SQLite developers on CVEs is also dim: https://www.sqlite.org/cves.html

Beautifully succinct. This quote: "Grey-hat hackers are rewarded based on the number and severity of CVEs that they write. This results in a proliferation of CVEs that have minor impact, or no impact at all, but which make exaggerated impact claims." Alignment of incentives is messed up. Goodhart-Strathern's and Campbell's laws apply.

Re: Npm Audit: broken by design?

#56

Post by nalekberov »

> It makes beginners miserable

Can we please tell beginners not to start programming with node.js?

Teaching beginners to start with “go-to” technologies became industry standard already as it helps corporations to become more and more monopolist and dictate new industry standards.

Re: Npm Audit: broken by design?

#57

Post by city41 »

raziel2p wrote: post #16
mbesto wrote: post #13

> this “vulnerability” is absurd in this context Yes that's exactly the point. The audit tool has no awareness of the context and nor do the people who create severities. If severities were absolute then there would be no reason for anyone to review them. You would simply upgrade your libraries and be done with it, but that can't always be achieved nor may make business sense. I do agree with the author's note about…

Exactly. Why does it matter if it's absurd in this context? Just upgrade it to be on the safe side. Asking vulnerability databases to judge whether vulnerabilities are safe in devDependencies or not is a ridiculous idea, even more so when you consider that the line between static and dynamic have gone blurry long ago.

> Just upgrade it to be on the safe side

Upgrading a dependency can go anywhere from trivial to absolute nightmare. Usually somewhere in the middle where it takes time and effort to do right. A typical JS app nowadays has hundreds if not thousands of dependencies. I'd love to see a world where "just upgrade" is reasonable advice, but we are not there.

Re: Npm Audit: broken by design?

#58

Post by preinheimer »

Honestly, this hit me.

I'm not a react developer, I was experimenting with it for a new project. I finished the tic-tac-toe tutorial, then tried to throw bootstrap on top to build from there. It told me there was 97 vulnerabilities (85 moderate, 12 high)...

I just deleted the directory and went back to vanilla JS. This is a fun side project, I don't need that.

My tweet about it: https://twitter.com/preinheimer/status/1402785757962592256

Re: Npm Audit: broken by design?

#59

Post by nkozyra »

JonathonW wrote: post #46
esens wrote: post #29

I found that much of the underlying cause is those mass reporting regex denial of services as being high severity bugs. So many people are reporting these in tons of different projects: https://github.com/search?q=regex+denial+of+service&type=iss... Anyhow it is just annoying and they broke NPM Audit based on these reports. It is good to fix all possible bugs, but many of these are not anywhere close to the level of…

A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users. But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a…

> A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users.

But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?

Re: Npm Audit: broken by design?

#60

Post by esens »

nkozyra wrote: post #59
JonathonW wrote: post #46

Earlier quoted context omitted.

A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users. But NPM Audit has no idea of context-- a "critical" bug in `browserlist`, which, in this context, is never used outside the development process and never takes input outside of what's in my package.json, gets the same prominence (or more so, since it's early in alphabetical order) as a…

> A regex "denial of service" "vulnerability" could be important, if it shows up in code that processes untrusted input from end users. But in this context what's the end result? Chrome locking up on the end user's (attacker's) machine? Again, an "attacker" doesn't have access to the source code for distribution. By inputting bad regexp data they're only DOSin themselves, no?

Could be in a service on a server, which in this case a RegEx DOS could lock the server for all users.
Post reply on HN