Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Always Review Your Dependencies, AGPL Edition

agwa.name

Post reply on HN
Print view View original

121–130 of 236 posts

Re: Always Review Your Dependencies, AGPL Edition

#121

Post by codeflo »

anton_gogolev wrote: post #116
lmm wrote: post #98

Earlier quoted context omitted.

Dependencies are cattle, not pets. There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk. In the JVM ecosystem, projects are only allowed in the Maven Central repository if they have their license documented in a machine readable fashion there; it's then trivial to check what licenses you're depending on (and there are several plugins available for doing so). I'm amaze…

> There's nothing wrong with having zillions of them... There's nothing wrong until something goes wrong an now you're royally screwed. With zillion dependencies you are at a mercy of zillion maintainers, and none of them has any obligation to you. They can break backwards compatibility in patch releases, introduce subtle behavior changes, steer the project in an unexpected direction or abandon it altogether.

I’m a bit torn on this. I have most of my experience in the .NET ecosystem, where dependencies are a lot more manageable. However, if something breaks, you’re screwed a lot harder, because it’s not so easy to replace a large library, and there are very likely fewer well-maintained alternatives than there would be on NPM.

In total, I find it hard to deny how productive the NPM ecosystem can be, despite my philosophical objections to the way the community is run. Am I crazy here?

Re: Always Review Your Dependencies, AGPL Edition

#122

Post by saagarjha »

robbya wrote: post #118
generationP wrote: post #107

> Aside: this is why I don't like to accept pull requests that move code around. Even if the new code organization is better, it's usually not worth the time it takes to ensure the pull request isn't doing anything extra. This sounds like a solvable problem no one has bothered to solve. We need an analogue of diff highlighting for move-around changes, ideally one that decomposes a changeset into the coarsest block pa…

For changes that are a pure file rename 'git mv' tracks those pretty cleanly. Anything beyond that... I don't know of any good tooling either.

git mv doesn't actually do anything special to track moves, so it can't figure out anything that's non-trivial.

Re: Always Review Your Dependencies, AGPL Edition

#123

Post by tsteenbe »

rurounijones wrote: post #54
kijin wrote: post #15

Earlier quoted context omitted.

GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.

Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere? It makes license scanning a doddle.

> It makes license scanning a doddle.

As someone working in the field of license compliance (leading an Open Source Program Office) and dealing with the licensing of tens of thousands of OSS dependencies on a daily basis for a large company I can tell you that license scanning / compliance is anything but a doddle. I do a lot of public speaking on this topic for a summary of the issues I recommend you to have a look at https://static.sched.com/hosted_files/ocs19/c7/OSS-Review-To...

Most modern package managers do offer project maintainers a way to declare the license for a project. However I can tell you that often the declared license does not match the licenses detected in the source code. What counts is the license stated in the source code files not what it's in the gemspec, package.json, pom.xml etc.

This is quite common issue especially for older or larger OSS projects where various contributors have added new code over time that may be licensed under an OSS license that is compatible but not the same as the main license of the projects (Think adding BSD-2-Clause in Apache-2.0 project) What happens is that this contributions get accepted but the project maintainers do not to update the declared license.

Not saying declaring a licensing in a .gemspec file is useless but just recommend you to take it as an indicator of the main license of the project - the project may include source code that licensed under a different license.

Lack of clarity around licenses and security vulnerabilities is a big issue within the OSS community especially as lack of clarity reduces engagement — that means fewer users, fewer contributors and a smaller community. Several organizations are working on a open source solution for this community problems see for further details https://clearlydefined.io/about

Full disclosure I am one of the maintainer of OSS Review Toolkit and contributor to ClearlyDefined.io

Re: Always Review Your Dependencies, AGPL Edition

#124

Post by saagarjha »

rurounijones wrote: post #54
kijin wrote: post #15

Earlier quoted context omitted.

GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.

Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere? It makes license scanning a doddle.

An even older tradition is a LICENSE file in the root of your project, which GitHub seems to read, but it's not great at matching up the license text to actual licenses.

Re: Always Review Your Dependencies, AGPL Edition

#125

Post by jsjohnst »

syshum wrote: post #86
marcus_holmes wrote: post #70

Earlier quoted context omitted.

similarly, I refuse to use JS libs that have ridiculous dependency trees (most annoyingly, including Webpack, which means my Vue setup is more interesting than it needs to be). I was explaining why to a friend, and it appears that no-one takes this threat seriously, not even in secure/financial apps. Like you, I wonder if I'm missing something obvious. Why does the npm dependency trust nightmare give me the screaming…

Because even in 2020, even in " secure/financial apps" most programmers have security and lic legality as a secondary (or even forth level) concerns, they just want to "make something cool" They play lip service to it being secure, and/or license compliant

I would argue the reason developers don’t have concerns around licensing is partly due to lack of knowledge around the differences between licenses at a fundamental level and partly due to a “why should I care?” mentality at least at a subconscious level, if not higher.

A got a taste of it yet again this past weekend when I commented here on HN about GPL and AGPL being licenses that corporations often have concerns with.

Re: Always Review Your Dependencies, AGPL Edition

#126

Post by cygned »

anton_gogolev wrote: post #116
lmm wrote: post #98

Earlier quoted context omitted.

Dependencies are cattle, not pets. There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk. In the JVM ecosystem, projects are only allowed in the Maven Central repository if they have their license documented in a machine readable fashion there; it's then trivial to check what licenses you're depending on (and there are several plugins available for doing so). I'm amaze…

> There's nothing wrong with having zillions of them... There's nothing wrong until something goes wrong an now you're royally screwed. With zillion dependencies you are at a mercy of zillion maintainers, and none of them has any obligation to you. They can break backwards compatibility in patch releases, introduce subtle behavior changes, steer the project in an unexpected direction or abandon it altogether.

That goes into the direction of my thinking. I am concerned about transitive security issues. It is impossible to check in node dependencies into version control (size/binaries). They have a lock file to pin versions, but dependencies that are downloaded upon each build are are not reproducible from my point of view. With Go, it’s easy to vendor and check in, it’s also straight forward to review them. There have been examples of targeted attacks using npm packages and that is something I am very concerned about.

People move billions with a node.js application we develop and the company will eventually be liable if the system is compromised through a targeted attack.

On a different note, I think the ecosystem moves too fast, packages and versions are getting deprecated and new ones getting released constantly. I have the feeling that the whole ecosystem is targeted towards building small MVP apps, not relying a long-term business on it. Maybe I am too harsh here, but that is a frustration growing for years now. I am happy to be proven wrong.

Re: Always Review Your Dependencies, AGPL Edition

#127

Post by ratww »

rurounijones wrote: post #54
kijin wrote: post #15

Earlier quoted context omitted.

GitHub's license detection algorithm is crap. It tends to get GPLv2 right, but most other licenses are hit-and-miss. And I still can't find a setting where I can manually specify the license when GitHub can't autodetect one or gets it wrong.

Ruby gems have the license as an explicit field in the gemspec (manifest file). Is this not the case elsewhere? It makes license scanning a doddle.

I think most modern package managers have something like that: NPM has a license field in package.json, pypi in pkg-info, nuget in the nuspec file, etc.

Github uses the LICENSE file to detect licenses by default IIRC.

Re: Always Review Your Dependencies, AGPL Edition

#128

Post by seemslegit »

lmm wrote: post #98
cygned wrote: post #41

I try to convince my team that the node.js ecosystem has gotten into a stage where it cannot be used for security/financial applications because the sheer amount of dependencies pose an inherent threat. I advocate for Go because of the tendency to less and easier reviewable dependencies. Nobody except me seems to see a problem there, despite me being able to point out specific security incidents. I am wondering if I…

Dependencies are cattle, not pets. There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk. In the JVM ecosystem, projects are only allowed in the Maven Central repository if they have their license documented in a machine readable fashion there; it's then trivial to check what licenses you're depending on (and there are several plugins available for doing so). I'm amaze…

In that case the npm ecosystem has some serious bovine spongiform encephalopathy risks to manage

Re: Always Review Your Dependencies, AGPL Edition

#130

Post by naikrovek »

lmm wrote: post #98
cygned wrote: post #41

I try to convince my team that the node.js ecosystem has gotten into a stage where it cannot be used for security/financial applications because the sheer amount of dependencies pose an inherent threat. I advocate for Go because of the tendency to less and easier reviewable dependencies. Nobody except me seems to see a problem there, despite me being able to point out specific security incidents. I am wondering if I…

Dependencies are cattle, not pets. There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk. In the JVM ecosystem, projects are only allowed in the Maven Central repository if they have their license documented in a machine readable fashion there; it's then trivial to check what licenses you're depending on (and there are several plugins available for doing so). I'm amaze…

> There's nothing wrong with having zillions of them

That is a scary point of view. We have forgotten that there is no such thing as a zero-cost abstraction, apparently, and shovelware developers are now employed by enterprises and write enterprise software...

CPUs aren't getting faster. Software is getting slower a LOT faster than hardware is getting faster, these days, and people are still apparently perfectly fine with adding dependency upon dependency upon abstraction upon abstraction and it's adding up extremely quickly.

A good first step to addressing this is to favor a small copy & paste operation over a small dependency. Lots of people will shriek at the idea of this, but I promise you, all the problems with copying and pasting code are nowhere nearly as severe as all the problems with dependencies. Working to avoid problems you don't have results in creating problems that you definitely do have.

Post reply on HN