Always Review Your Dependencies, AGPL Edition • Hacker News Board

Re: Always Review Your Dependencies, AGPL Edition

#181

Post by jen20 » Mon, Jan 06, 2020, 5:06 PM UTC

danShumway wrote: post #157

cygned wrote: post #41

I try to convince my team that the node.js ecosystem has gotten into a stage where it cannot be used for security/financial applications because the sheer amount of dependencies pose an inherent threat. I advocate for Go because of the tendency to less and easier reviewable dependencies. Nobody except me seems to see a problem there, despite me being able to point out specific security incidents. I am wondering if I…

I don't think the NPM ecosystem is in a great security position, but the pushback I'd make here is that you should be reviewing your Go dependencies as well. If you're looking at the NPM ecosystem and saying, "the number of dependencies is problematic because it takes a long time to review them", I agree with you. If you're looking at the Go ecosystem and saying, "there are fewer dependencies, so I don't need to revi…

I read the GP post as saying that having fewer dependencies makes reviewing them easier, not that one need not review them?

Re: Always Review Your Dependencies, AGPL Edition

#182

Post by vemv » Mon, Jan 06, 2020, 5:13 PM UTC

> It started off poorly when I noticed some Brown M&M's: despite being a library, it was logging messages to stdout

In many language ecosystems, logging to stdout tends to be the right thing to do in a lib as it does not impose any specific choice of logging implementation.

One always can recapture stdout and route it through one's logging solution of choice; doing something analog with an extraneous logging library can be harder if not unfeasible.

Are things different in Go land?

Re: Always Review Your Dependencies, AGPL Edition

#183

Post by danShumway » Mon, Jan 06, 2020, 5:16 PM UTC

jen20 wrote: post #181

danShumway wrote: post #157

Earlier quoted context omitted.

I don't think the NPM ecosystem is in a great security position, but the pushback I'd make here is that you should be reviewing your Go dependencies as well. If you're looking at the NPM ecosystem and saying, "the number of dependencies is problematic because it takes a long time to review them", I agree with you. If you're looking at the Go ecosystem and saying, "there are fewer dependencies, so I don't need to revi…

I read the GP post as saying that having fewer dependencies makes reviewing them easier, not that one need not review them?

My point being that "NPM is bad for security" is probably the wrong take to convince a software team to avoid the ecosystem.

Instead, phrase it as, "well obviously we need to review our dependencies, so why use an ecosystem where that's hard?" Don't phrase it as a security problem, phrase it as a development time and overhead problem.

Or better yet, let your team use NPM as long as they review the dependencies, and if your dependency tree is really a problem, your team will naturally become annoyed by the extra work and will avoid large dependencies on their own without a lot of extra prompting.

Re: Always Review Your Dependencies, AGPL Edition

#184

Post by brobinson » Mon, Jan 06, 2020, 5:20 PM UTC

herpderperator wrote: post #16

When this webpage loads, it briefly flashes in plaintext before loading CSS - at least in Chrome. This is unusual since it has the CSS stylesheet in the head which I thought would block until it's fully loaded. Does anyone know what's going on with that?

This is the classic FOUC that has been around for like 20+ years: https://en.wikipedia.org/wiki/Flash_of_unstyled_content

Re: Always Review Your Dependencies, AGPL Edition

#185

Post by dwheeler » Mon, Jan 06, 2020, 5:22 PM UTC

lmm wrote: post #98

cygned wrote: post #41

I try to convince my team that the node.js ecosystem has gotten into a stage where it cannot be used for security/financial applications because the sheer amount of dependencies pose an inherent threat. I advocate for Go because of the tendency to less and easier reviewable dependencies. Nobody except me seems to see a problem there, despite me being able to point out specific security incidents. I am wondering if I…

Dependencies are cattle, not pets. There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk. In the JVM ecosystem, projects are only allowed in the Maven Central repository if they have their license documented in a machine readable fashion there; it's then trivial to check what licenses you're depending on (and there are several plugins available for doing so). I'm amaze…

> Dependencies are cattle, not pets.

That doesn't make any sense. That analogy works for mostly-identical computers, where if your software won't run on one computer you can just use another mostly-identical computer.

Almost by definition dependencies are not interchangeable. You can't replace a routine to pad strings with a routine that is a web server. Or a matrix multiplier. Even dependencies that do the same overall job almost never have the same API. Heck, even the same dependency often ends up with a different incompatible API over time as versions change.

> There's nothing wrong with having zillions of them; what you need is good tools to manage them in bulk.

Every added dependency is a risk. Each unintentional vulnerability in each dependency increases the number of vulnerabilities that might be exploitable in your system. And that's just the unintentional vulnerabilities.

Practically all systems provide no useful sandboxing between dependencies, so if any one of your transitive dependencies is malicious, then your entire system is malicious.

Every new dependency also brings in potential license issues, per this article. I think it's unacceptable to have a scarefest about the AGPL, GPL, or LGPL; for a vast number of applications those licenses are just fine. The bigger risks are software that has no license at all, which are a legal risk for any project that uses them until governments change international treaties involving copyright (which is not likely any time soon). But it's certainly true that various licenses are not acceptable for certain situations, and every new dependency increases the risks of licensing problems.

Having no dependencies is absurd; it's uneconomic to build everything from scratch. But every time you add a dependency you need to think about the trade-off; it is sometimes wise to not reuse something.

Re: Always Review Your Dependencies, AGPL Edition

#186

Post by damnyou » Mon, Jan 06, 2020, 5:23 PM UTC

vmchale wrote: post #172

galangalalgol wrote: post #156

Earlier quoted context omitted.

Indeed, rust is a likely wasm choice and it is working on an npm nightmare of its own with crates.io. A blessed crates pack is sorely needed to fill out the intentionally thin stdlib in cases you can't afford a dependency tree 20 layers deep for all the reasons mentione on this thread.

It's unfortunate that Rust is so tied to cargo, and unfortunate that cargo insists on allowing multiple versions of a dependency. If you point out that cargo is poorly designed everyone just yells and you and says you don't understand.

Only incompatible versions though — Rust dedups dependencies up to semver compatibility.

I think that's the right balance. clap 3 is and should be treated as a totally different package from clap 2.

Re: Always Review Your Dependencies, AGPL Edition

#187

Post by nieksand » Mon, Jan 06, 2020, 5:29 PM UTC

vmchale wrote: post #172

galangalalgol wrote: post #156

Earlier quoted context omitted.

Indeed, rust is a likely wasm choice and it is working on an npm nightmare of its own with crates.io. A blessed crates pack is sorely needed to fill out the intentionally thin stdlib in cases you can't afford a dependency tree 20 layers deep for all the reasons mentione on this thread.

It's unfortunate that Rust is so tied to cargo, and unfortunate that cargo insists on allowing multiple versions of a dependency. If you point out that cargo is poorly designed everyone just yells and you and says you don't understand.

Can you elaborate on the design flaws that you perceive in Cargo?

Re: Always Review Your Dependencies, AGPL Edition

#188

Post by ones_and_zeros » Mon, Jan 06, 2020, 5:51 PM UTC

Johnny come lately but is that true of the AGPL, that merely linking to agpl code infects your code? No modification to the agpl code needed to trigger the clause?

What is the point of having a license like that? Trolling?

Re: Always Review Your Dependencies, AGPL Edition

#189

Post by robmccoll » Mon, Jan 06, 2020, 5:55 PM UTC

vemv wrote: post #182

> It started off poorly when I noticed some Brown M&M's: despite being a library, it was logging messages to stdout In many language ecosystems, logging to stdout tends to be the right thing to do in a lib as it does not impose any specific choice of logging implementation. One always can recapture stdout and route it through one's logging solution of choice; doing something analog with an extraneous logging library…

In what language or environment would recapturing and routing stdout be a desirable solution? Seeing that without a really good explanation would concern me. That's a global hack with fun side effects that can lead to very confusing bugs. Also, you would lose which library is which log if you have to do this for more than one library.

In Go, I would assume that a library that needs to log would do one or more of the following a) offer the ability to disable logging b) offer overriding the output that it uses for logging probably with an io.Writer interface or something c) accept a log.Logger or offer an interface that something like log.Logger meets. Ideally it would offer an interface that supports some key-value meta data to be able to have nicely structured logs with formatting independent of the library.

Re: Always Review Your Dependencies, AGPL Edition

#190

Post by robmccoll » Mon, Jan 06, 2020, 6:00 PM UTC

On the licensing front, one thing that has really helped me is adding a script to my build / CI process that scans every package I'm dependent on for a LICENSE / COPYING / etc. file and then runs through a set of dumb fingerprints to match on different licenses for a combined whitelist / blacklist approach. Packages with no recognized license file or license are rejected. Then it spits the licences out into a map[string]string in a file where the Go package maps to the license string which gets served up on its own HTTPS endpoint to keep in compliance with copyright notice requirements.

In case it's useful to anyone else, here's an older copy of it: https://gist.github.com/robmccoll/240317eceb73e3f4e29ea662e3...