Doesn't everyone? That's one of the annoying parts of using other people's code. You have no idea how good or bad it is until you have thoroughly vetted it.
Ask HN: Does anyone investigate open source packages before using in prod?
11–20 of 40 posts
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#12This article might be of interest:
https://medium.com/hackernoon/im-harvesting-credit-card-numb...
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#13I am paranoid about security of all those packages, so yes, even before just downloading, I check the authors, activity and read the source code. Not always – e.g. I skip the source code if it's something big AND very reputable AND I decided that I need it such as scala/scala or facebook/react – but I do my best. It's very annoying, it's not free, and it affects what kinds of libraries I use. My projects have fewer a…
React itself is big & reputable, but the dependency tree is massive and I doubt that it's getting fully vetted on an ongoing basis. Even if you vet it today, any given dependency can be updated to something else tomorrow.
There are definitely some things in React's dependency tree that are a bit questionable if you are sensitive enough to any given problem, beyond just security. For example, packages where the license being used is contradictory between the package.json vs the LICENSE file or the full license terms are not expressed within these but are clarified in the README.md.
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#14Yes, in depth. Not just the packages but their dependencies as well.
On what cadence do you do this? Every release?
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#15It's interesting to see so many people here checking the code of all their open source packages, so here's my take on it as a security consultant:
No, most people don't, they even have a hard time keeping library versions up to date.
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#16It's interesting to see so many people here checking the code of all their open source packages, so here's my take on it as a security consultant: No, most people don't, they even have a hard time keeping library versions up to date.
Do you have a pragmatic approach that you typically recommend, considering that (without version pinning thousands of packages) anything could change from one day to the next? (even if a package was "good" today, it could turn "bad" tomorrow)
The best I've been able to come up with is to pick things that have minimal dependencies of their own. It doesn't eliminate the threat, but it does at least reduce it.
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#17Yes, in depth. Not just the packages but their dependencies as well.
Really? What about all the dependencies from those dependencies?... For example, our company is working on an app that has 82 npm dependencies and over 17,000 resolved npm packages... It's absolutely ridiculous to investigate all of them... but it's also necessary if you want to be sure...
> Really? What about all the dependencies from those dependencies?...
Yep, all the way to the end.
I got the idea from a book called "Hollywood Secrets of Project Management Success" by James R. Persse. It's two books interleaved really, one is just a standard pitch for Agile methods (IMO), but the other is a presentation of the process that large film studios use to make movies. The movie industry is ~100 years old and mostly very good at bringing in projects on time and under budget.
Somewhere in there he talks about how they'll track their dependencies in a kind of "portfolio", I forget the details, but it translates in IT to a "dependency portfolio" and you would (if you're large enough) have an actual "Deps Dept." and a Deps Manager whose sole job is tracking dependencies and their updates and patches, etc.
> working on an app that has 82 npm dependencies
Ach! Well, see, there's your problem right there. :-)
Seriously though, one of the benefits of a dependency portfolio is to help you know when your system has gotten out of hand. The problems are still there even if you don't look at them, eh?
> It's absolutely ridiculous to investigate all of them... but it's also necessary if you want to be sure...
Ya feel. ;-)
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#18Yes, in depth. Not just the packages but their dependencies as well.
On what cadence do you do this? Every release?
The "deps portfolio" gets updated whenever the deps change. In practice the flow goes like this:
0. A dev wants to use a new dependency, likely after experimenting with it a little bit.
1. Preliminary evaluation, which includes a transitive dependency scan. ("Too many dependencies" is a valid fail condition all on it's own.)
2. If everything looks good we bring it and it's deps into our internal repo. This includes the plumbing to add it to our dev|test|production envs. (Using Docker or whatever.)
3. Now the devs can use it in code destined for prod. There's a nice page in the company wiki that lists the exact version(s) with links to the docs, bug trackers, mailing lists, etc. and also the internal company lore for that package.
It's tight.
- - - -
This might seem like a lot of work up front, but think about all the work it saves down the line.
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#19Earlier quoted context omitted.
On what cadence do you do this? Every release?
The "deps portfolio" gets updated whenever the deps change. In practice the flow goes like this: 0. A dev wants to use a new dependency, likely after experimenting with it a little bit. 1. Preliminary evaluation, which includes a transitive dependency scan. ("Too many dependencies" is a valid fail condition all on it's own.) 2. If everything looks good we bring it and it's deps into our internal repo. This includes t…
When one of the transitive dependencies fixes a security issue, is it then re-evaluated prior to being updated in the internal repo?
I'm guessing you work at a pretty large tech company. It seems wasteful that so many companies might be replicating this work. I wonder if there might be the opportunity for a body to review & approve packages on behalf of many companies. Perhaps npm will eventually move in this direction.
Re: Ask HN: Does anyone investigate open source packages before using in prod?
#20It's interesting to see so many people here checking the code of all their open source packages, so here's my take on it as a security consultant: No, most people don't, they even have a hard time keeping library versions up to date.
Do you have a pragmatic approach that you typically recommend, considering that (without version pinning thousands of packages) anything could change from one day to the next? (even if a package was "good" today, it could turn "bad" tomorrow) The best I've been able to come up with is to pick things that have minimal dependencies of their own. It doesn't eliminate the threat, but it does at least reduce it.
IIRC there is some kind of SaaS solution for this but right now I can’t recall its name. Maybe someone else here knows?
Edited: I reached out to some security people and it seems like the following are popular tools for this use case: Snyk / Dependabot / Whitesource.