Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

Post reply on HN
Print view View original

191–200 of 241 posts

Re: We check our node_modules folder into source control

#191

Post by anthomtb »

jdlshore wrote: post #112

Disappointed to see so many knee-jerk reactions to this. Vendoring dependencies is a simple way to ensure consistent build inputs, and has the bonus effect of decreasing build times. To respond the two major criticisms: 1) “It takes a lot of space” Don’t be so sure. Text diffs and compresses well. I have a 9-year old Node repo that I’ve been vendoring from the beginning and it’s only grown 200MB over that time. (Gran…

Dumb question: what does it mean to “vendor” your dependencies?

Best guess is something like “ship required source or binaries along with your end product.” Like static linking but extended to dynamic languages and source control.

Re: We check our node_modules folder into source control

#192

Post by somehnacct3757 »

mustyoshi wrote: post #171
somehnacct3757 wrote: post #169

Upsides from storing node_modules in repo are outweighed by the downsides. Unless of course you're Google-scale and can afford to contribute filesize fixes upstream, write fancy tooling to enforce commit-time workarounds, etc. Nobody working finger-to-feature has time for this. For your average npm shop which doesn't have infinite internet oil money, here is why the article recommendations won't work for you. Your CI…

Isn't --depth=1 all you need to avoid downloading every binary?

Maybe, but everyone's CI situation is so variable that it may not be that easy. For instance if you are using a monorepo then even a shallow clone can be overkill. And if you rely on git history for conditional CI then a shallow clone will ruin the output of many git commands. So you could end up in an either/or optimization situation depending on the order your CI/CD organically grew and other architectural decisions you made.

Re: We check our node_modules folder into source control

#193

Post by oftenwrong »

anthomtb wrote: post #191
jdlshore wrote: post #112

Disappointed to see so many knee-jerk reactions to this. Vendoring dependencies is a simple way to ensure consistent build inputs, and has the bonus effect of decreasing build times. To respond the two major criticisms: 1) “It takes a lot of space” Don’t be so sure. Text diffs and compresses well. I have a 9-year old Node repo that I’ve been vendoring from the beginning and it’s only grown 200MB over that time. (Gran…

Dumb question: what does it mean to “vendor” your dependencies? Best guess is something like “ship required source or binaries along with your end product.” Like static linking but extended to dynamic languages and source control.

To "vendor" a dependency is to check it into your project's source control, whether as source or binary.

Re: We check our node_modules folder into source control

#194

Post by jwlake »

jdlshore wrote: post #112

Disappointed to see so many knee-jerk reactions to this. Vendoring dependencies is a simple way to ensure consistent build inputs, and has the bonus effect of decreasing build times. To respond the two major criticisms: 1) “It takes a lot of space” Don’t be so sure. Text diffs and compresses well. I have a 9-year old Node repo that I’ve been vendoring from the beginning and it’s only grown 200MB over that time. (Gran…

There are only 2 problems I see with the existing solution in npm.

- "npm add package" puts in a "^ver", which is bad practice

- there is no good infrastructure to pull hash based blobs out of the ether in case npmjs is offline

npm-shrinkwrap has solved repeatability forever, people just didn't always use it. Auto-upgrading dependancies is the big problem, which should have never existed because it is not principled. I'd go further and say that dependancies and devDependances should only support exact versions, and peerDepenancies are the only thing that supports non-exact versions.

Re: We check our node_modules folder into source control

#195

Post by oftenwrong »

smarx007 wrote: post #172
abraxas wrote: post #70

This is basically a way of saying: to hell with those "package managers". It's a sentiment that I'm actually in agreement with. I've been coding mostly in Java for the past 22 years. Somewhere around 2010 Maven became the prevalent build tool quickly displacing the venerable Ant. With Ant we had builds that used checked in jar file dependencies. It was obvious what your builds consisted of and they were very fast onc…

It’s been a long while since I saw a POM file with unpinned dependencies… ca 12 years of Java experience. Had to deal with an Ant project recently that didn’t check in the JARs into Git, I thought my hair would become gray by the time I figure out how to pull the right dependency versions transitively to get the Ant build to work. But I share the overall sentiment. What if Maven Central goes down one day? It’s just a…

>What if Maven Central goes down one day?

Most Java shops run their own artefact repository acting as a pull-through repository to Maven Central and other 3rd party repositories. In addition to acting as insurance against losing access to critical dependencies, it also can provide a performance boost when downloading dependencies, and helps to offload your organisation's traffic from Maven Central.

Re: We check our node_modules folder into source control

#196

Post by ChrisMarshallNY »

This is fairly standard "conservative configuration management" stuff.

The company I worked for, used to do that with everything. In fact, one of the ways that they archived versions, was to create a bootable external hard disk clone, of the entire development machine, and store that.

If you want to be absolutely sure that you have the complete building blocks, then you don't trust your package manager. Make local dupes of the packages, and integrate them into your own version control.

Re: We check our node_modules folder into source control

#197

Post by marcosdumay »

oftenwrong wrote: post #193
anthomtb wrote: post #191

Earlier quoted context omitted.

Dumb question: what does it mean to “vendor” your dependencies? Best guess is something like “ship required source or binaries along with your end product.” Like static linking but extended to dynamic languages and source control.

To "vendor" a dependency is to check it into your project's source control, whether as source or binary.

It's to have it come from some place you control. It can be your source control, but a very common way to vendor dependencies on other languages is to just save them in a server somewhere and install pull them from that on your code.

Re: We check our node_modules folder into source control

#199

Post by oftenwrong »

abraxas wrote: post #70

This is basically a way of saying: to hell with those "package managers". It's a sentiment that I'm actually in agreement with. I've been coding mostly in Java for the past 22 years. Somewhere around 2010 Maven became the prevalent build tool quickly displacing the venerable Ant. With Ant we had builds that used checked in jar file dependencies. It was obvious what your builds consisted of and they were very fast onc…

You can save yourself some headaches by not using transitive dependencies in your Maven builds. This can be enforced with Maven Enforcer. The trade-off is that you will have to clean unused dependencies. This can be tool-assisted with something like: https://github.com/castor-software/depclean/ . You can also enforce dependency version convergence with Maven Enforcer. None of this saves you from "jar hell" directly, but it helps prevent you from unknowingly creating a disaster-in-waiting.

Re: We check our node_modules folder into source control

#200

Post by throwaway984393 »

WA wrote: post #130
throwaway984393 wrote: post #94

Why are people still writing server-side code in Javascript? It was a cute idea a decade ago. But now you have to twist yourself into knots to prevent a ton of problems. Nobody will use C today because "memory safety", but everybody will use Node.js despite the dependencies being both a security and usability nightmare. Imagine if I suggested using Bashscript for a web application backend.

What do you use for server-side code?

Visual Basic
Post reply on HN