Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

161–170 of 241 posts

Re: We check our node_modules folder into source control

#161

In my experience, it's about the inconsistency between Node.js versions (each version could produce different package-lock.json). So, for example, you install the dependencies with Node 12, but have to run locally the system with Node 14. So, i think just checkout the node_modules folder into git is not complete solution. You're avoiding the need to commit inconsistent package-lock.json, which is not hard to solve th…

Isn't that really a problem with npm and package maintainers? Node is really just a runtime.

Re: We check our node_modules folder into source control

#162
post #9

Earlier quoted context omitted.

"I delete my lock file" is so common that I would say it has no value.

I’ve heard lots of people claim that yarn gives no advantage over npm anymore, as of a year or two ago. But in 5 or so years of using yarn every day, on numerous projects, I’ve probably nuked node modules a couple of times, and never even considered deleting yarn.lock. Maybe yarn is still superior in this regard?

I have the exact same experience. I've tried to use npm again for multiple of times, but the experience has always become less stable overall.

Re: We check our node_modules folder into source control

#163

I currently have a more concrete problem than speed with leaving node_modules out of version control: buggy libraries that, as downloaded, need small corrections. Doing the right thing and fixing libraries for everyone on npm requires significantly more effort and it could be impossible.

patch-package (yet another dependency) does a decent job for that sort of thing

Re: We check our node_modules folder into source control

#166

Disappointed to see so many knee-jerk reactions to this. Vendoring dependencies is a simple way to ensure consistent build inputs, and has the bonus effect of decreasing build times. To respond the two major criticisms: 1) “It takes a lot of space” Don’t be so sure. Text diffs and compresses well. I have a 9-year old Node repo that I’ve been vendoring from the beginning and it’s only grown 200MB over that time. (Gran…

> Vendoring dependencies is a simple way to ensure consistent build inputs If consistent build inputs is your concern may I ask why using lock files wasn’t enough? That’s a problem they were designed to solve.

It says so in the post, in case another left-pad removal happens.

Re: We check our node_modules folder into source control

#167

I understand what he's getting at, but if you are trying to have an exact copy of code in order to guarantee an exact behaviour, then you would want to extend that to the operating system used to run the software. One example was having code where the test suite ran fine on my local MacBook, but would fail on CI (Linux). It turned out that on Linux finding files by name is case sensitive, whereas on Mac OS it isn't,…

Isn’t it where we are going with Nix and co?

Re: We check our node_modules folder into source control

#168
post #165

Does it work across x86, 64, arm, linux, mac and windows? Some modules straight up download binaries so I don’t see being so straightforward.

Would this still be an issue if the whole team uses docker to run the code?

> Would this still be an issue if the whole team uses docker to run the code?

It could be if it's a CPU architecture difference. For example an M1 Mac (ARM64) vs just about every other system (x86-64).

I know we had to switch out MySQL with MariaDB locally because the official MySQL Docker image doesn't support ARM64 devices but MariaDB does. That's just another example where even if you're using Docker there could be differences.

We've also had issues where developers aren't used to case sensitivity at the file system level and things work on their Mac but fail on Linux in CI because Docker's bind mounts (often used in dev) will use file system properties from the host OS which means even if your app runs in Linux within a container it may run differently on a macOS host vs Linux.

The moral of the story here is Docker is good but it isn't a 100% fool proof abstraction that spans across Linux, Windows and macOS on every combination of hardware.

Re: We check our node_modules folder into source control

#169
Upsides from storing node_modules in repo are outweighed by the downsides. Unless of course you're Google-scale and can afford to contribute filesize fixes upstream, write fancy tooling to enforce commit-time workarounds, etc. Nobody working finger-to-feature has time for this.

For your average npm shop which doesn't have infinite internet oil money, here is why the article recommendations won't work for you.

Your CI will pay the time penalty during git clone instead of npm ci. In fact, the node_modules folder will be bigger than your source folder almost immediately. And over time you won't be cloning just the head files you'll also be cloning every npm package binary ever committed. You can't undo this without investing in smarter git tooling. Which is time spent not writing features.

NPM packages which install arch-specific binaries will constantly flip flop from commits by devs on different OS's.

Nobody is safe from left-pad, not even Google, and committing your node_modules folder doesn't change that. Eventually someone is going to have to run npm i.

Running npm ci on everyone's machine is reproduceable, I don't know what OP is warning about. Package lock pins all the versions.

If you have a large enough team to invest in dev experience, there's way better ways to get the advantages of the article without the downsides. You can cache the npm ci result in a container layer for your CI/CD or use middleware like artifactory

Re: We check our node_modules folder into source control

#170

In my experience, it's about the inconsistency between Node.js versions (each version could produce different package-lock.json). So, for example, you install the dependencies with Node 12, but have to run locally the system with Node 14. So, i think just checkout the node_modules folder into git is not complete solution. You're avoiding the need to commit inconsistent package-lock.json, which is not hard to solve th…

Isn't that really a problem with npm and package maintainers? Node is really just a runtime.

When you install nodejs, you have npm installed. So in this case, there's not much you can do though. So it's not just npm issue, it's NodeJS issue, too.
Post reply on HN