Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

51–60 of 241 posts

Re: We check our node_modules folder into source control

#51

Maven, Gradle and the like always download dependencies as part of the build, and if a server is down (or flaky like jitpack) good lord is it annoying. What I don't understand is why dependency download isn't a separate task you do before compilation. FIRST you download all the dependencies to stabilize that, then you build the code. The reproducibility is the big one. IT drives me nuts that the dependencies are hidd…

> What I don't understand is why dependency download isn't a separate task you do before compilation. FIRST you download all the dependencies to stabilize that, then you build the code.

This is what bazel does. It also offers a `bazel fetch` to pre-download things before going offline (ex: flight).

Re: We check our node_modules folder into source control

#52
post #6

> Once you check your node_modules in, there's no need to run an install step before you can get up and running on the codebase Is this really true for packages with pre/post install scripts?

Only if those packages' post-install scripts only mutate their own node_modules directory contents - and I can see this quickly falling-apart as soon as the team's dev-boxes becomes a heterogenous environment (e.g. someone using Ubuntu-on-WSL onboarding an all-M1 Mac team). Anyway, it's a given that I disagree with the article's specific point (i.e. to commit node_modules to source-control), however I am sympathetic…

> the exact same npm/node tooling versions

I've only ever had issues with differing major versions. That is, sharing lock-files between any node v14.x should work, but expect things to break if you go to v16.

Re: We check our node_modules folder into source control

#53

Earlier quoted context omitted.

use yarn instead, the lockfile is much better.

Problem with yarn is that it doesn't actually use dependencies' lock files... So once you publish your library to npm your lock file doesn't do anything whatsoever.

That makes a lot of sense for file sizes—otherwise, common dependencies patch versions apart would be duplicated many time, and it would block you from upgrading a library's dependency for a security fix. You still get the important part of reproducible builds for your program. Rust's Cargo behaves the same way.

Re: We check our node_modules folder into source control

#54

Yarn offers "Plug'n'play" mode since v2, which basically promotes what the author says. It takes the idea further: dependencies are stored as zip archives instead of thousands of small files, which reduces the "git noise" and actually makes this viable as a performant workflow. https://yarnpkg.com/features/pnp

Unfortunately, reading files from .zip can be considerably slower than the filesystem.

On an Ubuntu x86_64 machine with an SSD, require("react") with Yarn PnP ends up 4x slower than when installed from npm (Node 14.17.6).

Re: We check our node_modules folder into source control

#55
The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any.

Horrible advice. Don't break the industry practice and check-in your node_modules

Re: We check our node_modules folder into source control

#56

Earlier quoted context omitted.

tbf Google is so far ahead that it’s kinda not a real company. Aka you don’t have to adopt every best practice google does since you aren’t that big, mature, secure or rich.

Now we just need to convince all those startups burning cycles trying to get Kubernetes working instead of working on their actual product...

TFW you realize k8s isn't even used that widely inside Google.

Re: We check our node_modules folder into source control

#57
In my experience, it's about the inconsistency between Node.js versions (each version could produce different package-lock.json).

So, for example, you install the dependencies with Node 12, but have to run locally the system with Node 14.

So, i think just checkout the node_modules folder into git is not complete solution. You're avoiding the need to commit inconsistent package-lock.json, which is not hard to solve though.

Re: We check our node_modules folder into source control

#58

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

At my previous company we committed a zipped node modules to git LFS so we could have easy reproducible offline builds without hosting an internal npm instance. Seemed to work well enough.

Re: We check our node_modules folder into source control

#59
Totally agreed with every step, that's what we do too -- except it's composer and composer managed packages here not npm but the reasoning is similar. On top, we too often need to actually patch composer managed packages and rolling patches without the code being version controlled is a PITA. It's git diff --relative if it's already in git otherwise it's .... I dunno, check out the package somewhere else, hope you get a close enough version (because what's released can differ a bit from version control), copy over the files , roll a patch, clean up the patch... what an unnecessary nightmare.

And composer patches makes life quite easy compared to maintaining a fork. If I were to fork something I would need to handle merging every time they have a new release, run the build etc. With composer patch, a new released version is installed and the patch on top. Sure, if there's a conflict that needs to manually resolved but that's usually minimal effort since most patches are absolutely tiny, a few kilobytes at most.

I never even understood the arguments for keeping the packages out of git. Trying to save disk space these days is pointless. Maybe npm is different but composer handles about 160MB of code here. Maybe I missed the memo but these days that's nothing. My laptop shipped with a 500 000MB SSD so it's like, what, half a percent? The speed advantage , on the other hand, is absolutely undeniable, git won on speed in the first place, these script language tools can't possibly compete with a git pull on speed. git diff, as the author notes, is not at all a problem, just separate the vendor commits from your commits. And as I noted: they are useful for vendor packages.

Re: We check our node_modules folder into source control

#60

Does it work across x86, 64, arm, linux, mac and windows? Some modules straight up download binaries so I don’t see being so straightforward.

We can thus infer that author being working in "Google Chrome DevTools team", they must have a way to unifying their dev env.
Post reply on HN