Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

121–130 of 241 posts

Re: We check our node_modules folder into source control

#121
post #96

Earlier quoted context omitted.

Yes, but a lot of people prefer to have every commit in master in a buildable state. Otherwise things like git bisect become a lot more tedious.

> have every commit in master Yes, but a PR commit is not in master.

I see you, little squasher :)

Re: We check our node_modules folder into source control

#122
post #116

Earlier quoted context omitted.

I’d say that the main link in that reasoning is that “Git can’t handle this without making the repo a gazillion GBs” which, can of course, can be solved if you weren’t using Git in the first place. Certain other SCMs, like Perforce, allow you to trim history and don’t require you to clone the whole history in the first place.

With git you can specify the clone depth and only get the latest X versions. And there are ways to trim history with external plugins ( git filter repo).

You have to remake every commit of the repository which basically means you have a new repository, and new commits. In order to do the latter thing you said, you cannot have done the former thing you said.

Re: We check our node_modules folder into source control

#123
post #43

Earlier quoted context omitted.

This file seems to be controlling what gets ("checked-in") - https://source.chromium.org/chromium/chromium/src/+/main:thi... In our industry (games) we often do that - checkin prebuilt code in the depots (typically "p4"). I'm not saying it's wrong/right, it's just what we do (not 100% fully, but almost, though people in IT/infra tend to do otherwise).

Ah Perforce... you have my sympathy. Does your team/company store art and asset blobs (even rendered FMVs?) in Perforce too, or do you have a separate asset-management system for that? If you do have two separate systems, how do you keep the asset-store and source-control in-sync? It's been a long time (easily 20 years now) since I dabbled in any high-end creative software (like 3ds, etc), but I remember they general…

The main reason to use perforce in the first place is because of assets.

Locking of files too, but that’s only necessary on non-diffable assets.

Re: We check our node_modules folder into source control

#124
post #115

Earlier quoted context omitted.

Whether he's right or wrong can be debated, but reducing the argument to "follow industry practice" is a perfect example of cargo culting. Industry practice needs to be based on something today, not something from ten years ago that might or might not be valid anymore. He offers a long list of arguments and even though I don't check in node_modules, some of the arguments are compelling - e.g. we already want reproduc…

> He offers a long list of arguments and even though I don't check in node_modules, some of the arguments are compelling (...) Are they, really? The less debatable point is arguing that making CICD pipelines slightly faster, but this feels like an appeal to microoptimization. Any free tier CICD system out there let's you do a single npm install and move these dependencies as far as you'd like into the pipeline as art…

The fact that there’s no mention of the `npm ci` command (or yarn or pnpm) makes me wonder how deeply this problem was investigated before using it as a justification for this hacky workaround.

One wonders why they aren’t checking in the binaries for their database and language runtimes. Surely this would save crucial seconds in project setup.

Re: We check our node_modules folder into source control

#125

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

You're drastically overestimating the storage requirements. Time is far, far more valuable than storage savings. The point about CI builds running faster is enough to sell the idea all by itself.

Oddly not mentioned in the article is that it allows you to build when npmjs.org is down or unreachable, which happens often enough to be frustrating, and if it happens when you're trying to deal with an emergency, it's downright infuriating.

Re: We check our node_modules folder into source control

#126
Suggestion how to manage node_modules: When testing different modules use npm/yarn, but once you have decided which module to use, fork the module on Github (most node_modules are on Github) then link directly to your fork in package.json like this: https://github.com/{user}/{module}/tarball/master Now you can use git/Github instead of npm to manage package updates.

Re: We check our node_modules folder into source control

#127
post #115

Earlier quoted context omitted.

Whether he's right or wrong can be debated, but reducing the argument to "follow industry practice" is a perfect example of cargo culting. Industry practice needs to be based on something today, not something from ten years ago that might or might not be valid anymore. He offers a long list of arguments and even though I don't check in node_modules, some of the arguments are compelling - e.g. we already want reproduc…

> He offers a long list of arguments and even though I don't check in node_modules, some of the arguments are compelling (...) Are they, really? The less debatable point is arguing that making CICD pipelines slightly faster, but this feels like an appeal to microoptimization. Any free tier CICD system out there let's you do a single npm install and move these dependencies as far as you'd like into the pipeline as art…

Here's a better idea, can't you have an npm cache/clone that keeps all the artifacts you use in your code? So you pull from it, it pulls from npm and caches?

Re: We check our node_modules folder into source control

#128
People here complaining about the size of the node modules folder - this is one of the reasons people use other solutions like perforce. Checking a 2GB folder into p4 is an absolute no brainer, and for all the flexibility people talk about with git, it's inability to handle this is pretty damning after so many years.

Re: We check our node_modules folder into source control

#129

No, this is what the lock file is for.

Note the other child comments of this; NPM has failed to make the lockfile reliable across systems and versions. Here's an example from March that isn't fixed: https://github.com/npm/cli/issues/2846 The fact that you have so many people believing "nuke node_modules and delete package-lock.json" is a reasonable step in diagnosing an error is damning to NPM. We don't check in our node_modules, but "use the lockfile" is…

npm went through a rough few years (lockfiles, leftpad) and obviously the hivemind of the JS ecosystem is not the most careful one (hence all the advice of nuke it and npm i).

but those who care use yarn, those who even want to be correct use yarn2, and so on.

Re: We check our node_modules folder into source control

#130

Why are people still writing server-side code in Javascript? It was a cute idea a decade ago. But now you have to twist yourself into knots to prevent a ton of problems. Nobody will use C today because "memory safety", but everybody will use Node.js despite the dependencies being both a security and usability nightmare. Imagine if I suggested using Bashscript for a web application backend.

What do you use for server-side code?
Post reply on HN