Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

Post reply on HN
Print view View original

81–90 of 241 posts

Re: We check our node_modules folder into source control

#81

Post by shric »

mohanmcgeek wrote: post #55

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

I remember an article a few years ago about Google's 86TB google3 source code repo. Someone on HN asked "I know Google is big but how on earth do they have 86TB?". Someone then quipped "someone accidentally checked in node_modules"

Re: We check our node_modules folder into source control

#82

Post by gregplaysguitar »

Bellend wrote: post #9
danlugo92 wrote: post #7

No, this is what the lock file is for.

"I delete my lock file" is so common that I would say it has no value.

I’ve heard lots of people claim that yarn gives no advantage over npm anymore, as of a year or two ago. But in 5 or so years of using yarn every day, on numerous projects, I’ve probably nuked node modules a couple of times, and never even considered deleting yarn.lock. Maybe yarn is still superior in this regard?

Re: We check our node_modules folder into source control

#83

Post by kevin_thibedeau »

abraxas wrote: post #70

This is basically a way of saying: to hell with those "package managers". It's a sentiment that I'm actually in agreement with. I've been coding mostly in Java for the past 22 years. Somewhere around 2010 Maven became the prevalent build tool quickly displacing the venerable Ant. With Ant we had builds that used checked in jar file dependencies. It was obvious what your builds consisted of and they were very fast onc…

Ant has shite syntax that "clever" coworkwers can turn into a morass.

Re: We check our node_modules folder into source control

#84

Post by klodolph »

mohanmcgeek wrote: post #55

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

I’d say that the main link in that reasoning is that “Git can’t handle this without making the repo a gazillion GBs” which, can of course, can be solved if you weren’t using Git in the first place. Certain other SCMs, like Perforce, allow you to trim history and don’t require you to clone the whole history in the first place.

Re: We check our node_modules folder into source control

#85

Post by deanc »

revskill wrote: post #57

In my experience, it's about the inconsistency between Node.js versions (each version could produce different package-lock.json). So, for example, you install the dependencies with Node 12, but have to run locally the system with Node 14. So, i think just checkout the node_modules folder into git is not complete solution. You're avoiding the need to commit inconsistent package-lock.json, which is not hard to solve th…

Nobody should be running different versions in a team. Unless you’re selling a product where this is a likely scenario you should be running the same environment either through some kind of virtualization or at worst nvm.

Re: We check our node_modules folder into source control

#86

Post by jakear »

So instead of downloading the current set of dependencies after cloning, you download every dependency ever used while cloning? And you do this to "save bandwidth"? Doesn't make much sense. (Yes I know about shallow clones, but it's often nice to have the full history around)

Any reasonable CI tool will have a way to cache generated assets based on file contents, that's the way to go here IMO.

Re: We check our node_modules folder into source control

#87

Post by ng12 »

> We've come up with a rule that helps us here: a change that updates node_modules may not touch any other code in the codebase

What? What if the dependency upgrade requires code changes? Master is just broker until the second MR merges?

This all sounds like terrible advice.

Re: We check our node_modules folder into source control

#88

Post by rarkins »

ilumanty wrote: post #13

Yarn offers "Plug'n'play" mode since v2, which basically promotes what the author says. It takes the idea further: dependencies are stored as zip archives instead of thousands of small files, which reduces the "git noise" and actually makes this viable as a performant workflow. https://yarnpkg.com/features/pnp

Yarn’s offline support is not directly tied to PNP. Yarn v1’s support for “offline” installs was a day one requirement, and as I understand it one of the primary drivers for Facebook engineers (at the time) to drive the creation of Yarn.

If offline installs is what you want, I don’t see any advantage of node_modules compared to this feature - only disadvantages (size, noise, and cross-platform incompatibilities).

Re: We check our node_modules folder into source control

#89

Post by batoure »

shric wrote: post #81
mohanmcgeek wrote: post #55

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

I remember an article a few years ago about Google's 86TB google3 source code repo. Someone on HN asked "I know Google is big but how on earth do they have 86TB?". Someone then quipped "someone accidentally checked in node_modules"

Basically came here to say this. The whole article felt very silly until I was like “oh wait OP stated early on he works at google… yeah seems about right ~closes tab~”

Re: We check our node_modules folder into source control

#90

Post by danShumway »

sedro wrote: post #73
andrewmcwatters wrote: post #66

Earlier quoted context omitted.

Reread what you just wrote for a moment and reflect on that. Also, you clearly have not written software using Node.js on a long enough time horizon. Pinned versions don't mean anything when sub-dependencies can have transient versioning resolution occur. The reality is that unless you can fully byte-for-byte assure what you have deployed today is what you can retrieve from an old tag, let's say weeks, months, or yea…

> Pinned versions don't mean anything when sub-dependencies can have transient versioning resolution occur. That's the purpose of lockfiles-- to pin the entire dependency tree.

Until very recently, the official LTS release of Node shipped with an npm version that would ignore lockfiles during certain situations when running `npm install`.

If the package.json listed a fuzzy dependency and the lockfile was pinned to an outdated version, it would just be updated anyway. This was fixed in later versions with the release of the lockfile v2 format, but the fix was never backported to older versions of npm, even though those versions of npm were the recommended, default versions that shipped with LTS Node installs if you went to the main website or installed from a software repo.

I think that for a non-trivial number of people, they may not have a lot of trust for lockfiles because they tried using them and they just straight-up didn't work.

Post reply on HN