Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

91–100 of 241 posts

Re: We check our node_modules folder into source control

#91
post #87

> We've come up with a rule that helps us here: a change that updates node_modules may not touch any other code in the codebase What? What if the dependency upgrade requires code changes? Master is just broker until the second MR merges? This all sounds like terrible advice.

You can have multiple commits in a PR

Re: We check our node_modules folder into source control

#93

Yarn offers "Plug'n'play" mode since v2, which basically promotes what the author says. It takes the idea further: dependencies are stored as zip archives instead of thousands of small files, which reduces the "git noise" and actually makes this viable as a performant workflow. https://yarnpkg.com/features/pnp

Also if you use the yarn deployment plugin (forgot the name), it will create a deployment folder for you that does not contain dev dependencies. This way we found a package that was installed as a dev dependency but in reality was a regular runtime dependency.

Re: We check our node_modules folder into source control

#94
Why are people still writing server-side code in Javascript? It was a cute idea a decade ago. But now you have to twist yourself into knots to prevent a ton of problems. Nobody will use C today because "memory safety", but everybody will use Node.js despite the dependencies being both a security and usability nightmare. Imagine if I suggested using Bashscript for a web application backend.

Re: We check our node_modules folder into source control

#95

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

Whether he's right or wrong can be debated, but reducing the argument to "follow industry practice" is a perfect example of cargo culting. Industry practice needs to be based on something today, not something from ten years ago that might or might not be valid anymore. He offers a long list of arguments and even though I don't check in node_modules, some of the arguments are compelling - e.g. we already want reproducible builds and use package-lock, but why not skip this step altogether? Why not skip setting up cache on your CI if you don't need to? What if knowing the details of your package manager and CI is useless because there are simpler ways of doing things. I'm not that convinced that a few commits would create a gazillion GB repo, so his arguments seem stronger.

But "not best practice", qed.

Re: We check our node_modules folder into source control

#96
post #91
post #87

> We've come up with a rule that helps us here: a change that updates node_modules may not touch any other code in the codebase What? What if the dependency upgrade requires code changes? Master is just broker until the second MR merges? This all sounds like terrible advice.

You can have multiple commits in a PR

Yes, but a lot of people prefer to have every commit in master in a buildable state. Otherwise things like git bisect become a lot more tedious.

Re: We check our node_modules folder into source control

#97
post #89
post #81

Earlier quoted context omitted.

I remember an article a few years ago about Google's 86TB google3 source code repo. Someone on HN asked "I know Google is big but how on earth do they have 86TB?". Someone then quipped "someone accidentally checked in node_modules"

Basically came here to say this. The whole article felt very silly until I was like “oh wait OP stated early on he works at google… yeah seems about right ~closes tab~”

Insomnia lead me to go dig into the repo… the average age of 98% of files in node_modules is 10 months old, attached to the commit when most of these files were added to the repo in the first place… so the entire argument is predicated on the changes to 2% of the dependencies

Re: We check our node_modules folder into source control

#98
post #75

Earlier quoted context omitted.

Running a proxy for dependencies, whether it be NPM or maven, is pretty common. This way you can build if those services go down. Also, performance.

Just another service that needs to be maintained but isn’t on the books as something that needs to be maintained leading to a wonderful day a couple of years in the future when a license cull of abandoned VMs means all your code stops building successfully on the same day.

I mean, yeah, gotta maintain it. If it breaks one day every 5 years but speeds up build times 2x every day until then, worth it.

Re: We check our node_modules folder into source control

#99

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

This is a fun shift in culture because for a long time checking in node_modules was the official advice of the early Node documentation unless you were building a library for npm.

It's already been long enough since that time that people seem to have forgotten that it even existed. Vendoring dependencies is one of those things where every once and a while a language will reintroduce the concept, and it always seems to catch people off guard. Go is a good example, although it seems to have varying advice about whether vendored dependencies should be checked in to version control. That might not be surprising considering that Go is also coming out of Google, just like this article.

It feels a little bit weird to say that this is just "industry practice" when you have the Chrome DevTools team telling you they don't do it, but :shrug:. Google does tend to be a bit of a rarity in how it treats monorepos. I'm just always interested to see how opinions on this have evolved; it's rare for me to see analysis that says, "we used to do this, and here's why we found out that it didn't work." Usually the opinions end up seeming more universalist, like the very idea of vendoring dependencies is somehow weird and unexpected, and not something that the industry was largely on board with for a decent amount of time.

Re: We check our node_modules folder into source control

#100
This is all horrible advice and other commenters have rightfully pointed this out already so I won't repeat it more… (okay, once more: This is all horrible advice, don't do that)

But, there is one thing I like from this, which is git diffs showing the actual final code diff when you upgrade dependencies.

Of course, this being horrible advice, it ignores how many JS packages ship minified which would make the diff as useful as binary noise. But I like it in theory. This could be a good opportunity to write a tool that replicates this specifically (and what's more, for other languages as well)

Post reply on HN