Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

21–30 of 241 posts

Re: We check our node_modules folder into source control

#21

Yarn offers "Plug'n'play" mode since v2, which basically promotes what the author says. It takes the idea further: dependencies are stored as zip archives instead of thousands of small files, which reduces the "git noise" and actually makes this viable as a performant workflow. https://yarnpkg.com/features/pnp

Exactly reading article I for a second felt like author was either not fully aware of tooling, or doesn't care about the noise it will cause (potentially complex conflicts) checking node_modules will cause.

Re: We check our node_modules folder into source control

#22
Whilst I fully sympathise with the points, none of them are convincing nor helpful to me. I'm all for a balanced helping of "just do whatever works", but that's my only takeaway from this article, namely: compromising on best practices can have its place.

Or is there actually a fundamental design problem with lock files in general? My sinking feeling is that it's rather just the Node ecosystem's implementation of them :/

Re: We check our node_modules folder into source control

#24

Yarn offers "Plug'n'play" mode since v2, which basically promotes what the author says. It takes the idea further: dependencies are stored as zip archives instead of thousands of small files, which reduces the "git noise" and actually makes this viable as a performant workflow. https://yarnpkg.com/features/pnp

It's awesome apart from the vast majority don't use yarn.

Re: We check our node_modules folder into source control

#25
post #18

This seems like an unnecessary space hog. If you are that concerned about reproducibility isn't that what npm shrinkwrap tries to solve?

yeah dependency fixing is the way to do it

with NPM shrinkwrap you lock down the version of the package you installed and their deps. That way you can use the same package on all envs. Helps with testing and debugging as you're removing a variable (bad deps, outdated deps, newer deps etc)

Re: We check our node_modules folder into source control

#27
In the past for a large Python project I've handled this using a separate repository for all of the dependencies - that way you can still get work done even if PyPI is unavailable for some reason, but you don't bloat your main repository with an extra few hundred MBs of stuff.

Re: We check our node_modules folder into source control

#28
post #6

> Once you check your node_modules in, there's no need to run an install step before you can get up and running on the codebase Is this really true for packages with pre/post install scripts?

Only if those packages' post-install scripts only mutate their own node_modules directory contents - and I can see this quickly falling-apart as soon as the team's dev-boxes becomes a heterogenous environment (e.g. someone using Ubuntu-on-WSL onboarding an all-M1 Mac team).

Anyway, it's a given that I disagree with the article's specific point (i.e. to commit node_modules to source-control), however I am sympathetic to arguments about avoiding another left_pad incident, but there are better solutions to that then simplistically committing node_modules:

1. package-lock (though this is an incomplete solution: it helps to protects you from vague dependency version numbers (as it uses cryptographic hashes), but it doesn't store a copy of the npm package, and you need to make sure everyone is using the exact same npm/node tooling versions otherwise your package-lock file will be clobbered by different users checking-in wildly different `lockfileVersion` versions.

2. git LFS: every so often (once a month or so?) in a separate directory off-to-the-side, add a heavily compressed 7z LZMA archive of a snapshot of your node_modules directory (ideally in a known-good-state). This allows you to keep a repo-local copy of your important dependencies without it cluttering up your commits. While these would be monthly updates - and your actual package.json/package-lock.json dependencies may change daily or weekly - in the event of catastrophe it won't be too much work to track-down any missing dependencies or to revert the deps back to the last known-good LFS file.

3. Use tools like `offline-npm` and Verdaccio, which are NPM caching proxies. If this was 2019 and everyone was working in a central office then you'd run Verdaccio on a single box in your LAN and have everyone configure their NPM clients to route through that box, which then stores every package ever requested - you could presumably run a cron-job to ensure that package cache is backed-up somewhere safe, maybe even with git-LFS as discussed above.

Re: We check our node_modules folder into source control

#29
post #17

Earlier quoted context omitted.

RTFA. It says, quite succinctly: > I currently work at Google on the Chrome DevTools team and we check our node_modules folder into source control

Yeah I read that and I disagree because at google if I checked in my node_modules I would fail my commit.

The author says they work on Chrome. Chrome doesn't use Google's much-feared monorepo, so maybe they made it work for them?

Can't you ping him internally (don't you guys still use WebEx? lol) to ask him to clarify, and find out how it works for their team?

Re: We check our node_modules folder into source control

#30
post #14

Perhaps you could achieve the best of both worlds by checking node_modules into a git submodule

I don't disagree with the general idea, but in practice, using git submodules always ends up feeling like I'm in the worst of most worlds

Subtrees, instead of submodules, are meant to be super-cool - and I'd love to try them, but I'm still far too wedded to existing git tooling (namely GitKraken) where there's still no support for subtrees.
Post reply on HN