Live data from Hacker News

We check our node_modules folder into source control

jackfranklin.co.uk

131–140 of 241 posts

Re: We check our node_modules folder into source control

#131

No, this is what the lock file is for.

Indeed. What's probably needed here is a way to review a diff of the contents of the updated packages. Checking them is is just a brute-force way to do that.

not to mention that unless someone is very familiar with the code of dependencies it's very hard to review hundreds of small near meaningless changes unrelated to your actual functional/business requirements.

something like cargo-crev for npm might be a long term solution

https://github.com/crev-dev/cargo-crev

Re: We check our node_modules folder into source control

#133

People here complaining about the size of the node modules folder - this is one of the reasons people use other solutions like perforce. Checking a 2GB folder into p4 is an absolute no brainer, and for all the flexibility people talk about with git, it's inability to handle this is pretty damning after so many years.

Can you expand on this? Why is the size smaller on p4 compared to git?

Re: We check our node_modules folder into source control

#134
post #87

> We've come up with a rule that helps us here: a change that updates node_modules may not touch any other code in the codebase What? What if the dependency upgrade requires code changes? Master is just broker until the second MR merges? This all sounds like terrible advice.

Dude he addressed that in the next few lines:

> There are times where this doesn't work; updating TypeScript may require us to update some code to fix errors that the new version of TypeScript is now detecting. In that case we have the ability to override the rule. As with anything in software engineering, most "rules" are guidelines, and we're able to side-step them when required.

Re: We check our node_modules folder into source control

#135
post #73

Earlier quoted context omitted.

> Pinned versions don't mean anything when sub-dependencies can have transient versioning resolution occur. That's the purpose of lockfiles-- to pin the entire dependency tree.

Until very recently, the official LTS release of Node shipped with an npm version that would ignore lockfiles during certain situations when running `npm install`. If the package.json listed a fuzzy dependency and the lockfile was pinned to an outdated version, it would just be updated anyway. This was fixed in later versions with the release of the lockfile v2 format, but the fix was never backported to older versio…

Now, that explains some recent oddities I bumped into.

Re: We check our node_modules folder into source control

#136

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

You're drastically overestimating the storage requirements. Time is far, far more valuable than storage savings. The point about CI builds running faster is enough to sell the idea all by itself. Oddly not mentioned in the article is that it allows you to build when npmjs.org is down or unreachable, which happens often enough to be frustrating, and if it happens when you're trying to deal with an emergency, it's down…

We cache the directory on CI and get the same benefits without needing to make our repo massive.

Re: We check our node_modules folder into source control

#138

The listed reasons are insufficient and we could achieve many of these by just pinning our dependencies versions. If we did this, our git repo after a few commits will tend towards a gazillion GBs. Costs outweigh the benefits, if at all there are any. Horrible advice. Don't break the industry practice and check-in your node_modules

You're drastically overestimating the storage requirements. Time is far, far more valuable than storage savings. The point about CI builds running faster is enough to sell the idea all by itself. Oddly not mentioned in the article is that it allows you to build when npmjs.org is down or unreachable, which happens often enough to be frustrating, and if it happens when you're trying to deal with an emergency, it's down…

We run an npm server that proxies request to npmjs.org and caches what it got.

Re: We check our node_modules folder into source control

#139

People here complaining about the size of the node modules folder - this is one of the reasons people use other solutions like perforce. Checking a 2GB folder into p4 is an absolute no brainer, and for all the flexibility people talk about with git, it's inability to handle this is pretty damning after so many years.

Can you expand on this? Why is the size smaller on p4 compared to git?

P4 does not store the entire history locally, so your only limit is the amount of space on the server.

Git will pack objects after a while (initially they’re stored as-is, just compressed), but large files or enormous amounts of changes can make the repository grow to unwieldy amounts, and while git is able to perform “shallow” clones (clones which only store part of the history), not everything handles them well

Re: We check our node_modules folder into source control

#140
post #123

Earlier quoted context omitted.

Ah Perforce... you have my sympathy. Does your team/company store art and asset blobs (even rendered FMVs?) in Perforce too, or do you have a separate asset-management system for that? If you do have two separate systems, how do you keep the asset-store and source-control in-sync? It's been a long time (easily 20 years now) since I dabbled in any high-end creative software (like 3ds, etc), but I remember they general…

The main reason to use perforce in the first place is because of assets. Locking of files too, but that’s only necessary on non-diffable assets.

I've only ever used Perforce through a command-line interface. What's the UX like when using Perforce for blobs/assets? Does everyone have to use the command-line or is there a GUI experience (with fast-rendering thumbnails?)?

Forgive the questions, I'm just curious about the minutia and peculiarities of the gaming-biz because I've never worked in the field.

Post reply on HN