21–30 of 52 posts
Why not use yarn? It has a much more reliable solution: https://yarnpkg.com/features/pnp
Useless files will still be there.
Also, when you create a Docker image, you avoid packing in dev tools that aren't absolutely essential (such as Yarn).
This is wildly unsafe. - Some packages contain non-JS files for good reasons, and they may break in subtle unpredictable ways when you mess with the contents of their package. - Node.js will happily run JavaScript files even if they're not "*.js": A file like "hello.alsdfhlshdfl" works just fine as long as its content parses. There is no guarantee that your dependencies (and their recursive dependencies) don't static…
- Of course, this entails the risk of occasional breakage. But for 99% of modules, this has no impact at runtime. - The patterns used to find files are specific enough to target only those files that are well known to be useless at runtime. - The license texts of these libraries can be copied and merged into a main LICENSE file. - Have you seen the number of modules installed by most major libraries? Making a pull re…
Traditionally, this wasn't an acceptable way to think about projects we engineers were being paid lots of money to build.
As you note, a project may hoover in some absurd number of dependent libraries and you have no tooling that tells you which of those might fall in the 1% and what code paths in those 1% intersect with call stacks in your project. You have no idea what impact blindly deleting some "They're probably unnecessary" files in somebody else's code will have on your application and no insight into how to make sure your testing unearths problems. It's an invitation to phantom bugs of unknown scope and the most frustrating kind of debugging effort that comes from chasing those kinds of phantoms.
It's already bad enough that people don't read and review their dependent code with the eye they bring to PR's from their on-team colleagues, but to then go futzing around and deleting things in the unread dependencies because you have a hunch that it's no big deal is about as far from software engineering as you can get.
Just use pnpm
Also, when you create a Docker image, you avoid packing in dev tools that aren't absolutely essential (such as pnpm).
Try this bash one-liner :)
find / -name node_modules -print0 | xargs -0 rm -rfYeah no. Npm already does it at the package registry with ignore/npmignore files, and that's the package authors choice. How much storage can you really save? 50MB? 200MB? is it really worth the risk of running rm on some glob pattern and cross your fingers the packages don't require any of the deleted files?
I tested it recently on a clean install of Strapi: about 250 MB are freed up. Storage is cheap but that still represents a lot, especially inside a Docker image.
The patterns used to find files are specific enough to target only those files that are well known to be useless at runtime.
> In deployment scripts: > > wget -qO- https://raw.githubusercontent.com/xthezealot/npmprune/master... | sh -- -p Serious question: Is this the norm now? Are people actually executing unversioned wget'd shell scripts from random github users as part of their deployment workflow?
> now For about the last 15 years
curl ... | sudo bash
> In deployment scripts: > > wget -qO- https://raw.githubusercontent.com/xthezealot/npmprune/master... | sh -- -p Serious question: Is this the norm now? Are people actually executing unversioned wget'd shell scripts from random github users as part of their deployment workflow?
The threat model is exactly the same as executing untrusted, uninspected content you've downloaded locally. I could do some tricks where I sent different files based on user agent, but still... most people aren't inspecting the download anyway before running it.
Not from githubusercontent you couldn't. Which I'd say is where the majority of these scripts are hosted.
Just for package authors (or people looking for some easy pull requests) out there that might not know this exists. NPMs package.json has a `files` field which allows you to define which files are included on an npm install: https://docs.npmjs.com/cli/v6/configuring-npm/package-json#f... . This also extends to an .npmignore file that works similar to a .gitignore file.
Just beware that some files may seem unnecessary but are expected from an idiomatic npm package. Three things that come to mind -- a markdown file named README.md, any generated typescript definitions, and typescript/babel sourcemaps. And something I've seen far too often: please don't give a minified, rolled up bundle as the only option, otherwise you are chucking your library's users back into the dark ages of Bowe…
We did exactly this when packaging and deploying large node manifests at one of my former companies. Be super careful of removing large swaths of files. Out of 150,000 node modules in your manifest, I'm willing to bet at least one of them is doing something by reading one of these non-source files.
This was my concern as well. Looking at the script source, it's just matching globs, so there isn't much smarts to this. I'm sure it works most of the time, but yeah.. Do JS packages need some kind of .prodignore file similar to other .ignore files? So with a flag passed, after doing an npm install, there's a extra cleanup step that removes explicitly marked files that aren't needed for running in prod? (Not a fully…
Yeah no. Npm already does it at the package registry with ignore/npmignore files, and that's the package authors choice. How much storage can you really save? 50MB? 200MB? is it really worth the risk of running rm on some glob pattern and cross your fingers the packages don't require any of the deleted files?
Not everyone uses the .npmignore file. Maybe it's the author's choice, but in the meantime, that's my personal storage space that's being used unnecessarily. I tested it recently on a clean install of Strapi: about 250 MB are freed up. Storage is cheap but that still represents a lot, especially inside a Docker image. The patterns used to find files are specific enough to target only those files that are well known t…
You are taking a big risk of subtle breakage right now, and a big risk of breakage as you change your project code in the future, as you may start to invoke a code path that needs that resource in the future.