Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hello Yarn 2, Goodbye node_modules

freddixx.medium.com

Post reply on HN
Print view View original

91–100 of 160 posts

Re: Hello Yarn 2, Goodbye node_modules

#91

Post by plorkyeran »

alerighi wrote: post #57
acemarke wrote: post #32

Earlier quoted context omitted.

That's the point, it's _not_ the same files. Instead of extracting 75K individual files on disk, the Plug 'n Play concept keeps every package as its zipped tarball on disk. So, N tarballs, one per package, instead of hundreds of files per package. Conceptually, it's great. The problem comes when all the other tools in the ecosystem are still expecting a `node_modules` folder to exist and to be able to read those indi…

The archive must be extracted to run the program. So they maybe consume less space on disk, but then you pay a penalty every time you run the program since (I assume) all these packages are extracted when you type yarn start (otherwise how can it work?) The problem of wasted disk space... start using a modern filesystem like ZFS (or BTRFS, but it's less stable) that does Copy on Write and you shouldn't have these pro…

Decompressing files from a single tar.gz into memory can very easily be faster than opening a very large number of small files.

Re: Hello Yarn 2, Goodbye node_modules

#92

Post by inopinatus »

codetrotter wrote: post #64
inopinatus wrote: post #51

Earlier quoted context omitted.

The first-party solution is ES Modules. Node is not Javascript.

ES Modules aren't package management though? In that ES Modules say nothing about how to get the modules in the first place, nor anything about versions. Not in any standardized way anyhow.

Solutions are defined by the problems they address, not the mechanics of how they address them, or even particularly how good or thorough they are.

Which is to say, yes, it’s painfully half-baked and riddled with inconsistencies. But that is pretty typical of JavaScript anyhow.

Re: Hello Yarn 2, Goodbye node_modules

#93

Post by joshmanders »

Aeolun wrote: post #58
drannex wrote: post #40

I may be the only one around who cares about this, but I _like_ node_modules. There are so many times where I am working on a project and either 1) I don't understand an API in a package so I quickly click into the module to learn more or 2) I find bugs, and can do a quickfix right there and then in the node_modules and test them out before submitting either a PR or work on a fork. I guess it would be possible(?) to…

I think pnpm retains your node_modules folder, but always has only a single copy of each package version on disk. I like that.

Which makes the usecase OP described still not doable.

Working on multiple projects that use the same dependency and you notice something and want to test it in node_modules like op described, and don't revert it now you're wondering why one project is broken.

Re: Hello Yarn 2, Goodbye node_modules

#94

Post by danielheath »

pmoleri wrote: post #83
ezekg wrote: post #80

Earlier quoted context omitted.

npm is one of the only dependency managers I’ve seen that does more than install packages. Every other package manager: pip, bundler, composer, all do one thing and do it well. npm does everything but nothing well.

Wow, bold statement. Can you give examples of things it does wrong?

I can.

In most ecosystems, I can share package sources between a VM and a host.

npm shares mutable content (eg compiled artifacts) with package sources in `node_modules`. That breaks a ton of workflows that are common in other ecosystems.

For instance, in ruby apps using bundler, I can commit my dependencies in vendor/cache; there are no network dependencies other than fetching the source code. That makes turning code into a running server faster and more reliable.

Go supported the same thing from v1 via GOPATH (because that's how google runs their repo). Commit your dependencies and carry on (go had different shortcomings in dependency management in those days).

This feature turns things like `left-pad` from a fiasco into a non-event.

Also, it took npm years to implement a lockfile, only for most npm commands to disregard it. I used to frequently get versions other than the one I wanted. It's been ~6 years since I switched away so it may have improved since then. After my experience using npm I fundamentally do not trust the brand, and until yarn screws up I have no reason to give npm another chance.

Re: Hello Yarn 2, Goodbye node_modules

#95

Post by _fat_santa »

enumjorge wrote: post #31
wilg wrote: post #20

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or…

> There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosy…

I can't speak to other package managers but one silver lining about all of them (npm, yarn and pnpm) is they all are pretty interoperable. Between yarn and npm for example, I'm pretty sure they have virtually the same API so migrating from one to the other is a simple matter of using `yarn ` instead of `npm `

Re: Hello Yarn 2, Goodbye node_modules

#96

Post by zdragnar »

errantspark wrote: post #15
zdragnar wrote: post #7

Earlier quoted context omitted.

For a good while most people were better served by yarn anyway. I don't know exactly the current state of npm, but part of yarn's draw (even prior to the pnp aspect) was having deterministic builds. Even with a package lockfile, npm couldn't do that.

Ahh, interesting, I wasn't aware of that issue. Personally the problem with npm for me has never been "the node_modules directory is too big" or "npm is non-deterministic" but rather "the quality of other people's code is terrible" or "this library solves 99 other cases in addition to the one I care about". I think if you're careful about what you pull in from npm it's totally fine.

Having deterministic builds is a godsent boon in an ecosystem without proper semantic versioning enforcement. The fact that minor version number changes can be breaking means that every build can break (a bit tautological but a point worth emphasizing).

I also dont like that some packages are too big, or too small, or have too many transitive dependencies. It is a fact of life in npm land, though, making deterministic builds simultaneously minor and significant.

Re: Hello Yarn 2, Goodbye node_modules

#97

Post by zdragnar »

Zababa wrote: post #18
zdragnar wrote: post #7

Earlier quoted context omitted.

For a good while most people were better served by yarn anyway. I don't know exactly the current state of npm, but part of yarn's draw (even prior to the pnp aspect) was having deterministic builds. Even with a package lockfile, npm couldn't do that.

> Even with a package lockfile, npm couldn't do that. NPM now has npm ci (which I think should be the default) for that.

Is there a way to install another package / add one to package.json and update the lockfile while otherwise preserving the behavior of running npm ci?

If so, ignore this, but if not, wouldn't adding a new package have all sorts of weird side effects from miscellaneous updates to irrelevant existing dependencies?

Re: Hello Yarn 2, Goodbye node_modules

#98

Post by diegof79 »

chii wrote: post #76
_6pvr wrote: post #38

Earlier quoted context omitted.

> ere really shouldn't be two standard package managers for JS. Why? Aren't there multiple package managers in many languages? SBT/Maven, Ant/Ivy/Maven, PIP/Poem(or poetry or whatever it is). What's the harm?

notice how maven appears twice in your list for anything jvm related - that's because the jvm ecosystem is mature, and there's really only 1 package manager.

This is because the Maven repository format is simple: it’s a directory convention + an xml file on each directory to indicate dependencies. It’s easy to implement and host in any web server. Other tools (Ivy, Gradle, SBT) can consume it. But, in essence it’s the same with npm and yarn: both get packages from the npm registry (the common denominator is the repository format).

Re: Hello Yarn 2, Goodbye node_modules

#99

Post by remexre »

wilg wrote: post #20

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or…

> it would be great to have a language-agnostic package manager that worked more or less like Bundler and can solve these problems once

Haven't used Bundler, but have you looked at Nix? It's got support for a bunch of languages/ecosystems, including the npm one: https://nixos.org/manual/nixpkgs/stable/#chap-language-suppo...

Re: Hello Yarn 2, Goodbye node_modules

#100

Post by q-rews »

enumjorge wrote: post #31
wilg wrote: post #20

This is probably a good direction, but it seems like it's still contributing to JS dependency management becoming even hotter of a mess. There really shouldn't be two standard package managers for JS. This also reminds me how much we keep reinventing the wheel on this. I know it's likely infeasible for various reasons but it seems like it would be great to have a language-agnostic package manager that worked more or…

> There really shouldn't be two standard package managers for JS. Amen to this. This type of stuff leads to so much confusion especially for beginners. I remember the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse when I started to get into front-end development. Yes having multiple choices lets us experiment with alternatives, but I think we underestimate the costs of complicating the ecosy…

> the whole CommonJs vs RequireJs vs AMD modules was really difficult to parse

Oh man sounds like nothing changed. Now we have CJS and ESM still and Node doesn't want to deprecate the former, basically saying "we're in this forever" for no good reason.

Post reply on HN