Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flat Tree Dependency Resolution in Npm v3

docs.npmjs.com

Post reply on HN
Print view View original

41–50 of 123 posts

Re: Flat Tree Dependency Resolution in Npm v3

#41

Post by ag_dubs »

tvanantwerp wrote: post #17
city41 wrote: post #12

Earlier quoted context omitted.

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

As someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.

hi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs).

npmjs cares about the enterprise + windows.

Re: Flat Tree Dependency Resolution in Npm v3

#43

Post by davexunit »

munificent wrote: post #28
davexunit wrote: post #19

Earlier quoted context omitted.

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implemen…

> Language-specific package manager developers would do well to either implement similar systems in their projects This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages. It's a challenge because constraint solving is NP-complete, but it works very well in practice. [1]: https://p…

>This is what we did with Dart's package manager[1]. It works like Bundler where it finds a single set of package versions that satisfy all of the constraints and there are no duplicate packages.

Constraint solving is the status quo, and I am strongly opposed to it. Rather than saying "I want foo >= 1.0" in a package manager, you should have the build system test the environment for the features you need. This is what the Autotools and other build systems that have been around for decades do. This way, the project isn't tied to a single package manager and, in the case of free software, greatly eases the burden on package maintainers that want to add your software to their distribution.

Functional package management doesn't need constraint solving because every package in the system precisely describes itself: exactly which dependencies are needed for build-time and runtime (which precisely describe themselves, recursively, all the way down to libc), precisely which source code (tarball/directory/whatever + checksum), and the exact build script that turns that source code into a (hopefully bit-reproducible) binary. There's no constraint solving to determine what the dependency tree is, it's already been encoded in the package objects themselves. This is what any robust package manager should enable, but the only two that I know of that do this are Nix and GNU Guix.

>The problem then is now your language has a dependency on an outside package manager, one which is often OS specific. Most modern languages need to support a variety of OSes.

A programming language should not depend on any specific package manager. This is what build systems are for! More and more language communities conflate the two, but this is a huge mistake that we are paying for by making our software nearly impossible to reproduce from source code. [0] Everyone just uses pre-built binaries full of circular dependencies and reproducibility issues because things are so tangled that no one actually knows how to build anything from source anymore.

I think the value of language-specific package managers lies in providing an easy way to fetch pure (no native code) modules to faciliate simple code sharing and helping newcomers get bootstrapped quickly. However, for serious software development and deployment they are terrible and we desperately need better tools. I think functional package managers are the tools we need, as they have greatly simplified software building and deployment for me whilst also greatly increasing the reliability of the systems that use them thanks to transactional upgrades and rollbacks.

[0] http://www.vitavonni.de/blog/201503/2015031201-the-sad-state...

Re: Flat Tree Dependency Resolution in Npm v3

#44

Post by tedmiston »

[deleted] wrote: post #14
amasad wrote: post #6

This is good news for client side bundlers like Webpack and Browserify -- where size really matters -- they don't have to end up with multiple copies of the same module. I would also assume for very large apps it may improve startup time because you don't have to initialize and retain multiple copies of the same module.

[deleted]

Not really -- unless you want to fork each top-level dependency and update the common dependencies to the latest common subversion in package.json (and maintain that). Far from ideal IMO.

Re: Flat Tree Dependency Resolution in Npm v3

#45

Post by boucher »

theGimp wrote: post #34
vec wrote: post #25

Earlier quoted context omitted.

Because then they'd have to change the semantics of how `require()` works. As it is, node packages are not version aware at runtime. Instead NPM is designed to place packages such that any library will have the correct version first in their load path. Changing the directory structure to encode versions is probably the Right Way in a grand sense, but it's a much more substantial change than just promoting packages by…

I thought about that, but clearly they have enough information to figure out whether to load the package from the top level or a nested directory. (I'm guessing simply by bubbling up and looking for the "node_packages" directory.) It doesn't seem too big of a leap to do it the way I was suggesting. Either bubble up and look for "node_packages", but this time store a symlink there, or add a dot file that tells you "th…

The information may be available, but ultimately the work would have to be done by node, not npm, and it seems reasonable to not want to change such a fundamental node behavior, especially just to support a change in npm.

Re: Flat Tree Dependency Resolution in Npm v3

#46

Post by ag_dubs »

[deleted] wrote: post #14
amasad wrote: post #6

This is good news for client side bundlers like Webpack and Browserify -- where size really matters -- they don't have to end up with multiple copies of the same module. I would also assume for very large apps it may improve startup time because you don't have to initialize and retain multiple copies of the same module.

[deleted]

forcing a single copy of a dep is exactly the opposite of the dependency strategy both the node module loader and npm take. the fact that the node module loader can load more than one version of a module into memory is its strength and npm plays to this, and in doing so avoids "dependency hell". you might checkout this page in the docs that talks more about this: https://docs.npmjs.com/how-npm-works/npm2

Re: Flat Tree Dependency Resolution in Npm v3

#47

Post by theGimp »

boucher wrote: post #45
theGimp wrote: post #34

Earlier quoted context omitted.

I thought about that, but clearly they have enough information to figure out whether to load the package from the top level or a nested directory. (I'm guessing simply by bubbling up and looking for the "node_packages" directory.) It doesn't seem too big of a leap to do it the way I was suggesting. Either bubble up and look for "node_packages", but this time store a symlink there, or add a dot file that tells you "th…

The information may be available, but ultimately the work would have to be done by node, not npm, and it seems reasonable to not want to change such a fundamental node behavior, especially just to support a change in npm.

Ah, fair enough.

I assumed the same team was in charge of both.

Re: Flat Tree Dependency Resolution in Npm v3

#48

Post by DrJokepu »

kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

Meh. Anyone who ever worked with the v8 C++ interop API knows that it's a rather unpleasant experience (unlike e.g. Python's C interop API, which is very nice). Nobody uses it out of pure laziness, but because They Have A Very Good Reason To Do So™.

Re: Flat Tree Dependency Resolution in Npm v3

#49

Post by city41 »

ag_dubs wrote: post #41
tvanantwerp wrote: post #17

Earlier quoted context omitted.

As someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.

hi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs). npmjs cares about the enterprise…

I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.

Re: Flat Tree Dependency Resolution in Npm v3

#50

Post by gelasio »

city41 wrote: post #12
kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

It sounds like you're getting annoyed that the devs of certain projects like Node.js have chosen to support an OS that you don't use without asking you about it first... If you're in control of the project - you have every right to ignore Windows users. If you're not in control, then you really have zero right to complain. Zero.

Devs certainly aren't forced to "pay the Windows tax" in any way. Plenty of OSS projects completely ignore Windows. I'm sure you're aware though that Windows has the majority market share of desktop operating systems. If your software is interesting to the vast, vast population of Windows users, you might get some requests to support Windows. And since it's not an outlier in the market, other desktop OSes should obviously strive to be more like Windows ;)

Post reply on HN