Live data from Hacker News

Flat Tree Dependency Resolution in Npm v3

docs.npmjs.com

81–90 of 123 posts

Re: Flat Tree Dependency Resolution in Npm v3

#81
post #74

Jspm solves this by installing the module in a directory appending the package version. There is no maximally flat tree, the tree is 100% flat. At most there are several versions of the same package side by side, but no nesting. It even supports circular dependencies.

Wow. Thanks for opening my eyes to do that. I was wondering why I would use another package manager.

Re: Flat Tree Dependency Resolution in Npm v3

#82

"dependency resolution depends on install order" Does this sound insane to anyone else? EDIT: I understand not wanting to modify node's `require` semantics, but this is an unacceptable sacrifice of consistency for efficiency. Surely it would have been possible for an `npm install --save x` to put the `node_modules` directory in a state identical to `npm install --save x && rm -rf node_modules && npm install`. It migh…

Totally. That was the first thing that stuck out to me.

Re: Flat Tree Dependency Resolution in Npm v3

#83

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implemen…

This is also what Composer (package manager for PHP) did. They ported openSUSE's libzypp to PHP to solve the dependency management problem. After resolving versions the dependencies are installed in a flat list.

Re: Flat Tree Dependency Resolution in Npm v3

#84
I just filed a bug: https://github.com/npm/npm/issues/10999

I guess I'm not sure what level of non-determinism they expect, but on this page: https://docs.npmjs.com/how-npm-works/npm3-nondet it appears to make the claim that the only effect is on tree structure, not the actual versions of packages that are picked up. And in fact in their example this IS the case. I think this is fine btw.

However, I have found edge cases where install order actually changes the versions of packages that are picked up, and in ways that make it very very difficult to work around (basically you will be forced to manually edit a shrink-wrap file -- so it is necessarily on the end user not the package writer).

Basically, if any package lists and absolute dependency (vs a semver range), it will affect ALL the packages alphabetically later than it and FORCE them to take the same dependency.

Re: Flat Tree Dependency Resolution in Npm v3

#85
post #78
post #63

Sorry to be negative but one of the things that I hate more than anything in my software development work is typing "npm install blah" and almost always being hit with wave after wave of errors typically related to dependencies. I don't know why it happens and I don't care I just wish they'd fix it. So many, many errors. Go on, try installing X packages at random using npm - did they install cleanly? The baseline out…

NPM has some dumb legacy features like optionalDependencies. optionalDependencies are often native code dependencies that might fail but the package is still usable, maybe just some specific feature isn't. That's the most common reason I come across for errors.

Whatever the reason, they should fix it - I spend all day fighting through vast numbers of problems and errors in all sorts of software systems but nothing ranks as high as "npm install" for bad user experience. It's the one thing that browns me off more than anything - I dread having to type "npm install" and its my number one gripe in a world of broken software which is really saying something when so much software is broken.

Re: Flat Tree Dependency Resolution in Npm v3

#86
post #74

Jspm solves this by installing the module in a directory appending the package version. There is no maximally flat tree, the tree is 100% flat. At most there are several versions of the same package side by side, but no nesting. It even supports circular dependencies.

Are there any trade-offs to this approach? This seems so obvious to me I'm confused why it is more widely used.

Re: Flat Tree Dependency Resolution in Npm v3

#87
post #53

Earlier quoted context omitted.

Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that), then you're in a world dominated by Linux and OSX. I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does. Sure, platform specific issues do happen for other OSes, but they are rar…

>> Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that) I think you need to get out more. Or at least go to meetups that don't only meet at coffee shops. You're cutting out a huge chunk there, and there are still huge chunks that use Windows to do Java, C++, etc. I mean, game development is a multi billion dollar industry that is…

>> This is different from Apple with OSX and iOS how?

I'll never run "npm install" or "gem install" or what have you on iOS, Android, etc. Proprietary OSes come in all shapes and sizes, all bringing their own pros and cons to the mix. I just happen to be focusing on a very large con of Windows at the moment.

As for OSX, Redhat, Debian, etc. Sure, they're all different, of course. But generally speaking, at least when it comes to Node and npm (and as far as I know, Ruby, Clojure, Python, etc, please correct me if I'm wrong), these OSes tend to get along pretty OK. Windows, does not.

>> And for crying out loud, file separator issues? That's not even a hard one.

And yet how many times has that problem been hit over the decades? I mean heck, look at the commit message, "Another Windows path issue". And yeah, it's not even a hard one, it just gets worse from there.

I came to that issue from here (which is my project): https://github.com/city41/reagent-breakout/issues/11

So for "not even a hard one", still took me 20 minutes of digging to find out what was going on. X minutes across all developers across how long Windows has been around really adds up. For just one of Windows's differences.

Re: Flat Tree Dependency Resolution in Npm v3

#88

"dependency resolution depends on install order" Does this sound insane to anyone else? EDIT: I understand not wanting to modify node's `require` semantics, but this is an unacceptable sacrifice of consistency for efficiency. Surely it would have been possible for an `npm install --save x` to put the `node_modules` directory in a state identical to `npm install --save x && rm -rf node_modules && npm install`. It migh…

Surely they will have to change that. Right? It can't last long.

Re: Flat Tree Dependency Resolution in Npm v3

#89
post #87

Earlier quoted context omitted.

>> Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that) I think you need to get out more. Or at least go to meetups that don't only meet at coffee shops. You're cutting out a huge chunk there, and there are still huge chunks that use Windows to do Java, C++, etc. I mean, game development is a multi billion dollar industry that is…

>> This is different from Apple with OSX and iOS how? I'll never run "npm install" or "gem install" or what have you on iOS, Android, etc. Proprietary OSes come in all shapes and sizes, all bringing their own pros and cons to the mix. I just happen to be focusing on a very large con of Windows at the moment. As for OSX, Redhat, Debian, etc. Sure, they're all different, of course. But generally speaking, at least when…

I don't think you realize just how different even various Linux distros running even the same kernel can be, and I think you're giving waaay too much credit to OSX/Linux for being "alike". MAKE build scripts have notoriously been a mess for decades because of the differences between distros.

Any reasonable language has the ability to query all of the platform-specific stuff directly from the standard library. You can't even count on two Windows users having their home directories in the same place, or two Arch users having all of their config files in the same place. It's a necessity to be able to abstract all of these away even for a single operating system, which means you get it for free when moving to other operating systems.

Re: Flat Tree Dependency Resolution in Npm v3

#90
post #65

Earlier quoted context omitted.

When we ship node modules to be embedded into Chocolat.app, we have to strip out all the tests and documentation because otherwise it bloats the download size. And the computer it eventually runs on may not be connected to the internet. The nice thing about node is that it's quite small (3.4MB zipped). We don't have to ship npm with the app, and certainly an OS level package manager would be way too large!

Each package should strip tests and documentation from the package. Of course, many don't..

True enough... it's not always easy to consider as creating an npm module and building/pushing to NPM is really easy to do.
Post reply on HN