Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Flat Tree Dependency Resolution in Npm v3

docs.npmjs.com

Post reply on HN
Print view View original

51–60 of 123 posts

Re: Flat Tree Dependency Resolution in Npm v3

#51

Post by alextgordon »

davexunit wrote: post #19
kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implemen…

When we ship node modules to be embedded into Chocolat.app, we have to strip out all the tests and documentation because otherwise it bloats the download size. And the computer it eventually runs on may not be connected to the internet.

The nice thing about node is that it's quite small (3.4MB zipped). We don't have to ship npm with the app, and certainly an OS level package manager would be way too large!

Re: Flat Tree Dependency Resolution in Npm v3

#52

Post by MatthewPhillips »

amasad wrote: post #6

This is good news for client side bundlers like Webpack and Browserify -- where size really matters -- they don't have to end up with multiple copies of the same module. I would also assume for very large apps it may improve startup time because you don't have to initialize and retain multiple copies of the same module.

I don't know why that would be the case. Browserify can do semver comparison of dependencies and not bundle multiple copies. I would assume they were already doing this; if not, why?

I work on a client-side module loader StealJS[1] that implements the npm algorithm (2 and 3) in the browser and 3 makes things worse for us. With NPM 2 we could load a project without causing 404s (except in the cause of require("./folder")) but since NPM 3 says "install order matters" now it's no longer deterministic and 404s are more common.

It would be nice if NPM had some working group to discuss their algorithm and invited in other implementers for feedback.

[1]http://stealjs.com/

Re: Flat Tree Dependency Resolution in Npm v3

#53

Post by city41 »

gelasio wrote: post #50
city41 wrote: post #12

Earlier quoted context omitted.

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

It sounds like you're getting annoyed that the devs of certain projects like Node.js have chosen to support an OS that you don't use without asking you about it first... If you're in control of the project - you have every right to ignore Windows users. If you're not in control, then you really have zero right to complain. Zero. Devs certainly aren't forced to "pay the Windows tax" in any way. Plenty of OSS projects…

Windows is very much an outlier in the world of software development. Unless you're specifically using Microsoft tech (ie .NET and all that), then you're in a world dominated by Linux and OSX.

I'm annoyed that Microsoft insists on going forward with a proprietary OS that does not conform to standards that pretty much every single other OS does. Sure, platform specific issues do happen for other OSes, but they are rare. They are extremely common for Windows.

For sure, I have no obligation to fix Windows specific issues in my projects. All of my projects are done in my spare time and given away for free. But Windows specific issues are still annoying and disappointing.

Another aspect of this that is annoying is that major, mission critical open source projects have to dedicate so many resources to Windows. For example this bug in ClojureScript: https://github.com/clojure/clojurescript/commit/80d46bdb7969...

Imagine if the Clojure team didn't have to worry about file separator differences, and the multitude of other Windows annoyances. How much extra energy, time and resources would they gain? I suspect a lot.

EDIT: And I would also say I have every right to complain about Microsoft. Ignoring standards and doing their own thing for their own financial gain at everyone else's expense is definitely worthy of complaint. Also see Internet Explorer.

Re: Flat Tree Dependency Resolution in Npm v3

#54

Post by gelasio »

city41 wrote: post #49
ag_dubs wrote: post #41

Earlier quoted context omitted.

hi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs). npmjs cares about the enterprise…

I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.

So, are you of the opinion that OSS projects should never be built for any commercial, proprietary OS or just Windows (i.e. because it's not Unix)?

For instance - OS X window managers suck. Do you think that a project like Spectacle should have never been built because they're just picking up Apple's slack?

Re: Flat Tree Dependency Resolution in Npm v3

#55

Post by vec »

theGimp wrote: post #47
boucher wrote: post #45

Earlier quoted context omitted.

The information may be available, but ultimately the work would have to be done by node, not npm, and it seems reasonable to not want to change such a fundamental node behavior, especially just to support a change in npm.

Ah, fair enough. I assumed the same team was in charge of both.

Probably a lot of overlap, but they're separate projects with different versions and release schedules.

Re: Flat Tree Dependency Resolution in Npm v3

#56

Post by city41 »

gelasio wrote: post #54
city41 wrote: post #49

Earlier quoted context omitted.

I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.

So, are you of the opinion that OSS projects should never be built for any commercial, proprietary OS or just Windows (i.e. because it's not Unix)? For instance - OS X window managers suck. Do you think that a project like Spectacle should have never been built because they're just picking up Apple's slack?

I don't see how a project like Spectacle, which is OSX specific, relates. If someone wants to make an OS specific project, more power to them. The issue arises on platforms that span OSes, and they just never quite span onto Windows correctly.

Microsoft is very happy that Node, Java, Clojure, etc work on Windows. It helps them a lot. But they make little to no effort to help bridge the gap, they usually leave that to everyone else. That's my gripe, right there.

And to be fair, this is not a black and white situation. It's complex and Microsoft's role in all of this is complex too. And sometimes this issue does crop up on other OSes, for example Docker on OSX is sometimes a little painful.

Re: Flat Tree Dependency Resolution in Npm v3

#57

Post by steveklabnik »

city41 wrote: post #49
ag_dubs wrote: post #41

Earlier quoted context omitted.

hi! i totally know this is a common feel for many npmjs users. this is why i've been running an empathy campaign for Windows, Windows Wednesday, where i do all my work (with/for npm) on windows. it's been good for finding bugs, either technical or UX. i'm working on improving the docs to make them friendlier to all platforms (not just OSX, which is the most common platform at npmjs). npmjs cares about the enterprise…

I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.

What problem is that? That they have a different operating system?

npm is also a bit different, in your objection: they're a company. Companies need users. Supporting more platforms to get more users seems like pretty much, well, business as usual.

Re: Flat Tree Dependency Resolution in Npm v3

#58

Post by city41 »

steveklabnik wrote: post #57
city41 wrote: post #49

Earlier quoted context omitted.

I think this is the wrong approach. We shouldn't pick up MS's slack, MS should. This just enforces the current situation and keeps enabling MS to ignore the problem.

What problem is that? That they have a different operating system? npm is also a bit different, in your objection: they're a company. Companies need users. Supporting more platforms to get more users seems like pretty much, well, business as usual.

That is a fair point. npm being a company does change the situation.

Re: Flat Tree Dependency Resolution in Npm v3

#59

Post by gelasio »

city41 wrote: post #56
gelasio wrote: post #54

Earlier quoted context omitted.

So, are you of the opinion that OSS projects should never be built for any commercial, proprietary OS or just Windows (i.e. because it's not Unix)? For instance - OS X window managers suck. Do you think that a project like Spectacle should have never been built because they're just picking up Apple's slack?

I don't see how a project like Spectacle, which is OSX specific, relates. If someone wants to make an OS specific project, more power to them. The issue arises on platforms that span OSes, and they just never quite span onto Windows correctly. Microsoft is very happy that Node, Java, Clojure, etc work on Windows. It helps them a lot. But they make little to no effort to help bridge the gap, they usually leave that to…

"But they make little to no effort to help bridge the gap, they usually leave that to everyone else."

OK, but that's also where you're wrong, right there. In general, but also particularly for this exact issue - Microsoft considered attempting to fix the file path limit for the first release of Windows 10. Maybe you don't understand, but it's going to take a herculean effort to fix this. They want to do it and if you need proof of that I can find posts of them talking about what I just said.

Re: Flat Tree Dependency Resolution in Npm v3

#60

Post by moron4hire »

vec wrote: post #25
theGimp wrote: post #13

That's a great improvement over the old behavior, but I'm wondering why the team didn't go a different route. Why not always store packages at the top level, and create a directory for each version that's required? For example: directory "A" contains two subdirectories: "v0.1.0" and "v0.1.1"

Because then they'd have to change the semantics of how `require()` works. As it is, node packages are not version aware at runtime. Instead NPM is designed to place packages such that any library will have the correct version first in their load path. Changing the directory structure to encode versions is probably the Right Way in a grand sense, but it's a much more substantial change than just promoting packages by…

Read the next-highest package.json file for whatever code is executing (dependencies copy their package.json file when they install), find the dependency listing in there, with the desired version constraint. If none exists (i.e. the dependency was installed manually), take wahtever is the highest version by default. If that's not what the developer wants, it's their own fault for not specifying the dependency in their package file.
Post reply on HN