Flat Tree Dependency Resolution in Npm v3 • Hacker News Board

Re: Flat Tree Dependency Resolution in Npm v3

#11

Post by frik » Thu, Dec 31, 2015, 4:53 PM UTC

shill wrote: post #4

This will help Windows users who have filesystem and zip compression issues with deeply nested dependency trees.

Windows to this day still has the 260 chars path limit.

http://msdn.microsoft.com/en-us/library/aa365247(VS.85).aspx...

https://en.wikipedia.org/wiki/Long_filename

Microsoft should finally fix such old limitations (-> update WinAPI), instead adding work arounds to third party projects like Nodejs.

https://github.com/Microsoft/nodejstools/issues/69

They could also improve the command line shell (cmd.exe) that also PowerShell relies on.

Re: Flat Tree Dependency Resolution in Npm v3

#12

Post by city41 » Thu, Dec 31, 2015, 4:55 PM UTC

kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened?

I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

Re: Flat Tree Dependency Resolution in Npm v3

#13

Post by theGimp » Thu, Dec 31, 2015, 4:55 PM UTC

That's a great improvement over the old behavior, but I'm wondering why the team didn't go a different route.

Why not always store packages at the top level, and create a directory for each version that's required?

For example: directory "A" contains two subdirectories: "v0.1.0" and "v0.1.1"

Re: Flat Tree Dependency Resolution in Npm v3

#14

Post by [deleted] » Thu, Dec 31, 2015, 4:57 PM UTC

amasad wrote: post #6

This is good news for client side bundlers like Webpack and Browserify -- where size really matters -- they don't have to end up with multiple copies of the same module. I would also assume for very large apps it may improve startup time because you don't have to initialize and retain multiple copies of the same module.

[deleted]

Re: Flat Tree Dependency Resolution in Npm v3

#15

Post by mavdi » Thu, Dec 31, 2015, 5:00 PM UTC

vec wrote: post #10

So, assuming I have two libraries, A and B, that both require the same version of library C, do those libraries still get their own separate in-memory copies of C, or do they share a singleton? It's terrible practice, but it's not unheard of for an NPM module to monkey patch its dependencies, since before this the library could assume it had sole ownership of its whole subtree.

Interesting. This would indeed be a problem. I hope they don't share them between modules because if one mutates a dependancy, it would be nearly impossible to debug/fix.

Re: Flat Tree Dependency Resolution in Npm v3

#16

Post by Klathmon » Thu, Dec 31, 2015, 5:00 PM UTC

kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

I love the path that some take which is to maintain both a "pure" JS-only solution and a "native" solution, and have the native one be optionally installed. That way it works as best it can automatically.

Now I know that can't always be the case as it adds a lot of work to the development and maintenance, but it is a really nice thing when i see it.

Re: Flat Tree Dependency Resolution in Npm v3

#17

Post by tvanantwerp » Thu, Dec 31, 2015, 5:01 PM UTC

city41 wrote: post #12

kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

As someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.

Re: Flat Tree Dependency Resolution in Npm v3

#18

Post by frank-weindel » Thu, Dec 31, 2015, 5:04 PM UTC

amasad wrote: post #6

This is good news for client side bundlers like Webpack and Browserify -- where size really matters -- they don't have to end up with multiple copies of the same module. I would also assume for very large apps it may improve startup time because you don't have to initialize and retain multiple copies of the same module.

This has been around for a number of months. The sad thing is because of this unpredictable (or rather arbitrarily alphabetical) `npm install` order different dependency trees can result which can still lead to a very common module being bundled multiple times. I was a fan of bower's strictly flat model because it prevents such duplication and even notifies you when incompatibilities occur. However bower seems to be losing the battle with NPM as the defacto web/javascript module repo.

The allowed/unpredictable duplication can even cause very hard to identify bugs when a peer dependency relies on "instanceof" checks and there are multiple versions of this dependency. I've seen it happen with React and Backbone to name a couple.

If the `npm install` allowed control over install order (instead of just being alphabetical) and there was a way to be notified of incompatibilities that would cause potentially unnecessary duplication that would be at least something that could prevent problems like this from occurring.

Re: Flat Tree Dependency Resolution in Npm v3

#19

Post by davexunit » Thu, Dec 31, 2015, 5:04 PM UTC

kylecordes wrote: post #8

I understand that this is a hairy, messy problem they are solving, and that they traded off one aspect of the problem for another (Install order dependency! as a new "feature" in 2015!). But I wish they had aimed higher. The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details (compiler used for native modules).…

>The better goal would be that the entire state of the node modules directory is a pure function of the contents of the package.json file plus the platform details

Yes, this would have been much better than this new, also-broken approach. Functional package managers like the one I work on, GNU Guix, have this problem solved and solved well. Language-specific package manager developers would do well to either implement similar systems in their projects, or realize the inherent problems with limiting the scope of dependency management to a single language rather than the full dependency graph and switch to using functional package managers.

Re: Flat Tree Dependency Resolution in Npm v3

#20

Post by city41 » Thu, Dec 31, 2015, 5:13 PM UTC

tvanantwerp wrote: post #17

city41 wrote: post #12

Earlier quoted context omitted.

Why do dev communities have to continually pay the Windows tax? If Microsoft keeps insisting on being an outlier in the world of OSes, shouldn't they be the ones burdened? I gotta say, as someone who never ever uses Windows but maintains a couple of open source packages, I'm really sick of the Windows only problems that crop up.

As someone who has no choice but to use Windows at work, I'm really sick of all the problems that crop up because so many tools use were written by people who won't use Windows and haven't tested anything on it.

I feel your pain, but am not convinced open source maintainers should have to be the ones to resolve the issue. Especially since so many projects are done using spare time and with no compensation.