Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Node's nested node_modules approach is basically incompatible with Windows

github.com

Post reply on HN
Print view View original

51–60 of 62 posts

Re: Node's nested node_modules approach is basically incompatible with Windows

#51

Post by brokenparser »

addisonj wrote: post #15

I am somewhat surprised at this topic. Of all the the issues Node.js has on windows, this seems like a fairly minor one. In my eyes, the biggest issue is simply this: A lot of module writers don't care about Windows. A big part of the node ecosystem is that there is a module for practically anything. While node core has great support for windows, and also helps aid developers in writing code that is windows compatibl…

Windows actually supports using / as a path delimiter. If a program is (following the robustness principle) liberal in what it accepts (both / and \\) and strict in what it sends (just /), it'll work (barring any other issues).

Re: Node's nested node_modules approach is basically incompatible with Windows

#52

Post by cwmma »

aikah wrote: post #29
jeswin wrote: post #18

Earlier quoted context omitted.

> second one is nodejs dependency on what google does/doesnt do,since google maintains V8 Google maintaining V8 is one of the big reasons why Node has been successful. There is no way an independent Node.JS foundation would have the resources to optimize and test JS like the Chrome team. So far, the problems have been minor and V8 has done a decent job keeping up with upcoming JS standards. So this isn't a problem. >…

I wonder why Mozilla didnt/couldnt jump in the ship and propose a viable alternative to node/V8,I know there is a node/spidermonkey project somewhere but it isnt maintained. V8 api changes made a lot of binary modules break and it will happen again,so it is a problem on the long run.Of course NodeJS team cant maintain V8. But please dont throw that problem under the carpet,It's a real issue. > Google maintaining V8 i…

TJ (the node lead) has suggested, fairly recently that making it easy to use other engine is something he is open to, at a node.js on the road event he mentioned that in his ideas for study so the line.

Re: Node's nested node_modules approach is basically incompatible with Windows

#53

Post by wolfgke »

Windows does support long paths (up to about 32k characters): Here is a good explanation from the .net perspective (but the same holds for non-managed APIs): http://blogs.msdn.com/b/bclteam/archive/2007/02/13/long-path...

Long paths are just most ugly to use: "Long paths with the \\?\ prefix can be used in most of the file-related Windows APIs, but not all Windows APIs. For example, LoadLibrary, which maps a module into the address of the calling process, fails if the file name is longer than MAX_PATH. So this means MoveFile will let you move a DLL to a location such that its path is longer than 260 characters, but when you try to load the DLL, it would fail. There are similar examples throughout the Windows APIs; some workarounds exist, but they are on a case-by-case basis."

Re: Node's nested node_modules approach is basically incompatible with Windows

#54

Post by lucisferre »

I actually find it surprising how many people feel recursive node_module dependency folders is justified. It seems absolutely the worst possible solution to me. Am I misunderstanding something about this. Is there any reason Bundler's approach doesn't work here?

It seems ridiculous to me how many folders are created when I just need a couple of node modules to do something.

Re: Node's nested node_modules approach is basically incompatible with Windows

#55

Post by lucisferre »

This is going on a bit of a tangent, but even Visual Studio used to run up against this problem all the time using some of Microsoft's own libraries. It was laughable at the time, but some of Microsofts teams (cough Patterns & Practices cough) loved to created DLLs with huge long namespaces. That coupled with the fact that Visual Studio projects were created in /Users/User.Name/Visual Studio/Visual Studio ##/Projects/ to begin with created a lot of problems.

As a result I almost always put all my projects at the root something like /src to reduce the odds of running into these problems.

Re: Node's nested node_modules approach is basically incompatible with Windows

#56

Post by dbbolton »

nopar8 wrote: post #14
dbbolton wrote: post #8

This definitely seems more like a Windows issue than a Node issue.

At first I agreed with you; but then I read one of the comments on github that swayed me. "You are the one playing games - calling core parts of Windows like Explorer "3rd party tools", and suggesting that not supporting long paths is a bug. Microsoft have made it clear repeatedly that non-support for long paths is not a bug, and not something that will change. A package manager creating paths that do not work with t…

Right, in practical terms it's up to Node to fix this (or go extreme and drop Windows support), but in a more abstract sense I still blame Microsoft for such a glaring design flaw in their software that they are unwilling/unable to fix.

Re: Node's nested node_modules approach is basically incompatible with Windows

#57

Post by malandrew »

jonpacker wrote: post #48
malandrew wrote: post #13

Earlier quoted context omitted.

Totally. This is actually a trivially solvable problem (I've been working on a meta package manager compatible with npm, bower and component). The strategy I took with node_modules/, components/ and bower_modules/ was to just follow whatever folder structure it found as far as necessary and look for manifest files (package.json, bower.json and component.json) and register modules at the path of wherever one of those…

I respect where you're coming from, but I disagree. One of the huge advantages of npm for me is that it's so transparent and I can see exactly how it's mapping to the file system. I know exactly which code is being run, from where—because it's incredibly easy to traverse the dependency tree. If you abstract away the fs level and flatten the dependencies into a single folder, you take this away, and you make the depen…

Nothing in what I'm doing prevents what you like about npm. In fact npm already abstracts away what you want in a little known command called `npm explore`.

Let's say you want to explore the dependency `foo` in your project. At the root of your project type `npm explore foo` and press enter and you'll be in a shell exploring the foo module. Then if you want to return, type `exit` and return and you'll be back in your project. Or if you want to go down the rabbit hole to a dependency of foo called bar, type `npm explore bar` and press enter.

`npm explore` is a great way to explore things and navigate to the dependencies used by the dependency you want to investigate. It basically uses the information in the current package.json to find the dependency you want.

Re: Node's nested node_modules approach is basically incompatible with Windows

#58

Post by xpaulbettsx »

jonpacker wrote: post #47
xpaulbettsx wrote: post #28

Earlier quoted context omitted.

Is creating dozens of nested folders inside each other really a sane thing to do? Run `find .` on any reasonably complicated node project and look at the crazy paths that show up. npm's strategy of nesting like that made things simpler when it was more of a new project, but it's time to come up with something more robust.

I disagree. I think node's approach is easily understandable, and quite robust. It's incredibly easy to dive down the dependency chain and see every package, and know exactly which one is being used where. Not something that can be said about package managers that use the flattened approach.

Pretty Printing the full dependency tree is a feature that could be easily added to npm itself, and in a way easier way to understand than trying to read through a directory tree

Re: Node's nested node_modules approach is basically incompatible with Windows

#59

Post by WorldWideWayne »

russell_h wrote: post #7
rtpg wrote: post #4

Was the recursive "every package include its dependencies" done just because it was easier? Or was it a conscious design choice.

Personally, I consider the way Node does module loading one of its best features. No need to worry about version conflicts, because every module can (optionally) pull in a specific version if necessary. And no need to worry about constructing some ridiculous environment variable, because of the implicit relative pathing. I've been writing Python and Go for the last 12 months, and by comparison their module systems ar…

Multiple versions of the same dependency could still work with a flat module file system and I don't see any reason why an environment variable would be needed.

Re: Node's nested node_modules approach is basically incompatible with Windows

#60

Post by rando289 »

tlrobinson wrote: post #38
rando289 wrote: post #30

Earlier quoted context omitted.

windows has path length limits. Figure out how to live in them, whatever way is best. If that means using cryptic names, then do it. cryptic folder names, while not a good thing, are extremely common in windows.

Or, don't support Windows.

Yes, fine by me.
Post reply on HN