Live data from Hacker News

Node's nested node_modules approach is basically incompatible with Windows

github.com

11–20 of 62 posts

Re: Node's nested node_modules approach is basically incompatible with Windows

#11
The Win32API was initially designed to make it very easy to port from Win16API in the early 1990s. Microsoft neglected to improve the Win32API starting with the dotNet adventures. Instead of an improved Win64API we have the identical API for 32 and 64 bit (and WinRT API). The shell still doesn't support UNC path. The Windows NT series operating system would support different subsystems other than Win32.

Naming Files, Paths, and Namespaces on MSDN explains the details: http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...

Re: Node's nested node_modules approach is basically incompatible with Windows

#12
The easiest solution I've found is to just run a VM and not sync whatever folder you install node_modules in. I just had this problem like a week ago with gulp because I was syncing the folder I was building in. It's a good time to learn how to use Vagrant.

Re: Node's nested node_modules approach is basically incompatible with Windows

#13

The title looks exaggerated. I thought NodeJS cannot just run on Windows anymore. It's not the case though, it does - it's just it creates folders that are nested.

Totally. This is actually a trivially solvable problem (I've been working on a meta package manager compatible with npm, bower and component).

The strategy I took with node_modules/, components/ and bower_modules/ was to just follow whatever folder structure it found as far as necessary and look for manifest files (package.json, bower.json and component.json) and register modules at the path of wherever one of those three file types is found. This approach allows any folder organization whatsoever. This information was then saved to a key-value store and the store is only updated when the mtime of the folder changes. This makes startup relatively fast since folders are only re-indexed if something has genuinely changed.

Writing an algorithm that is dependent on the folder structure to me is a fundamentally bad idea most of the time, especially when you have a manifest file that better identifies a module. Additionally, for modules with semver ranges or installed from unusual locations (i.e. URIs: git:// git+ssh://, https://, etc.), you can write an additional metadata file with install information and version locking information (like gemfile.lock)

If people don't rely on the file system organization as an API, npm and node's require() algorithm can trivially reorganize or even use different folder organization schemes on different operating systems if there were a good reason to do so.

Re: Node's nested node_modules approach is basically incompatible with Windows

#14
post #8

This definitely seems more like a Windows issue than a Node issue.

At first I agreed with you; but then I read one of the comments on github that swayed me.

"You are the one playing games - calling core parts of Windows like Explorer "3rd party tools", and suggesting that not supporting long paths is a bug. Microsoft have made it clear repeatedly that non-support for long paths is not a bug, and not something that will change.

A package manager creating paths that do not work with the majority of the software written for an OS, then claiming compatibility with the OS, is playing games, at your users expense."

At the end of the day Windows wont be the thing changing. haha.

Re: Node's nested node_modules approach is basically incompatible with Windows

#15
I am somewhat surprised at this topic. Of all the the issues Node.js has on windows, this seems like a fairly minor one. In my eyes, the biggest issue is simply this:

A lot of module writers don't care about Windows.

A big part of the node ecosystem is that there is a module for practically anything. While node core has great support for windows, and also helps aid developers in writing code that is windows compatible, from what I have seen, its not often followed.

Any number of simple things break modules on windows, using a '/' as a path separator instead of path.join, native modules, bash or shell scripts used in pre- or post-install scripts and probably a hundred other small ways to screw it up. Even trying to make your own module Windows compatible is often thwarted by a dependency that doesn't. Like Java's promise of "Write once, run anywhere", its a great idea, but hasn't completely panned out.

I really hope that as the node community matures, there can be more of a focus on making sure things work on Windows. Node is a great platform partially because of all those modules and getting them all working will make writing cross platform apps and services a heck of a lot easier.

Re: Node's nested node_modules approach is basically incompatible with Windows

#16
post #14
post #8

This definitely seems more like a Windows issue than a Node issue.

At first I agreed with you; but then I read one of the comments on github that swayed me. "You are the one playing games - calling core parts of Windows like Explorer "3rd party tools", and suggesting that not supporting long paths is a bug. Microsoft have made it clear repeatedly that non-support for long paths is not a bug, and not something that will change. A package manager creating paths that do not work with t…

I don't follow, should we then rename everything to cryptic, short characters like some people do in MongoDB (as someone suggested using n_m instead of node_modules)? If it works well in other platforms, it's an issue with that specific platform

Re: Node's nested node_modules approach is basically incompatible with Windows

#18
post #5
post #4

Was the recursive "every package include its dependencies" done just because it was easier? Or was it a conscious design choice.

Well at least there is no possibility of conflicts(unlike some other pm). The problem is people writing modules that have like 1 function while requiring 10 other modules that have just 1 function themself that require 10*10 other modules ... Of course it doesnt scale,and of course the package manager wont scale either... But it's not a NodeJS problem it is a NPM problem.NodeJS has other problems though(the lack of g…

> second one is nodejs dependency on what google does/doesnt do,since google maintains V8

Google maintaining V8 is one of the big reasons why Node has been successful. There is no way an independent Node.JS foundation would have the resources to optimize and test JS like the Chrome team. So far, the problems have been minor and V8 has done a decent job keeping up with upcoming JS standards.

So this isn't a problem.

> 1 function while requiring 10 other modules that have just 1 function themself that require 10*10 other modules

As of now NPM is working (in spite of these problems) because those modules are tiny js files. I agree it would be beneficial to have better package management.

In fact, that's exactly what's giving me problems right now. My browserify task breaks due to the NoYieldInGenerator error in parsing. The dependency (esprima parser) has been patched upstream, but now that has to make its way into 4 dependencies inside browserify, and then browserify has to update itself. My alternative is to make 5 forks just to solve this. :/

It would have been great if they all used a single shared lib, and even better if I could manually override dependency versions of my dependencies.

Re: Node's nested node_modules approach is basically incompatible with Windows

#19
post #17

[deleted]

Look, I totally get where you're coming from. I haven't used Windows (other than the modern.ie virtual machines for IE testing) in over 7 years now...

But a lot of devs use windows. Convincing them to switch has always been painful, and I stopped when I took a look at myself and asked why I cared so much (there are legitimate answers to that question, but my answers were not).

Where I work we use all three OS's. I personally use OS X, coupled with Linux VM's (where all my projects actually run). But the developers running Windows are just as productive as I am (if not more so, one die-hard Windows user is twice the developer I am, which is why he's a tech lead here). Yeah sometimes they have to deal with small annoyances when using *nix software on Windows which a lot of FOSS dev tools are, but hey, so do I on OS X, and I have on Debian as well.

Anyway, that's a long winded way of saying: that's just like, your opinion, man.

Re: Node's nested node_modules approach is basically incompatible with Windows

#20
post #17

[deleted]

> Or just don't develop on Windows. If you're developing using open source software, I honestly cannot comprehend why you would use Windows as your OS over something Unixy.

I'll give you one reason.

I work on Linux. But (as a consultant) many of clients are large Windows shops. Now it would be good if I can use the tools I am familiar with to get things done in primarily-Windows environments. Compatibility helps here.

Eventually, great Open Source tools existing on Windows draws more Windows users into Open Source. It is a net win, although I am not suggesting that anyone bend backwards to accommodate this.

Post reply on HN