Node's nested node_modules approach is basically incompatible with Windows

[jonrimmer]

Node needs an alternative approach to endless, recursively nested node_modules folders on Windows. Most Windows tools, utilities and shells cannot handle file and folder paths longer than 260 characters at most. This limit is easily exceeded, and once it is, install scripts start breaking and node_modules folders can no longer be deleted using conventional methods.

This will only get worse as Node packages get more complicated, with bigger and deeper dependency hierarchies. There should be a way to install and run packages that does not use file system recursion to represent the dependency hierarchy.

[Mithgol]

Suggestion:   to mitigate the above mentioned problem, make Node.js version 1.0 (and the corresponding npm) use n_m (3 characters) instead of node_modules (12 characters) by default. This change is expected to double the possible hierarchy depth for modules with names containing nine letters or less.

If a backwards compatibility is necessary, node_modules could still be supported (though not as default). For example,

• require('modulename') could look in node_modules if the module is not found in n_m,
• npm update modulename could kill node_modules/modulename (if it exists) and install n_m/modulename and dependencies.

[OrangeDog]

That doesn't solve the problem, just gives you slightly more breathing room.

The only real solution is going to have all modules at a single level, requiring each other, instead of all having private copies of their dependencies. This would either have to be versioned, and/or module maintainers need to be more careful about breaking changes.
Most package managers work this way, and seem to be getting along nicely.

[indutny]

As an idea: node_modules/module@version

[Mithgol]

The module@version solution eliminates the problem much better than n_m, but not without introducing a bunch of other problems.

Example: if somemodulename@1.2.3 and somemodulename@4.5.6 are both present, what should npm update somemodulename do?

In the same example, what should npm uninstall somemodulename do?

[randunel]

@Mithgol it should probably install somemodule@la.te.st, alongside the others. But npm purge or npm clean should probably be added.

[OrangeDog]

@Mithgol if the latest version is 4.5.6 do nothing, otherwise add the latest version. If you can be clever and determine that a previous version is no longer needed then remove it, otherwise don't worry about it.

@randunel somemodule@latest would be far more sensible if there's an actual need for it, though for backwards-compatibility, I'd guess somemodule would symlink to the latest anyway.

[jdalton]

👍, devs are running into this with individual lodash packages too.
See lodash/lodash#501, gulpjs/gulp-util#23, & twitter/theporchrat.

There looks to be the start of addressing the issue here, but it may need to be used in other methods.
Related to #6891.

[jzaefferer]

Some docs for the code @jdalton linked to: http://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx#maxpath

[Mithgol]

I guess Node's developers would have to rewrite both require() and npm to use \\?\… paths. If it's possible, 256-character limitation becomes void — even without resorting to n_m or @version.

(Maybe not exactly require() and npm, but rather the entire underlying fs module, also process.cwd(), etc.)

[jonrimmer]

Even if all of Node and NPM are rewritten to use long paths, so long as NPM supports scripts there is a danger of them randomly blowing up when run on Windows due to finding themselves inside a very deep hierarchy then trying to run utilities that don't support long paths.

[Mithgol]

@jonrimmer Many third-party scripts for NPM (such as Grunt, or Mocha, or node-pre-gyp) are Node.js scripts and thus they are to become automagically fixed when Node.js core modules (such as fs and path) and path-related methods (such as process.cwd()) all start using \\?\… unilaterally.

The other utilities would indeed break and thus the community would have to replace them or demand upgrades. (A similar ruckus once happened when Node.js started supporting Windows and many UN*X-only tools and CLI scripts experienced a lot of incompatibility-related problems. There was much suffering; however, the community eventually fixed the problems or developed some workarounds, sometimes as simple as using path.join instead of a former + '/' +. Or using os.EOL instead of a former \n.)

[langri-sha]

Hello everyone! I was unpleasantly surprised when I found out our module was hitting an unexpected wall in Windows host environments. If anyone needs a quick remedy:

• You can delete the Windows paths by slicing out a branch of the Node dependency tree and moving it to a shorter path in the host environment.
• If you are developing on a Windows host, but using a Unix environment in production, you can use Vagrant to create a virtual machine, and run you application in the guest environment.
  If you would like to keep your sources and Node modules synced with the Windows host, you can install lodash in the guest environment and use npm link to resolve it as a module dependency for your application.