"Error: EMFILE: too many open files, open.." on Windows

[TimNZ]

That's pretty much the issue.
Works fine on OS X

Have googled and can't find any way to increase Windows open file limit.
Is now also happening on OS X 'ENFILE: file table overflow'

As far as I can tell this module always works down to an individual file level in async.

I've worked around this by changing the app structure (it will be the ~8000 files in node_modules) and doing a shell cp instead.

Is it something you need to look at handling via something like graceful-fs?

[qiaolb]

I have a same problem.

[kevlened]

This is tricky, because node-glob no longer uses graceful-fs and just including it before node-glob won't work because graceful-fs has stopped monkey-patching fs. The graceful-fs docs say not to patch fs in a library (like this one), so it'd have to be added as an option. The problem is, the option would have to be checked before any other module loaded fs. An environment variable is the only reasonable way I can think to do that, which seems sloppy.

If you're running into this issue, you'll need to monkey-patch fs yourself. Here's how:

// At the top of your webpack config, insert this
var fs = require('fs');
var gracefulFs = require('graceful-fs');
gracefulFs.gracefulify(fs);

I'd add this info to the docs.

[TimNZ]

The irony of me suggesting graceful-fs based on comments I real elsewhere for different issues and then never looking at the module myself and seeing that it would help :)

[kevlened]

No worries! Others had the same issue, so I'm glad it's in the docs now.

[keyiis]

@kevlened i have same issues,i already monkey-patch fs,below is my code:

var fs = require('fs');
var gracefulFs = require('graceful-fs');
gracefulFs.gracefulify(fs);
const webpackMerge = require('webpack-merge');
const ExtractTextPlugin = require('extract-text-webpack-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');
const commonConfig = require('./webpack.common.js');
const helpers = require('./helpers');

module.exports = webpackMerge(commonConfig, {
  ......
plugins: [
    new CopyWebpackPlugin([
      { from: helpers.root('node_modules/ckeditor/ckeditor.js'), to: 'node_modules/ckeditor' },
      { from: helpers.root('node_modules/ckeditor/config.js'), to: 'node_modules/ckeditor' },
      { from: helpers.root('node_modules/ckeditor/styles.js'), to: 'node_modules/ckeditor' },
      { from: helpers.root('node_modules/ckeditor/contents.css'), to: 'node_modules/ckeditor' },
      { from: helpers.root('node_modules/ckeditor/skins/moono'), to: 'node_modules/ckeditor/skins/moono' },
      { from: helpers.root('node_modules/ckeditor/lang/zh-cn.js'), to: 'node_modules/ckeditor/lang' },
      { from: helpers.root('node_modules/ckeditor/plugins'), to: 'node_modules/ckeditor/plugins' },
      { from: helpers.root('node_modules/ckeditor/adapters'), to: 'node_modules/ckeditor/adapters' },
    ])
  ]
}

error msg:

95% emittingfs.js:634
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: EMFILE: too many open files, open 'D:\work\nodejs\angular\angular2-electron\node_modules\ckeditor\plugins\sourcedialog\lang\ar.js'
    at Error (native)
    at Object.fs.openSync (fs.js:634:18)
    at Object.fs.readFileSync (fs.js:502:33)
    at Object.source (D:\work\nodejs\angular\angular2-electron\node_modules\.npminstall\copy-webpack-plugin\3.0.1\copy-webpack-plugin\dist\writeFileToAssets.js:45:31)
    at Compiler.writeOut (D:\work\nodejs\angular\angular2-electron\node_modules\.npminstall\webpack\2.1.0-beta.15\webpack\lib\Compiler.js:290:26)
    at D:\work\nodejs\angular\angular2-electron\node_modules\.npminstall\mkdirp\0.5.1\mkdirp\index.js:48:26
    at FSReqWrap.oncomplete (fs.js:117:15)

[craighutch]

@kevlened I'm seeing the same issue that @keyiis is reporting. Even after applying this patch, the error still occurs when trying to copy the "ckeditor" folder.

[jgrosso]

+1

[keyiis]

@kb3eua I use gulp to copy file.

[craighutch]

@keyiis I didn't want to introduce another task runner. I was able to get it working by upgrading to the latest version of this plugin and using the following configuration to ignore certain files:

 new CopyWebpackPlugin([{
    from: './node_modules/ckeditor',
    to: './dist/scripts/ckeditor',
    ignore: [
        '**/samples/**',
        '**/lang/!(en.js)'
    ]
}]);

[tallaxes]

Having the same error on Windows with many files (Angular 2 CLI Webpack ng build, bower_components in assets; yeah, I know). graceful-fs does not help, most likely because it does not patch readFileSync/openSync codepath. Webpack's Compiler.js emitFiles uses unconstrained async.ForEach over assets, hitting the Node limit on open files (2048 on Windows, due to use of CRT).

require("async").forEach(Object.keys(compilation.assets), function(file, callback) {

Workaround: I don't understand webpack/copy-webpack-plugin/graceful-fs well enough to suggest the right way of fixing this, but constraining webpack to a reasonable number of async operations (using forEachLimit) works fine:

require("async").forEachLimit(Object.keys(compilation.assets), 2000,
  require("async").ensureAsync(function(file, callback) {

EDIT: Wrapped callback with async.ensureAsync - fixes stack overflow for when assets are served from memory

[id0Sch]

having this issue on OSX (patch applied and all)