webpack watch screwed up (v13.3.4 => v13.4.3)

[jelmd]

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Node v18.16.0, Linux (Ubuntu 20.04), nextjs v13.4.3

Which area(s) of Next.js are affected? (leave empty if unsure)

Turbopack (--turbo)

Link to the code that reproduces this issue or a replay of the bug

https://nextjs.org/learn/basics/create-nextjs-app/setup

To Reproduce

1. create a nextjs app
2. make sure package.json dependencies contains "next": "13.4.3", e.g.

{
	"name": "foobar",
	"description": "Test",
	"repository": "https://github.com/x/y",
	"bugs": {
		"url": "https://github.com/x/y/issues"
	},
	"author": "Foo Bar <foobar@do.main>",
	"contributors": [],
	"version": "0.1.0",
	"private": true,
	"scripts": {
		"db": "NODE_OPTIONS='--no-warnings --loader ts-node/esm' nodemon --watch scripts scripts/db.ts",
		"dev": "next dev",
		"build": "next build",
		"start": "next start",
		"lint": "next lint",
		"clean": "rm -rf .next",
		"realclean": "rm -rf .next node_modules package-lock.json"
	},
	"dependencies": {
		"@next-auth/prisma-adapter": "^1.0.6",
		"@prisma/client": "^4.14.1",
		"next": "13.4.3",
		"nodemailer": "^6.9.2",
		"react": "18.2.0",
		"react-dom": "18.2.0"
	},
	"devDependencies": {
		"@styled-icons/material": "^10.47.0",
		"@styled-icons/fa-solid": "^10.47.0",
		"@types/busboy": "^1.5.0",
		"@types/node": "20.2.3",
		"@types/nodemailer": "^6.4.8",
		"@types/react": "18.2.6",
		"@types/react-dom": "18.2.4",
		"eslint": "8.41.0",
		"eslint-config-next": "13.4.3",
		"prisma": "^4.14.1",
		"typescript": "5.0.4"
	}
}

3. add watchOptions for webpack to your next.config.js, e.g. the one used looks like this:

/** @type {import('next').NextConfig} */
const nextConfig = {
	reactStrictMode: true,
	webpack: function (config, options) {
		if (!config.watchOptions) {
			config.watchOptions = {
				aggregateTimeout: 5,
				ignored: [ '**/node_modules/**', '**/.git/**', '**/.next/**' ]
			};
		}
		return config;
	},
	experimental: {
		/* appDir: true, */
		instrumentationHook: true
	}
};
/*
process.on('unhandledRejection', error => {
	console.log('unhandledRejection', error);
});
*/
module.exports = nextConfig

4. make sure, user watch settings has a reasonable value, i.e. 4-8K is even more than that (however, this seems to be the default on many systems)
   cat /proc/sys/fs/inotify/max_user_watches. If some one already screwed it up, use e.g. sysctl fs.inotify.max_user_watches=4096 to get it back to an acceptable value.
5. run npm run dev

Describe the Bug

300+ K of error messages like this:

Watchpack Error (watcher): Error: ENOSPC: System limit for number of file watchers reached, watch '~/work/nextapp/pages'
Watchpack Error (watcher): Error: ENOSPC: System limit for number of file watchers reached, watch '~/work/nextapp'
Watchpack Error (watcher): Error: ENOSPC: System limit for number of file watchers reached, watch '~/work'
Watchpack Error (watcher): Error: ENOSPC: System limit for number of file watchers reached, watch '~'
Watchpack Error (watcher): Error: ENOSPC: System limit for number of file watchers reached, watch '/home'
...

NOTE: The current working directory is ~/work/nextapp/ !!! Why does it want to monitor the whole user filesystem?

Expected Behavior

For nextjs v13.3.4 everything worked with the given configs as expected, i.e. no problems at all when running npm run dev (even if VScode is running side-by-side).

IMHO nextjs should pre-configure webpack etc. to monitor at most the current working directory and exclude by default ./node_modules (and possibly ./.next ?), because it is just a huge waste of resources for no (or not justifiable reason).

Which browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

[jelmd]

Just found out, this even screws up VScode!!!

E.g. if one does a code . within the app root dir, code seems to more or less ignore ~/.config/Code/User/settings.json::files.watcherExclude and creates ~8K watches instead of ~120!

Going back to nextjs 13.3.4 watches get back to ~ 30+-10 when running npm run dev.

[laneme]

Same exact issue. Happened after upgrading from 13.3.4 to 13.4.9.

[caomicc]

Also having this issue, any resolutions?

[tianjianchn]

Similar issue, but my log is

Watchpack Error (watcher): Error: EMFILE: too many open files, watch '/home/xx/Codes/foo'
Watchpack Error (watcher): Error: EMFILE: too many open files, watch '/home/xx/Codes'
Watchpack Error (watcher): Error: EMFILE: too many open files, watch '/home/xx'
Watchpack Error (watcher): Error: EMFILE: too many open files, watch '/home'

nextjs version 13.5.6, linux.

Changing webpack config.watchOptions.ignored is meanless.

It's watchpack error, which is called by nextjs to watch project dir. watchpack use fs.watch() to watch directory, which calls inotify in linux. So we could check those related settings:

ulimit -n
cat /proc/sys/fs/inotify/max_user_instances
cat /proc/sys/fs/inotify/max_user_watches

For my case, ulimit -n is big enough(1 million), but max_user_instances is 128, which is too small. Increased it by echo 256 | sudo tee /proc/sys/fs/inotify/max_user_instances, all is OK now.

Hopes that help!

[ezeamin]

Hi! I'm having a very similar issue when implementing Next.js 14 with Node 21 and pnpm inside a monorepo handled by turborepo. I've already created an issue there but they sent me back here. My error is:

Error: EMFILE: too many open files, open 'C:\Users\Amin\Documents\Codigo\PJ\sistemas-judiciales-monorepo\apps\backoffice-psae\node_modules\ui\node_modules\hooks\node_modules\react-hook-form\dist\utils\isWeb.d.ts'

The issue is at this link. Any help is welcomed. Thanks!!!

[ryan-williams]

I upgraded a Next 12 project to 13.5, dev and build started failing with:

"TypeError: config.watchOptions.ignored.filter is not a function or its return value is not iterable""

No hits for that on the internet, but this was close, and downgrading to 13.3.4 "fixed" it. As noted above, problems seem to appear in ≥13.4.x.

[dnaranjo89]

In my case I was using next-transpile-modules which is deprecated. Transpiling modules is now natively supported by nextjs.
Removing this package fixed the issue.

[talhaqaisar676]

in my case i added the webpack watchOption in webpack configurations in next.config.js
webpack: (config, {dev}) => { if (dev) { watchOptions: { poll: true } return config } return config }

You can set the poll to any interval according to your environment or let it be true which is the default interval

[trautonen]

in my case i added the webpack watchOption in webpack configurations in next.config.js webpack: (config, {dev}) => { if (dev) { watchOptions: { poll: true } return config } return config }

You can set the poll to any interval according to your environment or let it be true which is the default interval

Just a small note here that the provided config is not actually working as it returns the config as is. You can find the webpack poll configuration from here https://webpack.js.org/configuration/watch/#watchoptionspoll and change the above a little or add any other feature the watch options support.

  webpack: (config, { dev }) => {
    if (dev) {
      config.watchOptions = {
        poll: true,
      }
      return config
    }
    return config
  }

The fixed version changes the config before returning it in dev mode.