The Incremental API aka "The Greedy File Watcher"

[johnnyreilly]

Heya!

Maintainer of the fork-ts-checker-webpack-plugin here. Under the covers, the plugin uses the TypeScript incremental API. Various issues have been raised when using this with the level of CPU usage on idle; i.e. it's high!

• CPU usage on idle? TypeStrong/fork-ts-checker-webpack-plugin#236
• React-scripts/fork-ts-checker-webpack-plugin constant high CPU usage when using material/icons facebook/create-react-app#6792

The marvellous @NeKJ did some digging and discovered that this could be remedied by setting an environment variable: TSC_WATCHFILE=UseFsEventsWithFallbackDynamicPolling.

This changes the behaviour of TypeScripts approach to file watching in a highly desirable fashion. You can see details of @NeKJ's investigation here:

TypeStrong/fork-ts-checker-webpack-plugin#256

As you'll see, this is a PR for fork-ts-checker-webpack-plugin that changes the TSC_WATCHFILE environment variable. If we merge it the users of fork-ts-checker-webpack-plugin should gain a better experience on the resources front.

However, it feels like this is potentially the wrong place to make this change. So before we merge and ship this I wanted to throw it back to you. If this affects users of fork-ts-checker-webpack-plugin then I'd hazard a guess other people who use the API are being burned too. Changing the default in fork-ts-checker-webpack-plugin won't help them.

Should the default file watching approach used by TypeScript be switched to TSC_WATCHFILE=UseFsEventsWithFallbackDynamicPolling or similar? So all users of this API have a better experience?

[DanielRosenwasser]

@sheetalkamat, thoughts? We did chat with VS Code a bit about this. Did they see improvements?

[johnnyreilly]

Hey @DanielRosenwasser / @sheetalkamat,

I was just wondering if there's been any discussion around this? I can certainly testify to lots of people who use create-react-app / fork-ts-checker-webpack-plugin logging issues related to this.

It would be great to know if this is something you're looking to address?

In the meantime (time permitting) I may blog about this to spread the word on the workaround that people can already use.

I've blogged about it in the meantime: https://blog.johnnyreilly.com/2019/05/typescript-and-high-cpu-usage-watch.html

Thanks for all your excellent work! ♥️🤗

[DanielRosenwasser]

I remember chatting with Sheetal about this in person, but it's been a couple of weeks. My understanding is that this really is something that can only be determined by the end-user because different setups can affect this (e.g. operating system, file system, remote file system, etc.)

Is that accurate @sheetalkamat? @RyanCavanaugh?

[johnnyreilly]

My understanding is that this really is something that can only be determined by the end-user because different setups can affect this (e.g. operating system, file system, remote file system, etc.)

Would it be possible for you to share the reasoning behind why the current default is the one used?

I ask as, to quote @NeKJ:

TS uses internally the fs.watch and fs.watchFile API functions of nodejs for their watch mode. The latter function is even not recommended by nodejs documentation for performance reasons, and urges to use fs.watch instead.

NodeJS doc:

Using fs.watch() is more efficient than fs.watchFile and fs.unwatchFile. fs.watch should be used instead of fs.watchFile and fs.unwatchFile when possible.

He suggests that the present default is a choice that node themselves advocate against using.

Also, there seems to a good number of people who are finding the present default problematic. Each day that passes seems to bring another user either voting up the existing issue or commenting 😄

Also, those people who have tried setting the environment variable to UseFsEventsWithFallbackDynamicPolling have come back with happy comments and upvotes 😁 - see facebook/create-react-app#6792 (comment)

It would great to understand the rationale for the current choice over one of the alternatives. Oh and just to add a bit more context, here's some investigation which @NeKJ performed:

I measured all of them, using $ top -b -n2 -d 30 -p <pid of node that runs service.js> which is a sample of 30 seconds, run 10+ seconds after the build has been completed and everything has settled.

Value	CPU usage on idle
TS default (TSC_WATCHFILE not set)	7.4%
UseFsEventsWithFallbackDynamicPolling	0.2%
UseFsEventsOnParentDirectory	0.2%
PriorityPollingInterval	6.2%
DynamicPriorityPolling	0.5%
UseFsEvents	0.2%

As we can see, the worst option of them all is the default behaviour of TS and next to it is the PriorityPollingInterval where it is a tad bit lower. DynamicPriority is at 0.5% and all rest at 0.2%. Therefore the UseFsEventsWithFallbackDynamicPolling is the best choice as it will in the best case choose UseFsEvents with 0.2% or in the worst use DynamicPriorityPolling which goes up to 0.5%.

These all are measurements on my own computer which is a PC running linux 64bit. To do this right,we will need measurements from other systems too (OSX, Windows etc).

[DanielRosenwasser]

So my only qualification here is that I wrote the initial implementation of --watch so I don't know as much as Sheetal (and probably you) on this. That said, we had the same choice 4 years back as well - watch or watchFile.

The problem that came up is that last part:

Using fs.watch() is more efficient than fs.watchFile and fs.unwatchFile. fs.watch should be used instead of fs.watchFile and fs.unwatchFile when possible.

To be more specific, you'll find this under the docs for watch:

The fs.watch API is not 100% consistent across platforms, and is unavailable in some situations.

So back then we just always went with watchFile. Not sure what, if anything, has changed since then, and I'm not familiar whether we try one and fall back to the other now.

[johnnyreilly]

I'm no expert either to be honest.

Obviously it's a real subset of people that have come back with positive feedback. It would be awesome if someone with more node knowledge in the watch space could share more information so it would be clear if making a change is unwise.

I wanted to make a bit of noise about this as I'm certainly hearing it in the various projects I'm involved with. 😄

I see what you mean about the caveats ... There is more detail:

https://nodejs.org/docs/latest/api/fs.html#fs_availability

This feature depends on the underlying operating system providing a way to be notified of filesystem changes.

On Linux systems, this uses inotify(7).
On BSD systems, this uses kqueue(2).
On macOS, this uses kqueue(2) for files and FSEvents for directories.
On SunOS systems (including Solaris and SmartOS), this uses event ports.
On Windows systems, this feature depends on ReadDirectoryChangesW.
On Aix systems, this feature depends on AHAFS, which must be enabled.

If the underlying functionality is not available for some reason, then fs.watch will not be able to function. For example, watching files or directories can be unreliable, and in some cases impossible, on network file systems (NFS, SMB, etc), or host file systems when using virtualization software such as Vagrant, Docker, etc.

It is still possible to use fs.watchFile(), which uses stat polling, but this method is slower and less reliable.

It sounds from this that most TypeScript users would probably have this available. If I read this right it sounds like it's only some *nix distros where it's not available?

Not quite sure what

If the underlying functionality is not available for some reason,

... means?

[mk0x9]

... means?

Few corner cases, like when you have docker container with webpack running and the source files are mounted in a volume. Or you have source files mounted via network share (NFS/SMB/WEBDAV).

[johnnyreilly]

Thanks @mk0x9 - that's helpful!

If these are as niche as it sounds I feel it's worth considering changing the default. People will always be able to change it if there's problems; and by the sounds of it that may be a problem that never arises anyway....

[sheetalkamat]

@johnnyreilly We use watchFile for the important files like files in your program and defer to watch to watch directories. This makes us more accurate in almost all cases and hence the decision.
watch has many consistency issues apart from accuracy issues. Eg on Linux there is limit on number of things you want watch using this. Given that this differs so much with environment we cant change the default as that would make us loose accuracy.
Because usage of watch needs much deeper understanding with compromises you are making on accuracy, we suggest that users make informed decision to use it instead. Making it default is asking everyone to know those details.

[johnnyreilly]

Thanks for the detailed explanation @sheetalkamat. So is it the plan that this behaviour won't be changed? I'm assuming the answer to that question is "yes" - but I want to be sure. 😁

Users of both fork-ts-checker-webpack-plugin and create-react-app app have expressed interest in this issue. @NeKJ has raised a PR which would change the default behaviour using the environment variable:

TypeStrong/fork-ts-checker-webpack-plugin#256

I haven't merged the PR as I thought it would make more sense for this change to be made in TypeScript itself. It sounds like that's not going to happen and for good reasons.

From what you've said, it sounds like you'd recommend not merging that PR. Is that fair?

It sounds like if we did we'd potentially solve one problem and create another. You may be saving us from making a mistake 🤗

I'm thinking about just documenting this clearly in order that people will know the configuration is possible. But here be dragons 🐉 and all that 😁

[sheetalkamat]

@johnnyreilly We document this in https://github.com/microsoft/TypeScript-Handbook/blob/master/pages/Configuring%20Watch.md#background
In general that page is for more advanced users, so to answer your question we wont be changing default behavior.

[OneCyrus]

if the default doesn't get changed, any chance we get this as a tsconfig option instead of just an env variable?

[johnnyreilly]

I really like @OneCyrus suggestion - the environment variable approach works but the developer experience isn't great; friction is high. I think making this part of tsconfig.json is a great idea!

Also, it being part of the tsconfig.json means this feature will be much more discoverable. At the moment the file watching approach is not well known

[OliverJAsh]

I hope I'm not adding noise to this issue, but I've discovered something odd which seems to relate to the choice of fs.watch/fs.watchFile.

When you git commit a change to a watched file, tsc's watch will be triggered and tsc will recompile. That's odd because git commit doesn't actually change the file.

This is concerning because I'm now wondering how many wasted recompiles are happening during advanced git operations, like rebasing.

I was curious whether this was a tsc thing or something deeper. It turns out this is the behaviour of fs.watch (which is what tsc --watch uses, IIUC):

const fs = require('fs');

fs.watch(__dirname + '/main.ts', () => {
    console.log('watch');
});

If you run that script and git commit a change to the watched file, you'll see watch logged.

OTOH, if you use fs.watchFile, you will not see this logged:

fs.watchFile(__dirname + '/main.ts', () => {
    console.log('watchFile');
});

Is this a known bug? Is there a recommended workaround?

[mk0x9]

Sounds like an issue in the node; IIRC fs.watch* is a platform dependent. Would be interesting to test this issue on the different platforms and compare fs.Stats objects in the callback.

[deftomat]

I would rather think about introducing it in tsconfig.json again. Looks like this issue really depends on specific configuration, so using some flag which every member of team pull from repo is probably not a good experience either.

[jasonwilliams]

@sheetalkamat Is it possible to re-open this issue with an option to set the tsconfig.json?
Having all devs set an environment variable isn't the nicest experience, especially when other watch tools let you choose via config (like watchpack or webpack watch etc).
This will help with discoverability also.

[makranelhoucine]

@johnnyreilly thank you so much you saved my day

I use create react app, this line helped me to fix CPU 100% problm
"start": "cross-env TSC_WATCHFILE=UseFsEventsWithFallbackDynamicPolling react-scripts start",

[LucasMatuszewski]

@DanielRosenwasser / @sheetalkamat - please, would you consider @OneCyrus suggestion? Or/and could you add some information about this issue to documentation and help to reach more developers?

Finding that my CPU load is connected with TS and solving it took many hours of my life + my company IT Support time. Many people have the same problem. It still occurs with TypeScript 3.7.2

[nicoburns]

@DanielRosenwasser @sheetalkamat Would you consider switching to Chokidar? It's a battle tested solution used very widely throughout the Node ecosystem. We're in the process of switching to TypeScript at work, and the watch processes are taking ~45% cpu each on my 2015 MacBook Pro. As far as I am aware Node's built-in file watching capabilities are generally considered broken and not suitable for production use.

[jasonwilliams]

As far as I am aware Node's built-in file watching capabilities are generally considered broken and not suitable for production use.

@nicoburns I'm not sure this is true anymore. Webpack 5 have just moved from Chokidar to Node's native fs system

The watcher used by webpack was refactored. It was previously using chokidar and the native dependency fsevents (only on OSX). Now it's only based on native Node.js fs
https://webpack.js.org/blog/2020-10-10-webpack-5-release/#new-watching

I don't think they would have made that change if it was broken.