fs.watch() appears to be very broken on Windows VMs

[cowboy]

TL;DR: In a Windows 7 VM (I've seen this same behavior in a Windows XP VM) false positives are fired on fs.watch() watchers when once-watched files are recreated, and changing files doesn't trigger a change event. In addition, an Unknown system errno 995 exception is thrown. A lot. But everything appears to work just fine in OS X 10.7.

Anyways... this script (included at the bottom of this post) watches the files 1, 2, and 3 for changes (only if they exist). Once any of them has changed, they are all un-watched, and then any that still exist are re-watched. I've tested this in Node.js v0.6.11.

In the following transcripts, <<WAITING>> means the script had paused, waiting for something to happen. Any other << ... >> describe some action I took.

In OS X 10.7, this behaves as you'd expect. When files are deleted or changed, a change event is fired, When files are recreated after being deleted, no change event is fired. After all files have been deleted, the process doesn't exit (bug?)

(The following assumes that files 1, 2 and 3 already exist)

$ node watch_test.js 
=== START ===
watchFile 1
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I modify file "1" and...>>
event change 1
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I modify file "2" and...>>
event change 2
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I delete file "1" and...>>
event delete 1
unWatchFile 1
=== END ===
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I recreate file "1", nothing happens>>
<<In another window I delete file "2" and...>>
event delete 2
unWatchFile 2
=== END ===
unWatchFile 3
=== START ===
watchFile 1
watchFile 3
<<WAITING>>
<<In another window I recreate file "2", nothing happens>>
<<In another window I delete files "1" "2" and "3" and...>>
event delete 1
unWatchFile 1
=== END ===
unWatchFile 3
=== START ===
<<WAITING>>

In Windows 7 (I've seen this same behavior in Windows XP) false positives are fired on watchers when once-watched files are recreated, and changing files doesn't trigger a change event. In addition, an Unknown system errno 995 exception is thrown. A lot.

(The following assumes that files 1, 2 and 3 already exist)

X:> node watch_test.js

=== START ===
watchFile 1
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I modify file "1" and "2", nothing happens>>
<<In another window I delete file "1" and...>>
event delete 1
unWatchFile 1
=== END ===
unWatchFile 2
unWatchFile 3
ERROR Unknown system errno 995
ERROR Unknown system errno 995
=== START ===
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I recreate file "1" and...>>
event change 2
=== END ===
unWatchFile 2
unWatchFile 3
ERROR Unknown system errno 995
=== START ===
watchFile 1
watchFile 2
watchFile 3
<<WAITING>>
<<In another window I delete file "2" and...>>
event change 1
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
ERROR Unknown system errno 995
ERROR Unknown system errno 995
=== START ===
watchFile 1
watchFile 3
<<WAITING>>
<<In another window I recreate file "2" and...>>
event change 1
=== END ===
unWatchFile 1
unWatchFile 3
ERROR Unknown system errno 995
=== START ===
watchFile 1
watchFile 2
watchFile 3
<< WAITING>>
<<In another window I delete files "1" "2" and "3" and...>>
event delete 1
unWatchFile 1
=== END ===
unWatchFile 2
unWatchFile 3
=== START ===

X:> 

And here's the script used in these examples:

var fs = require('fs');
var path = require('path');

// Watched files.
var watchedFiles = {};

// Watch a file.
function watchFile(filepath) {
  console.log('watchFile', filepath);
  // Watch this file for changes.
  watchedFiles[filepath] = fs.watch(filepath, function(event) {
    // Was the file deleted?
    var deleted = !path.existsSync(filepath);
    console.log('event', deleted ? 'delete' : event, filepath);
    // If file was deleted, stop watching file.
    if (deleted) { unWatchFile(filepath); }
    end();
  }).on('error', function(err) {
    // There should never be any errors.
    console.log('ERROR', err.code);
  });
}

// Unwatch a file.
function unWatchFile(filepath) {
  console.log('unWatchFile', filepath);
  // Close watcher.
  watchedFiles[filepath].close();
  // Remove file from index.
  delete watchedFiles[filepath];
}

// Start watching the files 1, 2 and 3, but only if they exist.
function start() {
  console.log('=== START ===');
  ['1', '2', '3'].forEach(function(n) {
    if (path.existsSync(n)) {
      watchFile(n);
    }
  });
}

// Unwatch any watched files, and start over again in 100ms.
function end() {
  console.log('=== END ===');
  // Unwatch all files.
  Object.keys(watchedFiles).forEach(unWatchFile);
  // Watch all files.
  setTimeout(start, 100);
}

// Watch all files.
start();

[cowboy]

Note: This was in Windows 7 / XP Parallels VM images running on OS X.

[Jorge-Gonzalez]

I tryed your test in XP and it works well:

watch_test.js
=== START ===
watchFile 1
watchFile 2
watchFile 3
((change file 3))
event change 3
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
((change file 1))
event change 1
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
((change file 3))
event change 3
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
((change file 3))
event change 3
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
((change file 2))
event change 2
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
((delete file 2))
event delete 2
unWatchFile 2
=== END ===
unWatchFile 1
unWatchFile 3
=== START ===
watchFile 1
watchFile 3
((create file 2 and no event was triggered))
((delete file 3))
event delete 3
unWatchFile 3
=== END ===
unWatchFile 1
=== START ===
((delete event restart watching both files))
watchFile 1
watchFile 2

[naxhh]

All go ok

Tested on windows 7 32bits

I edited the 3 of them and deleted the last.

C:\xampp\htdocs\bugtrack\nodejs\2839>node test.js
=== START ===
watchFile 1
watchFile 2
watchFile 3
event change 1
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
event change 2
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
event change 3
=== END ===
unWatchFile 1
unWatchFile 2
unWatchFile 3
=== START ===
watchFile 1
watchFile 2
watchFile 3
event delete 3
unWatchFile 3
=== END ===
unWatchFile 1
unWatchFile 2
=== START ===
watchFile 1
watchFile 2

[unbalanced]

Hi Ben!

i setup a test on Virtualbox ubuntu guest command line.
i put the files on a shared folder and tested 2 cases:

1. updating / deleting files from the ubuntu guest. resulted in fs.watch working perfectly fine every time.
2. updating / deleting files from the windows 7 host. resulted in fs.watch keeps on waiting for eternity.

i also investigated further and got to this issue from 3 months ago - remy/nodemon#75
do you know it?

Remy documented this problem and seem to have some sort of a workaround - https://github.com/remy/nodemon#help-my-changes-arent-being-detected

[cowboy]

Maybe parts of Node.js just don't work with Windows 7 running in a VM.

[terrywh]

there is a WIN32 API "SHChangeNotifyRegister". Can it be used ?
http://msdn.microsoft.com/en-us/library/bb762120(VS.85).aspx

[jasnell]

@orangemocha ... thoughts on this one?

[orangemocha]

Cloud not reproduce this issue, so I am closing it.

If this is still an issue feel free to reopen, ideally attaching an updated test case.