avoid race condition during chunk write

When the chunk file is first removed before the new version
is moved into place, racing reads may encounter a missing chunk.

Using rename() or replace() without remove() avoids the issue
on Posix-Systems as the methods are atomic. The fallback of
remove() -> rename() is included for Windows pre Python 3.3.

Fixes zarr-developers#263

Author: weatherfrog

zarr/storage.py

@@ -772,9 +772,18 @@ def __setitem__(self, key, value):
                 f.write(value)
 
             # move temporary file into place
-            if os.path.exists(file_path):
-                os.remove(file_path)
-            os.rename(temp_path, file_path)
+            if hasattr(os, 'replace'):
+                # Python >= 3.3 has replace() which can replace existing
+                # files on all platforms.
+                os.replace(temp_path, file_path)
+            else:
+                # In Python < 3.3 use rename().
+                if os.name == 'nt' and os.path.exists(file_path):
+                    # On windows, rename() can't overwrite files. So
+                    # the file is removed first.
+                    os.remove(file_path)
+
+                os.rename(temp_path, file_path)
 
         finally:
             # clean up if temp file still exists for whatever reason