Skip to content

to_zarr with append_dim behavior changed in 0.16.0 release #4261

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
maximemorariu opened this issue Jul 23, 2020 · 4 comments
Closed

to_zarr with append_dim behavior changed in 0.16.0 release #4261

maximemorariu opened this issue Jul 23, 2020 · 4 comments
Labels
topic-zarr Related to zarr storage library

Comments

@maximemorariu
Copy link

What happened:
In version 0.15.1, calling to_zarr on a Dataset with a given append_dim would create a new zarr if it did not already exist. In version 0.16.0, this is not the case anymore: the call fails if the zarr does not exist.

Minimal Complete Verifiable Example:

import xarray as xr

a = xr.DataArray([1, 2], {"t": [1, 2]}, ("t",))
ds = xr.Dataset({"v": a})
ds.to_zarr("CHOOSE_PATH", append_dim="t")

Environment:

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.7.7 (default, May 7 2020, 21:25:33)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 4.18.0-177.el8.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8
libhdf5: None
libnetcdf: None

xarray: 0.16.0
pandas: 1.0.5
numpy: 1.18.1
scipy: 1.5.0
netCDF4: None
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: 2.3.2
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.14.0
distributed: 2.14.0
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.2.0.post20200714
pip: 20.1.1
conda: None
pytest: 5.4.3
IPython: None
sphinx: None

@dcherian dcherian added the topic-zarr Related to zarr storage library label Jul 23, 2020
@shoyer
Copy link
Member

shoyer commented Jul 25, 2020

Interesting, I'm not quite sure how this happened! We probably didn't have test coverage.

I agree, it is convenient to be able to create a file with appending -- otherwise the first iteration of a loop needs special logic.

@maximemorariu
Copy link
Author

So just to know, should we hope for a fix in the next release? Many thanks

@shoyer
Copy link
Member

shoyer commented Aug 14, 2020

We would welcome a pull request to fix this, but I don't think anyone is currently working on it

luoq added a commit to luoq/xarray that referenced this issue Oct 24, 2020
@rabernat
Copy link
Contributor

Fixed by #4035.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-zarr Related to zarr storage library
Projects
None yet
Development

No branches or pull requests

4 participants