Error when using xarray's built in interp function with dask chunking

[rdPatmore]

This is a copied issue posted on stack overflow: https://stackoverflow.com/questions/57572433/error-when-using-xarrays-built-in-interp-function-with-dask

I am trying to use xarray's interp function together with the chunk function, which is a dask array feature.

Taking the example under 'Example' at http://xarray.pydata.org/en/stable/interpolation.html I am able to use the interp function just fine. The simplified code I use to do this is:

at = xr.tutorial.open_dataset('air_temperature').isel(time=0)

x = np.linspace(240, 300, 100)
z = np.linspace(20, 70, 100)
lat = xr.DataArray(z, dims=['z'], coords={'z': z})
lon = xr.DataArray((x[:, np.newaxis]-270)/np.cos(z*np.pi/180)+270,
                    dims=['x', 'z'], coords={'x': x, 'z': z})

dsi = at.interp(lon=lon, lat=lat)`

The problem arises when I try to combine this interp function with xarray chunk by modifying the above code in the following way:

at = xr.tutorial.open_dataset('air_temperature').isel(time=0)
at = at.chunk({'lat':10})        # added chunking inducing a dask array

x = np.linspace(240, 300, 100)
z = np.linspace(20, 70, 100)
lat = xr.DataArray(z, dims=['z'], coords={'z': z})
lon = xr.DataArray((x[:, np.newaxis]-270)/np.cos(z*np.pi/180)+270,
                    dims=['x', 'z'], coords={'x': x, 'z': z})

dsi = at.interp(lon=lon, lat=lat)

The code fails at line

dsi = at.interp(lon=lon, lat=lat)

The error I get is:

ValueError: Can't specify drop_axis and new_axis together

I think what is happening is that the interp function is replacing a coordinate and dask cannot deal with this. Is this a bug that needs fixing within xarray or is there something that I have done wrong?

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.6 |Anaconda custom (64-bit)| (default, Oct 9 2018, 12:34:16)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 3.10.0-957.5.1.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8
libhdf5: 1.8.18
libnetcdf: 4.4.1.1

xarray: 0.12.3
pandas: 0.25.0
numpy: 1.16.4
scipy: 1.3.0
netCDF4: 1.3.1
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: None
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 0.13.0
distributed: None
matplotlib: 2.2.2
cartopy: None
seaborn: 0.7.1
numbagg: None
setuptools: 40.8.0
pip: 9.0.1
conda: 4.7.11
pytest: 3.0.5
IPython: 5.1.0
sphinx: 1.5.1

[jhamman]

I think this would work with a newer version of dask. We currently say dask >= 0.16 (you are at 0.13), but if you want all the features dask/xarray offer, I'd strongly recommend getting the latest release (dask == 2.4).

[crusaderky]

The current situation is that optional dependencies older than the very latest version available may work, but are completely untested: #3222

[jhamman]

Right. But to be clear, dask==0.13 is explicitly not supported so there should be no expectation that things will work as advertised.

[rdPatmore]

Thanks both for your response on this. I didn't realise my version of dask was so outdated. I have now updated to the latest version (2.4.0) and the error outline above no longer seems to be an issue.