Error when using xarray where that results in zero dimensions.

[imcslatte]

Xarray 0.17.0

Executing the following code:

import xarray as xray

ncurl='http://tds.coaps.fsu.edu/thredds/dodsC/samos/data/intermediate/ZMFR/2021/ZMFR_20210303v20001.nc'
dat =xray.open_dataset(ncurl)
#dat=dat.where(dat.lat>-90.0,drop=True)
dat=dat.where(dat.lat>-11.0,drop=True)
dat=dat.where(dat.lat<30.0,drop=True)
dat=dat.where(dat.lon>91.0,drop=True)
dat=dat.where(dat.lon<139.0,drop=True)
print(dat)

Results in the error:

Traceback (most recent call last):
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 98, in _getitem
    array = getitem(original_array, key)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/backends/common.py", line 53, in robust_getitem
    return array[key]
  File "src/netCDF4/_netCDF4.pyx", line 4397, in netCDF4._netCDF4.Variable.__getitem__
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/netCDF4/utils.py", line 467, in _out_array_shape
    c = count[..., i].ravel()[0] # All elements should be identical.
IndexError: index 0 is out of bounds for axis 0 with size 0

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./test_xarray_error.py", line 7, in <module>
    dat=dat.where(dat.lat>-11.0,drop=True)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/common.py", line 1273, in where
    return ops.where_method(self, cond, other)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/ops.py", line 195, in where_method
    return apply_ufunc(
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 1116, in apply_ufunc
    return apply_dataset_vfunc(
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 428, in apply_dataset_vfunc
    result_vars = apply_dict_of_variables_vfunc(
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 373, in apply_dict_of_variables_vfunc
    result_vars[name] = func(*variable_args)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 628, in apply_variable_ufunc
    input_data = [
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 629, in <listcomp>
    broadcast_compat_data(arg, broadcast_dims, core_dims)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/computation.py", line 542, in broadcast_compat_data
    data = variable.data
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/variable.py", line 374, in data
    return self.values
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/variable.py", line 554, in values
    return _as_array_or_item(self._data)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/variable.py", line 287, in _as_array_or_item
    data = np.asarray(data)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 693, in __array__
    self._ensure_cached()
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 690, in _ensure_cached
    self.array = NumpyIndexingAdapter(np.asarray(self.array))
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 663, in __array__
    return np.asarray(self.array, dtype=dtype)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/numpy/core/_asarray.py", line 102, in asarray
    return array(a, dtype, copy=False, order=order)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 568, in __array__
    return np.asarray(array[self.key], dtype=None)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/coding/strings.py", line 236, in __getitem__
    return _numpy_char_to_bytes(self.array[key])
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/coding/strings.py", line 193, in _numpy_char_to_bytes
    arr = np.array(arr, copy=False, order="C")
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 568, in __array__
    return np.asarray(array[self.key], dtype=None)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 85, in __getitem__
    return indexing.explicit_indexing_adapter(
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/core/indexing.py", line 853, in explicit_indexing_adapter
    result = raw_indexing_method(raw_key.tuple)
  File "/home/hunter/miniconda3/lib/python3.8/site-packages/xarray/backends/netCDF4_.py", line 108, in _getitem
    raise IndexError(msg)
IndexError: The indexing operation you are attempting to perform is not valid on netCDF4.Variable object. Try loading your data into memory first by calling .load().

It seems to be related to the fact that:
dat=dat.where(dat.lat>-11.0,drop=True)
results in a zero dimension (time).

However, if I uncomment one line and run:

import xarray as xray


ncurl='http://tds.coaps.fsu.edu/thredds/dodsC/samos/data/intermediate/ZMFR/2021/ZMFR_20210303v20001.nc'
dat =xray.open_dataset(ncurl)
dat=dat.where(dat.lat>-90.0,drop=True)
dat=dat.where(dat.lat>-11.0,drop=True)
dat=dat.where(dat.lat<30.0,drop=True)
dat=dat.where(dat.lon>91.0,drop=True)
dat=dat.where(dat.lon<139.0,drop=True)
print(dat)

There is no error, even though the line with dat=dat.where(dat.lat>-11.0,drop=True) still results in a zero dimension.

Perhaps this is not a bug, but maybe I am misunderstanding how where() works in xarray. But it seems inconsistent.

Eli Hunter
Rutgers University

[max-sixty]

Hi @imcslatte — after following the suggestion to call .load(), the example works for me.

If that's not sufficient — e.g. it's a lot of data — please could you fill out the issue template?

[mathause]

Similar to #4524

[imcslatte]

Hi @max-sixty Yep, that worked fine. Should have paid more attention to the error message I suppose.

Thanks,

[dcherian]

Thanks @imcslatte please reopen if your issue is unresolved.