Merge branch 'main' into index-repr

Author: keewis

.pre-commit-config.yaml

@@ -19,7 +19,7 @@ repos:
     hooks:
       - id: isort
   - repo: https://github.com/asottile/pyupgrade
-    rev: v2.37.1
+    rev: v2.37.2
     hooks:
       - id: pyupgrade
         args:
@@ -46,7 +46,7 @@ repos:
   #     - id: velin
   #       args: ["--write", "--compact"]
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v0.961
+    rev: v0.971
     hooks:
       - id: mypy
         # Copied from setup.cfg

doc/api.rst

@@ -282,7 +282,6 @@ ndarray attributes
    DataArray.shape
    DataArray.size
    DataArray.dtype
-   DataArray.nbytes
    DataArray.chunks
 
 
@@ -648,11 +647,23 @@ DataArray methods
 Coordinates objects
 ===================
 
+Dataset
+-------
+
 .. autosummary::
    :toctree: generated/
 
-   core.coordinates.DataArrayCoordinates
    core.coordinates.DatasetCoordinates
+   core.coordinates.DatasetCoordinates.dtypes
+
+DataArray
+---------
+
+.. autosummary::
+   :toctree: generated/
+
+   core.coordinates.DataArrayCoordinates
+   core.coordinates.DataArrayCoordinates.dtypes
 
 Plotting
 ========
@@ -736,6 +747,7 @@ Dataset
    DatasetGroupBy.all
    DatasetGroupBy.any
    DatasetGroupBy.count
+   DatasetGroupBy.cumsum
    DatasetGroupBy.max
    DatasetGroupBy.mean
    DatasetGroupBy.median
@@ -765,6 +777,7 @@ DataArray
    DataArrayGroupBy.all
    DataArrayGroupBy.any
    DataArrayGroupBy.count
+   DataArrayGroupBy.cumsum
    DataArrayGroupBy.max
    DataArrayGroupBy.mean
    DataArrayGroupBy.median
@@ -810,6 +823,7 @@ DataArray
    :toctree: generated/
 
    DataArrayRolling
+   DataArrayRolling.__iter__
    DataArrayRolling.construct
    DataArrayRolling.reduce
    DataArrayRolling.argmax

doc/user-guide/io.rst

@@ -790,7 +790,7 @@ Chunk sizes may be specified in one of three ways when writing to a zarr store:
 The resulting chunks will be determined based on the order of the above list; dask
 chunks will be overridden by manually-specified chunks in the encoding argument,
 and the presence of either dask chunks or chunks in the ``encoding`` attribute will
-supercede the default chunking heuristics in zarr.
+supersede the default chunking heuristics in zarr.
 
 Importantly, this logic applies to every array in the zarr store individually,
 including coordinate arrays. Therefore, if a dataset contains one or more dask

doc/user-guide/plotting.rst

@@ -585,7 +585,7 @@ Faceting here refers to splitting an array along one or two dimensions and
 plotting each group.
 Xarray's basic plotting is useful for plotting two dimensional arrays. What
 about three or four dimensional arrays? That's where facets become helpful.
-The general approach to plotting here is called “small multiples”, where the same kind of plot is repeated multiple times, and the specific use of small multiples to display the same relationship conditioned on one ore more other variables is often called a “trellis plot”.
+The general approach to plotting here is called “small multiples”, where the same kind of plot is repeated multiple times, and the specific use of small multiples to display the same relationship conditioned on one or more other variables is often called a “trellis plot”.
 
 Consider the temperature data set. There are 4 observations per day for two
 years which makes for 2920 values along the time dimension.

doc/whats-new.rst

@@ -14,32 +14,77 @@ What's New
 
     np.random.seed(123456)
 
-.. _whats-new.2022.06.0:
+.. _whats-new.2022.07.0:
 
-v2022.06.0 (unreleased)
+v2022.07.0 (unreleased)
 -----------------------
 
 New Features
 ~~~~~~~~~~~~
 
-- Add :py:meth:`Dataset.dtypes`, :py:meth:`DatasetCoordinates.dtypes`,
-  :py:meth:`DataArrayCoordinates.dtypes` properties: Mapping from variable names to dtypes.
+
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+
+Deprecations
+~~~~~~~~~~~~
+
+
+Bug fixes
+~~~~~~~~~
+
+- :py:attr:`DataArray.nbytes` now uses the ``nbytes`` property of the underlying array if available.
+  By `Max Jones <https://github.com/maxrjones>`_.
+
+Documentation
+~~~~~~~~~~~~~
+
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+
+.. _whats-new.2022.06.0:
+
+v2022.06.0 (July 21, 2022)
+--------------------------
+
+This release brings a number of bug fixes and improvements, most notably a major internal
+refactor of the indexing functionality, the use of `flox`_ in ``groupby`` operations,
+and experimental support for the new Python `Array API standard <https://data-apis.org/array-api/latest/>`_.
+It also stops testing support for the abandoned PyNIO.
+
+Much effort has been made to preserve backwards compatibility as part of the indexing refactor.
+We are aware of one `unfixed issue <https://github.com/pydata/xarray/issues/6607>`_.
+
+Please also see the `whats-new.2022.06.0rc0`_ for a full list of changes.
+
+Many thanks to our 18 contributors:
+Bane Sullivan, Deepak Cherian, Dimitri Papadopoulos Orfanos, Emma Marshall, Hauke Schulz, Illviljan,
+Julia Signell, Justus Magin, Keewis, Mathias Hauser, Michael Delgado, Mick, Pierre Manchon, Ray Bell,
+Spencer Clark, Stefaan Lippens, Tom White, Travis A. O'Brien,
+
+New Features
+~~~~~~~~~~~~
+
+- Add :py:attr:`Dataset.dtypes`, :py:attr:`core.coordinates.DatasetCoordinates.dtypes`,
+  :py:attr:`core.coordinates.DataArrayCoordinates.dtypes` properties: Mapping from variable names to dtypes.
   (:pull:`6706`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
 - Initial typing support for :py:meth:`groupby`, :py:meth:`rolling`, :py:meth:`rolling_exp`,
   :py:meth:`coarsen`, :py:meth:`weighted`, :py:meth:`resample`,
   (:pull:`6702`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
-
-Deprecations
-~~~~~~~~~~~~
-
+- Experimental support for wrapping any array type that conforms to the python
+  `array api standard <https://data-apis.org/array-api/latest/>`_. (:pull:`6804`)
+  By `Tom White <https://github.com/tomwhite>`_.
 
 Bug fixes
 ~~~~~~~~~
 
-- :py:meth:`xarray.save_mfdataset` now passes ``**kwargs`` on to ``to_netcdf``,
-  allowing the ``encoding`` and ``unlimited_dims`` options with ``save_mfdataset``.
+- :py:meth:`save_mfdataset` now passes ``**kwargs`` on to :py:meth:`Dataset.to_netcdf`,
+  allowing the ``encoding`` and ``unlimited_dims`` options with :py:meth:`save_mfdataset`.
   (:issue:`6684`)
   By `Travis A. O'Brien <https://github.com/taobrienlbl>`_.
 - Fix backend support of pydap versions <3.3.0  (:issue:`6648`, :pull:`6656`).
@@ -58,16 +103,12 @@ Bug fixes
   (:issue:`6739`, :pull:`6744`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
 
-Documentation
-~~~~~~~~~~~~~
-
-
 Internal Changes
 ~~~~~~~~~~~~~~~~
 
-- :py:meth:`xarray.core.groupby`, :py:meth:`xarray.core.rolling`,
-  :py:meth:`xarray.core.rolling_exp`, :py:meth:`xarray.core.weighted`
-  and :py:meth:`xarray.core.resample` modules are no longer imported by default.
+- ``xarray.core.groupby``, ``xarray.core.rolling``,
+  ``xarray.core.rolling_exp``, ``xarray.core.weighted``
+  and ``xarray.core.resample`` modules are no longer imported by default.
   (:pull:`6702`)
 
 .. _whats-new.2022.06.0rc0:
@@ -120,13 +161,17 @@ New Features
   elements which trigger summarization rather than full repr in (numpy) array
   detailed views of the html repr (:pull:`6400`).
   By `Benoît Bovy <https://github.com/benbovy>`_.
-- Allow passing chunks in ``**kwargs`` form to :py:meth:`Dataset.chunk`, :py:meth:`DataArray.chunk`, and
+- Allow passing chunks in ``kwargs`` form to :py:meth:`Dataset.chunk`, :py:meth:`DataArray.chunk`, and
   :py:meth:`Variable.chunk`. (:pull:`6471`)
   By `Tom Nicholas <https://github.com/TomNicholas>`_.
+- Add :py:meth:`core.groupby.DatasetGroupBy.cumsum` and :py:meth:`core.groupby.DataArrayGroupBy.cumsum`.
+  By `Vladislav Skripniuk <https://github.com/VladSkripniuk>`_ and `Deepak Cherian <https://github.com/dcherian>`_. (:pull:`3147`, :pull:`6525`, :issue:`3141`)
+- Expose `inline_array` kwarg from `dask.array.from_array` in :py:func:`open_dataset`, :py:meth:`Dataset.chunk`,
+  :py:meth:`DataArray.chunk`, and :py:meth:`Variable.chunk`. (:pull:`6471`)
 - Expose the ``inline_array`` kwarg from :py:func:`dask.array.from_array` in :py:func:`open_dataset`,
   :py:meth:`Dataset.chunk`, :py:meth:`DataArray.chunk`, and :py:meth:`Variable.chunk`. (:pull:`6471`)
   By `Tom Nicholas <https://github.com/TomNicholas>`_.
-- :py:meth:`xr.polyval` now supports :py:class:`Dataset` and :py:class:`DataArray` args of any shape,
+- :py:func:`polyval` now supports :py:class:`Dataset` and :py:class:`DataArray` args of any shape,
   is faster and requires less memory. (:pull:`6548`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
 - Improved overall typing.
@@ -159,7 +204,7 @@ Breaking changes
   zarr            2.5   2.8
   =============== ===== ====
 
-- The Dataset and DataArray ``rename*`` methods do not implicitly add or drop
+- The Dataset and DataArray ``rename```` methods do not implicitly add or drop
   indexes. (:pull:`5692`).
   By `Benoît Bovy <https://github.com/benbovy>`_.
 - Many arguments like ``keep_attrs``, ``axis``, and ``skipna`` are now keyword
@@ -172,11 +217,6 @@ Breaking changes
   (:pull:`6548`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
 
-
-Deprecations
-~~~~~~~~~~~~
-
-
 Bug fixes
 ~~~~~~~~~
 
@@ -204,16 +244,16 @@ Bug fixes
   By `Stan West <https://github.com/stanwest>`_.
 - Fix bug in :py:func:`where` when passing non-xarray objects with ``keep_attrs=True``. (:issue:`6444`, :pull:`6461`)
   By `Sam Levang <https://github.com/slevang>`_.
-- Allow passing both ``other`` and ``drop=True`` arguments to ``xr.DataArray.where``
-  and ``xr.Dataset.where`` (:pull:`6466`, :pull:`6467`).
+- Allow passing both ``other`` and ``drop=True`` arguments to :py:meth:`DataArray.where`
+  and :py:meth:`Dataset.where` (:pull:`6466`, :pull:`6467`).
   By `Michael Delgado <https://github.com/delgadom>`_.
 - Ensure dtype encoding attributes are not added or modified on variables that contain datetime-like
   values prior to being passed to :py:func:`xarray.conventions.decode_cf_variable` (:issue:`6453`,
   :pull:`6489`).
   By `Spencer Clark <https://github.com/spencerkclark>`_.
 - Dark themes are now properly detected in Furo-themed Sphinx documents (:issue:`6500`, :pull:`6501`).
   By `Kevin Paul <https://github.com/kmpaul>`_.
-- :py:meth:`isel` with `drop=True` works as intended with scalar :py:class:`DataArray` indexers.
+- :py:meth:`Dataset.isel`, :py:meth:`DataArray.isel` with `drop=True` works as intended with scalar :py:class:`DataArray` indexers.
   (:issue:`6554`, :pull:`6579`)
   By `Michael Niklas <https://github.com/headtr1ck>`_.
 - Fixed silent overflow issue when decoding times encoded with 32-bit and below
@@ -229,15 +269,17 @@ Documentation
   sizes. In particular, correct the syntax and replace lists with tuples in the
   examples. (:issue:`6333`, :pull:`6334`)
   By `Stan West <https://github.com/stanwest>`_.
-
+- Mention that :py:meth:`DataArray.rename` can rename coordinates.
+  (:issue:`5458`, :pull:`6665`)
+  By `Michael Niklas <https://github.com/headtr1ck>`_.
 - Added examples to :py:meth:`Dataset.thin` and :py:meth:`DataArray.thin`
   By `Emma Marshall <https://github.com/e-marshall>`_.
 
 Performance
 ~~~~~~~~~~~
 
 - GroupBy binary operations are now vectorized.
-  Previously this involved looping over all groups. (:issue:`5804`,:pull:`6160`)
+  Previously this involved looping over all groups. (:issue:`5804`, :pull:`6160`)
   By `Deepak Cherian <https://github.com/dcherian>`_.
 - Substantially improved GroupBy operations using `flox <https://flox.readthedocs.io/en/latest/>`_.
   This is auto-enabled when ``flox`` is installed. Use ``xr.set_options(use_flox=False)`` to use

xarray/backends/api.py

@@ -895,6 +895,22 @@ def open_mfdataset(
     combine_nested
     open_dataset
 
+    Examples
+    --------
+    A user might want to pass additional arguments into ``preprocess`` when
+    applying some operation to many individual files that are being opened. One route
+    to do this is through the use of ``functools.partial``.
+
+    >>> from functools import partial
+    >>> def _preprocess(x, lon_bnds, lat_bnds):
+    ...     return x.sel(lon=slice(*lon_bnds), lat=slice(*lat_bnds))
+    ...
+    >>> lon_bnds, lat_bnds = (-110, -105), (40, 45)
+    >>> partial_func = partial(_preprocess, lon_bnds=lon_bnds, lat_bnds=lat_bnds)
+    >>> ds = xr.open_mfdataset(
+    ...     "file_*.nc", concat_dim="time", preprocess=_preprocess
+    ... )  # doctest: +SKIP
+
     References
     ----------
 

xarray/coding/calendar_ops.py

@@ -98,7 +98,7 @@ def convert_calendar(
     Notes
     -----
     Passing a value to `missing` is only usable if the source's time coordinate as an
-    inferrable frequencies (see :py:func:`~xarray.infer_freq`) and is only appropriate
+    inferable frequencies (see :py:func:`~xarray.infer_freq`) and is only appropriate
     if the target coordinate, generated from this frequency, has dates equivalent to the
     source. It is usually **not** appropriate to use this mode with:
 

xarray/coding/cftime_offsets.py

@@ -1200,7 +1200,7 @@ def date_range_like(source, calendar, use_cftime=None):
     freq = infer_freq(source)
     if freq is None:
         raise ValueError(
-            "`date_range_like` was unable to generate a range as the source frequency was not inferrable."
+            "`date_range_like` was unable to generate a range as the source frequency was not inferable."
         )
 
     use_cftime = _should_cftime_be_used(source, calendar, use_cftime)

xarray/core/alignment.py

@@ -285,7 +285,7 @@ def find_matching_unindexed_dims(self) -> None:
         self.unindexed_dim_sizes = unindexed_dim_sizes
 
     def assert_no_index_conflict(self) -> None:
-        """Check for uniqueness of both coordinate and dimension names accross all sets
+        """Check for uniqueness of both coordinate and dimension names across all sets
         of matching indexes.
 
         We need to make sure that all indexes used for re-indexing or alignment