Calendar utilities (#5233)

* dt.calendar and date_range

* Migrate calendar utils from xclim | add dt.calendar

* upd whats new

* skip calendar tests with no cftime

* add requires cftime 1.1.0

* import date_ranges in main

* Apply suggestions from code review

Co-authored-by: Mathias Hauser <[email protected]>

* Add docs - use already existing is np datetime func

* update from suggestions

* Apply suggestions from code review

Co-authored-by: Spencer Clark <[email protected]>

* Modifications following review

* Add DataArray and Dataset methods

* use proper type annotation

* Apply suggestions from code review

Co-authored-by: Spencer Clark <[email protected]>

* some more modifications after review

* Apply suggestions from code review

The code will break with this commit. Variable renaming to be done throughout all functions.

Co-authored-by: Spencer Clark <[email protected]>

* Finish applying suggestions from review

* Put back missing @require_cftime

* Apply suggestions from code review

Co-authored-by: Spencer Clark <[email protected]>

* Add tests - few fixes

* wrap docstrings

* Change way of importing/testing for cftime

* Upd the weather-climate doc page

* fix doc examples

* Neat docs

* fix in tests after review

* Apply suggestions from code review

Co-authored-by: Spencer Clark <[email protected]>

* Better explain missing in notes - copy changes to obj methods

* Apply suggestions from code review

Co-authored-by: Spencer Clark <[email protected]>

* Remove unused import

Co-authored-by: Mathias Hauser <[email protected]>
Co-authored-by: Spencer Clark <[email protected]>
Co-authored-by: Illviljan <[email protected]>

Author: 4 people

doc/api-hidden.rst

@@ -77,6 +77,7 @@
    core.accessor_dt.DatetimeAccessor.floor
    core.accessor_dt.DatetimeAccessor.round
    core.accessor_dt.DatetimeAccessor.strftime
+   core.accessor_dt.DatetimeAccessor.calendar
    core.accessor_dt.DatetimeAccessor.date
    core.accessor_dt.DatetimeAccessor.day
    core.accessor_dt.DatetimeAccessor.dayofweek

doc/api.rst

@@ -109,6 +109,8 @@ Dataset contents
    Dataset.drop_dims
    Dataset.set_coords
    Dataset.reset_coords
+   Dataset.convert_calendar
+   Dataset.interp_calendar
    Dataset.get_index
 
 Comparisons
@@ -308,6 +310,8 @@ DataArray contents
    DataArray.drop_duplicates
    DataArray.reset_coords
    DataArray.copy
+   DataArray.convert_calendar
+   DataArray.interp_calendar
    DataArray.get_index
    DataArray.astype
    DataArray.item
@@ -526,6 +530,7 @@ Datetimelike properties
    DataArray.dt.season
    DataArray.dt.time
    DataArray.dt.date
+   DataArray.dt.calendar
    DataArray.dt.is_month_start
    DataArray.dt.is_month_end
    DataArray.dt.is_quarter_end
@@ -1064,6 +1069,8 @@ Creating custom indexes
    :toctree: generated/
 
    cftime_range
+   date_range
+   date_range_like
 
 Faceting
 --------

doc/user-guide/weather-climate.rst

@@ -127,6 +127,23 @@ using the same formatting as the standard `datetime.strftime`_ convention .
     dates.strftime("%c")
     da["time"].dt.strftime("%Y%m%d")
 
+Conversion between non-standard calendar and to/from pandas DatetimeIndexes is
+facilitated with the :py:meth:`xarray.Dataset.convert_calendar` method (also available as
+:py:meth:`xarray.DataArray.convert_calendar`). Here, like elsewhere in xarray, the ``use_cftime``
+argument controls which datetime backend is used in the output. The default (``None``) is to
+use `pandas` when possible, i.e. when the calendar is standard and dates are within 1678 and 2262.
+
+.. ipython:: python
+
+    dates = xr.cftime_range(start="2001", periods=24, freq="MS", calendar="noleap")
+    da_nl = xr.DataArray(np.arange(24), coords=[dates], dims=["time"], name="foo")
+    da_std = da.convert_calendar("standard", use_cftime=True)
+
+The data is unchanged, only the timestamps are modified. Further options are implemented
+for the special ``"360_day"`` calendar and for handling missing dates. There is also
+:py:meth:`xarray.Dataset.interp_calendar` (and :py:meth:`xarray.DataArray.interp_calendar`)
+for `interpolating` data between calendars.
+
 For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:
 
 - `Partial datetime string indexing`_:
@@ -150,7 +167,8 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:
 
 - Access of basic datetime components via the ``dt`` accessor (in this case
   just "year", "month", "day", "hour", "minute", "second", "microsecond",
-  "season", "dayofyear", "dayofweek", and "days_in_month"):
+  "season", "dayofyear", "dayofweek", and "days_in_month") with the addition
+  of "calendar", absent from pandas:
 
 .. ipython:: python
 
@@ -160,6 +178,7 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:
     da.time.dt.dayofyear
     da.time.dt.dayofweek
     da.time.dt.days_in_month
+    da.time.dt.calendar
 
 - Rounding of datetimes to fixed frequencies via the ``dt`` accessor:
 
@@ -214,30 +233,6 @@ For data indexed by a :py:class:`~xarray.CFTimeIndex` xarray currently supports:
 
     da.resample(time="81T", closed="right", label="right", base=3).mean()
 
-.. note::
-
-
-   For some use-cases it may still be useful to convert from
-   a :py:class:`~xarray.CFTimeIndex` to a :py:class:`pandas.DatetimeIndex`,
-   despite the difference in calendar types. The recommended way of doing this
-   is to use the built-in :py:meth:`~xarray.CFTimeIndex.to_datetimeindex`
-   method:
-
-   .. ipython:: python
-       :okwarning:
-
-       modern_times = xr.cftime_range("2000", periods=24, freq="MS", calendar="noleap")
-       da = xr.DataArray(range(24), [("time", modern_times)])
-       da
-       datetimeindex = da.indexes["time"].to_datetimeindex()
-       da["time"] = datetimeindex
-
-   However in this case one should use caution to only perform operations which
-   do not depend on differences between dates (e.g. differentiation,
-   interpolation, or upsampling with resample), as these could introduce subtle
-   and silent errors due to the difference in calendar types between the dates
-   encoded in your data and the dates stored in memory.
-
 .. _Timestamp-valid range: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timestamp-limitations
 .. _ISO 8601 standard: https://en.wikipedia.org/wiki/ISO_8601
 .. _partial datetime string indexing: https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#partial-string-indexing

doc/whats-new.rst

@@ -156,6 +156,8 @@ New Features
 - Added ``storage_options`` argument to :py:meth:`to_zarr` (:issue:`5601`, :pull:`5615`).
   By `Ray Bell <https://github.com/raybellwaves>`_, `Zachary Blackwood <https://github.com/blackary>`_ and
   `Nathan Lis <https://github.com/wxman22>`_.
+- Added calendar utilities :py:func:`DataArray.convert_calendar`, :py:func:`DataArray.interp_calendar`, :py:func:`date_range`, :py:func:`date_range_like` and :py:attr:`DataArray.dt.calendar` (:issue:`5155`, :pull:`5233`).
+  By `Pascal Bourgault <https://github.com/aulemahal>`_.
 - Histogram plots are set with a title displaying the scalar coords if any, similarly to the other plots (:issue:`5791`, :pull:`5792`).
   By `Maxime Liquet <https://github.com/maximlt>`_.
 - Slice plots display the coords units in the same way as x/y/colorbar labels (:pull:`5847`).

xarray/__init__.py

@@ -9,7 +9,7 @@
 )
 from .backends.rasterio_ import open_rasterio
 from .backends.zarr import open_zarr
-from .coding.cftime_offsets import cftime_range
+from .coding.cftime_offsets import cftime_range, date_range, date_range_like
 from .coding.cftimeindex import CFTimeIndex
 from .coding.frequencies import infer_freq
 from .conventions import SerializationWarning, decode_cf
@@ -65,6 +65,8 @@
     "combine_by_coords",
     "combine_nested",
     "concat",
+    "date_range",
+    "date_range_like",
     "decode_cf",
     "dot",
     "cov",