Bump xarray from 2024.3.0 to 2025.1.2 #378

dependabot · 2025-02-03T22:00:37Z

Bumps xarray from 2024.3.0 to 2025.1.2.

Release notes

v2025.01.2

This release brings non-nanosecond datetime and timedelta resolution to xarray, sharded reading in zarr, suggestion of correct names when trying to access non-existent data variables and bug fixes!

Thanks to the 16 contributors to this release: Deepak Cherian, Elliott Sales de Andrade, Jacob Prince-Bieker, Jimmy Westling, Joe Hamman, Joseph Nowak, Justus Magin, Kai Mühlbauer, Mattia Almansi, Michael Niklas, Roelof Rietbroek, Salaheddine EL FARISSI, Sam Levang, Spencer Clark, Stephan Hoyer and Tom Nicholas

What's Changed

dev whats-new by @dcherian in pydata/xarray#9936

Remove outdated quantile test. by @dcherian in pydata/xarray#9945

Relax nanosecond datetime restriction in CF time decoding by @kmuehlbauer in pydata/xarray#9618

fix upstream dev issues by @kmuehlbauer in pydata/xarray#9953

Suggest the correct name when no key matches in the dataset by @Illviljan in pydata/xarray#9943

cast type to PDDatetimeUnitOptions by @kmuehlbauer in pydata/xarray#9963

Update time coding tests to assert exact equality by @spencerkclark in pydata/xarray#9961

remove dask-expr from CI runs, fix related tests by @kmuehlbauer in pydata/xarray#9971

Use zarr-fixture to prevent thread leakage errors by @kmuehlbauer in pydata/xarray#9967

fix weighted polyfit for arrays with more than 2 dimensions by @malmans2 in pydata/xarray#9974

Fix test_doc_example on big-endian systems by @QuLogic in pydata/xarray#9949

Remove unnecessary a article by @salahelfarissi in pydata/xarray#9980

Fix some typing by @dcherian in pydata/xarray#9988

Use flox for grouped first, last by @dcherian in pydata/xarray#9986

Add shards to valid_encodings to enable sharded Zarr writing by @jacobbieker in pydata/xarray#9948

Add shxarray to the xarray ecosystem list by @strawpants in pydata/xarray#9995

Remove repetitive that (replace it with the) by @salahelfarissi in pydata/xarray#9994

Enable DataTree.to_zarr(compute=False) by @slevang in pydata/xarray#9958

Add time_unit argument to CFTimeIndex.to_datetimeindex by @spencerkclark in pydata/xarray#9965

fix mean for datetime-like using the respective time resolution unit by @kmuehlbauer in pydata/xarray#9977

Enable passing a CFTimedeltaCoder to decode_timedelta by @spencerkclark in pydata/xarray#9966

Revert "Use flox for grouped first, last (#9986)" by @dcherian in pydata/xarray#10001

Fix infer_freq, check for subdtype "datetime64"/"timedelta64" by @kmuehlbauer in pydata/xarray#9999

Migrate Zarr region="auto" tests to a class by @dcherian in pydata/xarray#9990

Fix the push method when the limit parameter is bigger than the chunk… by @josephnowak in pydata/xarray#9940

Use freq="D" instead of freq="d" in pd.timedelta_range by @spencerkclark in pydata/xarray#10004

cast numpy scalars to arrays in NamedArray.from_array by @keewis in pydata/xarray#10008

remove outdated type-ignores by @kmuehlbauer in pydata/xarray#10006

Preserve order of variables in combine_by_coords by @kmuehlbauer in pydata/xarray#9070

Add FAQ answer about API stability & backwards compatibility by @TomNicholas in pydata/xarray#9855

release notes: 2025.01.2 by @kmuehlbauer in pydata/xarray#10007

New Contributors

@salahelfarissi made their first contribution in pydata/xarray#9980

@jacobbieker made their first contribution in pydata/xarray#9948

@strawpants made their first contribution in pydata/xarray#9995

Full Changelog: pydata/xarray@v2025.01.1...v2025.01.2

v2025.01.1

This is a quick release to bring compatibility with the Zarr V3 release. It also includes an update to the time decoding

... (truncated)

Commits

d8d1d9e release notes: 2025.01.2 (#10007)
326dbe7 Add FAQ answer about API stability & backwards compatibility (#9855)
5b3f127 Preserve order of variables in combine_by_coords (#9070)
97a4a71 fix typing (#10006)
f91306a cast numpy scalars to arrays in NamedArray.from_array (#10008)
e84e421 Use freq='D' instead of freq='d' in pd.timedelta_range (#10004)
d7ac79a Fix the push method when the limit parameter is bigger than the chunk… (#9940)
5fdceff Migrate Zarr region="auto" tests to a class (#9990)
018b241 Fix infer_freq, check for subdtype "datetime64"/"timedelta64" (#9999)
8ccf1cb Revert "Use flox for grouped first, last (#9986)" (#10001)
Additional commits viewable in compare view

You can trigger a rebase of this PR by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.

Bumps [xarray](https://github.com/pydata/xarray) from 2024.3.0 to 2025.1.2. - [Release notes](https://github.com/pydata/xarray/releases) - [Changelog](https://github.com/pydata/xarray/blob/main/HOW_TO_RELEASE.md) - [Commits](pydata/xarray@v2024.03.0...v2025.01.2) --- updated-dependencies: - dependency-name: xarray dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]>

JMGilbert · 2025-05-14T19:00:11Z

See #406 for details on the numcodecs pin. The pin may be removable alongside the update to zarr in #391.

JMGilbert · 2025-05-14T19:03:07Z

src/dscim/preprocessing/input_damages.py

@@ -808,6 +809,8 @@ def prep(
            if damages[v].dtype == object:
                damages[v] = damages[v].astype("unicode")

+        damages["gcm"] = damages["gcm"].astype("U" + str(max_gcm_len))
+


It seems like the xarray previously automatically updated the datatype when appending additional GCMs. With the update to xarray 2025.1.2 this broke and the code was throwing an error when trying to append a GCM with shorter name. This change ensures that all GCMs use the same datatype.

astype("unicode") is pretty old (like, python 2) way of handling this because python strings at the time were not natively unicode. Would swapping all these over to astype("str") fix this and hopefully not create trouble elsewhere? I think this would use a variable-length string representation -- meaning you don't need all this manual checking for max string length in a GCM name.

codecov · 2025-05-14T19:06:44Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.94%. Comparing base (0cc033c) to head (a4d819d).
Report is 11 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #378      +/-   ##
==========================================
+ Coverage   68.91%   68.94%   +0.03%     
==========================================
  Files          15       15              
  Lines        1750     1752       +2     
==========================================
+ Hits         1206     1208       +2     
  Misses        544      544

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

JMGilbert · 2025-05-14T19:07:37Z

tests/test_main_recipe.py

    )  # the first value only should be censored
    assert as_share[0] != 5 and [as_share[1], as_share[2]] == [30, 60]
    all_cons = menu_instance.weitzman_min(
-        [20, 50, 100], [5, 30, 60], all_cons
+        np.array([20, 50, 100]), np.array([5, 30, 60]), all_cons


A test was failing because of the default python lists. Nowhere in the code do we pass the weitzman_min function with anything except an xarray array, so I thought it would be okay to "lose" the functionality. And in fact the docstring of the weitzman_min function says it expects an xarray array anyway.

That seems good enough, especially if that's how this is documented. A more involved workaround may be another issue/PR. What was it erroring on? Do you have a copy of the error anywhere?

Here's the error:

> as_share = menu_instance.weitzman_min( [20, 50, 100], [5, 30, 60], as_share ) # the first value only should be censored tests/test_main_recipe.py:260: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/dscim/menu/main_recipe.py:910: in weitzman_min clipped_cons = xr.where( /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/computation/computation.py:737: in where result = apply_ufunc( /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/computation/apply_ufunc.py:1280: in apply_ufunc return apply_array_ufunc(func, *args, dask=dask) /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/computation/apply_ufunc.py:891: in apply_array_ufunc return func(*args) /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/core/duck_array_ops.py:387: in where return xp.where(condition, *as_shared_dtype([x, y], xp=xp)) /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/core/duck_array_ops.py:281: in as_shared_dtype dtype = dtypes.result_type(*scalars_or_arrays, xp=xp) /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/core/dtypes.py:267: in result_type array_api_compat.result_type(preprocess_types(t), xp=xp) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ xp = <module 'numpy' from '/home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/numpy/__init__.py'> arrays_and_dtypes = ([5, 30, 60],) def result_type(*arrays_and_dtypes, xp) -> np.dtype: if xp is np or any( isinstance(getattr(t, "dtype", t), np.dtype) for t in arrays_and_dtypes ): > return xp.result_type(*arrays_and_dtypes) E TypeError: Field elements must be 2- or 3-tuples, got '5' /home/jonahmgilbert/anaconda3/envs/dscim-testing/lib/python3.12/site-packages/xarray/compat/array_api_compat.py:44: TypeError

Cool. Thanks.

I think the original fix is fine.

The other aggressive approach might be to cast no_cc_consumption and cc_consumption to xr.DataArray at the start of the function like:

no_cc_consumption = xr.DataArray(no_cc_consumption) cc_consumption = xr.DataArray(cc_consumption)

That might have side-effects for edge cases and it feels kinda magical for an internal-ish function. Could be a solution if this PR's fix accidentally breaks things.

brews

Thanks for this fix!

I raised some points in response to your inline comments, but I don't think any of that discussion should hold back this PR just for my sake.

I was also worried about breaking compatibility with older versions of xarray, given how far behind this is, but these changes seem innocuous.

Thanks again for the PR.

brews · 2025-05-15T22:30:22Z

src/dscim/preprocessing/input_damages.py

@@ -808,6 +809,8 @@ def prep(
            if damages[v].dtype == object:
                damages[v] = damages[v].astype("unicode")

+        damages["gcm"] = damages["gcm"].astype("U" + str(max_gcm_len))
+


astype("unicode") is pretty old (like, python 2) way of handling this because python strings at the time were not natively unicode. Would swapping all these over to astype("str") fix this and hopefully not create trouble elsewhere? I think this would use a variable-length string representation -- meaning you don't need all this manual checking for max string length in a GCM name.

brews · 2025-05-15T22:38:44Z

tests/test_main_recipe.py

    )  # the first value only should be censored
    assert as_share[0] != 5 and [as_share[1], as_share[2]] == [30, 60]
    all_cons = menu_instance.weitzman_min(
-        [20, 50, 100], [5, 30, 60], all_cons
+        np.array([20, 50, 100]), np.array([5, 30, 60]), all_cons


That seems good enough, especially if that's how this is documented. A more involved workaround may be another issue/PR. What was it erroring on? Do you have a copy of the error anywhere?

dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Feb 3, 2025

dependabot bot mentioned this pull request Feb 3, 2025

Bump xarray from 2024.3.0 to 2025.1.1 #366

Closed

dependabot bot force-pushed the dependabot/pip/xarray-2025.1.2 branch from 3c8474f to 2279110 Compare February 4, 2025 00:29

dependabot bot force-pushed the dependabot/pip/xarray-2025.1.2 branch from 2279110 to bc3a0c9 Compare February 20, 2025 00:44

JMGilbert added 3 commits May 14, 2025 13:51

More strictly define data types

72ab4f6

Add necessary requirement for current version of zarr

26bfb17

Merge branch 'main' into dependabot/pip/xarray-2025.1.2

a4d819d

JMGilbert reviewed May 14, 2025

View reviewed changes

JMGilbert requested review from kemccusker and brews May 14, 2025 19:09

brews approved these changes May 15, 2025

View reviewed changes

kemccusker approved these changes May 22, 2025

View reviewed changes

kemccusker merged commit 43e9fd1 into main May 22, 2025
4 checks passed

dependabot bot deleted the dependabot/pip/xarray-2025.1.2 branch May 22, 2025 21:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Bump xarray from 2024.3.0 to 2025.1.2 #378

Bump xarray from 2024.3.0 to 2025.1.2 #378

Uh oh!

dependabot bot commented on behalf of github Feb 3, 2025 •

edited

Loading

Uh oh!

JMGilbert commented May 14, 2025

Uh oh!

JMGilbert May 14, 2025

Uh oh!

brews May 15, 2025

Uh oh!

codecov bot commented May 14, 2025 •

edited

Loading

Uh oh!

JMGilbert May 14, 2025

Uh oh!

brews May 15, 2025

Uh oh!

JMGilbert May 16, 2025

Uh oh!

brews May 21, 2025

Uh oh!

brews left a comment

Uh oh!

brews May 15, 2025

Uh oh!

brews May 15, 2025

Uh oh!

Uh oh!

Uh oh!

Bump xarray from 2024.3.0 to 2025.1.2 #378

Bump xarray from 2024.3.0 to 2025.1.2 #378

Uh oh!

Conversation

dependabot bot commented on behalf of github Feb 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

v2025.01.2

What's Changed

New Contributors

v2025.01.1

Uh oh!

JMGilbert commented May 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented May 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brews left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

dependabot bot commented on behalf of github Feb 3, 2025 •

edited

Loading

codecov bot commented May 14, 2025 •

edited

Loading