Skip to content

Commit 447e5a3

Browse files
authored
Speed up netCDF4, h5netcdf backends (#9067)
* Speed up netCDF4 backend. xref #9058 Accessing `.shape` on a netCDF4 variable is ~20-40ms. This can add up for large numbers of variables, e.g.: #9058 We already request the shape when creating NetCDF4ArrayWrapper, so we can reuse that. * Update h5netcdf backend too
1 parent 1f3bc7e commit 447e5a3

File tree

3 files changed

+8
-2
lines changed

3 files changed

+8
-2
lines changed

doc/whats-new.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,12 @@ v2024.05.1 (unreleased)
2323
New Features
2424
~~~~~~~~~~~~
2525

26+
Performance
27+
~~~~~~~~~~~
28+
29+
- Small optimization to the netCDF4 and h5netcdf backends (:issue:`9058`, :pull:`9067`).
30+
By `Deepak Cherian <https://github.com/dcherian>`_.
31+
2632

2733
Breaking changes
2834
~~~~~~~~~~~~~~~~

xarray/backends/h5netcdf_.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -221,7 +221,7 @@ def open_store_variable(self, name, var):
221221

222222
# save source so __repr__ can detect if it's local or not
223223
encoding["source"] = self._filename
224-
encoding["original_shape"] = var.shape
224+
encoding["original_shape"] = data.shape
225225

226226
vlen_dtype = h5py.check_dtype(vlen=var.dtype)
227227
if vlen_dtype is str:

xarray/backends/netCDF4_.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -454,7 +454,7 @@ def open_store_variable(self, name: str, var):
454454
pop_to(attributes, encoding, "least_significant_digit")
455455
# save source so __repr__ can detect if it's local or not
456456
encoding["source"] = self._filename
457-
encoding["original_shape"] = var.shape
457+
encoding["original_shape"] = data.shape
458458

459459
return Variable(dimensions, data, attributes, encoding)
460460

0 commit comments

Comments
 (0)