Hypothesis strategies in xarray.testing.strategies #6908
Conversation
|
I also added my chunking strategy from HypothesisWorks/hypothesis#3433 |
TomNicholas
added
the
topic-hypothesis
github-actions
bot
added
topic-testing
documentation
and removed
topic-hypothesis
github-actions
bot
added
CI
for more information, see https://pre-commit.ci
doc/user-guide/testing.rst
Outdated
| but building a dataset from scratch (i.e. method (2)) requires building the dataset object in such as way that all of | ||
| the data variables have compatible dimensions. You can build up a dictionary of the form ``{var_name: data_variable}`` | ||
| yourself, or you can use the ``data_vars`` argument to the ``data_variables`` strategy (TODO): | ||
|
|
||
| .. ipython:: python | ||
| :okexcept: | ||
|
|
||
| sparse_data_vars = xrst.data_variables(data=sparse_arrays()) | ||
| sparse_datasets = xrst.datasets(data_vars=sparse_data_vars) | ||
|
|
||
| sparse_datasets.example() |
There was a problem hiding this comment.
I had intended to push .pin in some form upstream, but I of course forgot about the other types of strategies so I can see why that would not be desirable.
Putting the code into the definition of the composite strategy is much better than what I had before (constructing the examples using data.draw directly in the test), so that would be fine with me.
Do you know if it is possible to use make_strategies_namespace with additional parameters to the array's constructor, like units for pint or chunks for dask? I guess if we use the pint_arrays function from above we could use partial for this (and anyway, pint does not implement __array_namespace__ at the moment).
|
How do we move this forward? Even Xarray objects with just numpy arrays would be quite useful |
|
I think #8404 made a lot of progress on this, including shipping the user-facing documentation. If you wanted to open a PR rebasing this set of changes on main, I think that might be most of the remaining work. |
|
So I just did a monster merge of
It's mostly just dealing with the above and also making sure we can generate sets of variables with alignable dimensions efficiently. We also probably should think about what we want the signatures of the more complicated strategies to be: e.g. are we wanting to pass
A lot of the work that went into #8404 was working out how to make it general enough to handle non-numpy arrays. |
* main: (214 commits) Adds copy parameter to __array__ for numpy 2.0 (pydata#9393) `numpy 2` compatibility in the `pydap` backend (pydata#9391) pyarrow dependency added to doc environment (pydata#9394) Extend padding functionalities (pydata#9353) refactor GroupBy internals (pydata#9389) Combine `UnsignedIntegerCoder` and `CFMaskCoder` (pydata#9274) passing missing parameters to ZarrStore.open_store when opening a datatree (pydata#9377) Fix tests on big-endian systems (pydata#9380) Improve error message on `ds['x', 'y']` (pydata#9375) Improve error message for missing coordinate index (pydata#9370) Add flaky to TestNetCDF4ViaDaskData (pydata#9373) Make chunk manager an option in `set_options` (pydata#9362) Revise (pydata#9371) Remove duplicate word from docs (pydata#9367) Adding open_groups to BackendEntryPointEngine, NetCDF4BackendEntrypoint, and H5netcdfBackendEntrypoint (pydata#9243) Revise (pydata#9366) Fix rechunking to a frequency with empty bins. (pydata#9364) whats-new entry for dropping python 3.9 (pydata#9359) drop support for `python=3.9` (pydata#8937) Revise (pydata#9357) ...
Co-authored-by: Justus Magin <[email protected]>
These seem like O(ε) improvements to a really great PR. |
|
@TomNicholas I think these strategies would be really helpful for zarr-developers/VirtualiZarr#394 and zarr-developers/VirtualiZarr#490. Is there anything I can do to help move this forward? |
|
Oh man I left this one for so long... 😞 I think the reason I didn't merge it is because currently the API for the It may be that if we merge it now we have to go back and change those APIs to be consistent later. But it also maybe that that isn't too bad, and it basically just limits the usefulness of this PR for now. |
Adds a whole suite of hypothesis strategies for generating xarray objects, inspired by and separated out from the new hypothesis strategies in #4972. They are placed into the namespace
xarray.testing.strategies, and publicly mentioned in the API docs, but with a big warning message. There is also a newtestingpage in the user guide documenting how to use these strategies.whats-new.rstapi.rstEDIT: A
variablesstrategy and user-facing documentation were shipped in #8404