Skip to content

Expose get_node() in the public API #2947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changes/xxxx.doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added migration guide for ``zarr.storage.contains_array()`` and ``zarr.storage.contains_group()``.
1 change: 1 addition & 0 deletions changes/xxxx.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added `zarr.storage.get_node` to get a node within a store.
29 changes: 25 additions & 4 deletions docs/user-guide/v3_migration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,8 +123,7 @@ The Group class
The Store class
~~~~~~~~~~~~~~~

The Store API has changed significant in Zarr-Python 3. The most notable changes to the
Store API are:
The Store API has changed significant in Zarr-Python 3.

Store Import Paths
^^^^^^^^^^^^^^^^^^
Expand Down Expand Up @@ -156,8 +155,30 @@ Common replacements:
change ensures that all store methods are non-blocking and are as performant as
possible.

Beyond the changes store interface, a number of deprecated stores were also removed in
Zarr-Python 3. See :issue:`1274` for more details on the removal of these stores.
Removed store API
^^^^^^^^^^^^^^^^^

``zarr.storage.contains_array()`` and ``zarr.storage.contains_group()`` have been removed.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

contains_array and contains_group have not been removed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a user perspective they have, because they're in a private sub-module. I can make them public in this PR if they were always intended to be public?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can say that contains_array and contains_group are not yet part of the public API. I don't have any problem adding them to the public API but maybe that's better for another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well since putting them back in the public API would make this migration guide section redundant, I think I'll just put them back in a PR to replace this one at some point 😄

Instead, use `zarr.storage.get_node`, and check the return type, e.g.,

>>> import zarr
>>> import zarr.storage
>>>
>>> store = zarr.storage.MemoryStore()
>>> root = zarr.create_group(store=store)
>>> isinstance(zarr.storage.get_node(store, path="", zarr_format=3), zarr.Group)
True
>>>
>>> root.create_array("b", shape=(1,), dtype=int)
<Array memory://.../b shape=(1,) dtype=int64>
>>> isinstance(zarr.storage.get_node(store, path="b", zarr_format=3), zarr.Array)
True

Removed stores
^^^^^^^^^^^^^^

A number of deprecated stores were removed in Zarr-Python 3. See :issue:`1274` for more
details on the removal of these stores.

- ``N5Store`` - see https://github.com/zarr-developers/n5py for an alternative interface to
N5 formatted data.
Expand Down
23 changes: 0 additions & 23 deletions src/zarr/core/sync_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,13 @@
from zarr.core.group import create_hierarchy as create_hierarchy_async
from zarr.core.group import create_nodes as create_nodes_async
from zarr.core.group import create_rooted_hierarchy as create_rooted_hierarchy_async
from zarr.core.group import get_node as get_node_async
from zarr.core.sync import _collect_aiterator, sync

if TYPE_CHECKING:
from collections.abc import Iterator

from zarr.abc.store import Store
from zarr.core.array import Array
from zarr.core.common import ZarrFormat
from zarr.core.metadata import ArrayV2Metadata, ArrayV3Metadata


Expand Down Expand Up @@ -138,24 +136,3 @@ def create_rooted_hierarchy(
"""
async_node = sync(create_rooted_hierarchy_async(store=store, nodes=nodes, overwrite=overwrite))
return _parse_async_node(async_node)


def get_node(store: Store, path: str, zarr_format: ZarrFormat) -> Array | Group:
"""
Get an Array or Group from a path in a Store.

Parameters
----------
store : Store
The store-like object to read from.
path : str
The path to the node to read.
zarr_format : {2, 3}
The zarr format of the node to read.

Returns
-------
Array | Group
"""

return _parse_async_node(sync(get_node_async(store=store, path=path, zarr_format=zarr_format)))
3 changes: 2 additions & 1 deletion src/zarr/storage/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from types import ModuleType
from typing import Any

from zarr.storage._common import StoreLike, StorePath
from zarr.storage._common import StoreLike, StorePath, get_node
from zarr.storage._fsspec import FsspecStore
from zarr.storage._local import LocalStore
from zarr.storage._logging import LoggingStore
Expand All @@ -23,6 +23,7 @@
"StorePath",
"WrapperStore",
"ZipStore",
"get_node",
]


Expand Down
25 changes: 25 additions & 0 deletions src/zarr/storage/_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,14 @@
from zarr.abc.store import ByteRequest, Store
from zarr.core.buffer import Buffer, default_buffer_prototype
from zarr.core.common import ZARR_JSON, ZARRAY_JSON, ZGROUP_JSON, AccessModeLiteral, ZarrFormat
from zarr.core.sync import sync
from zarr.errors import ContainsArrayAndGroupError, ContainsArrayError, ContainsGroupError
from zarr.storage._local import LocalStore
from zarr.storage._memory import MemoryStore
from zarr.storage._utils import normalize_path

if TYPE_CHECKING:
from zarr import Array, Group
from zarr.core.buffer import BufferPrototype


Expand Down Expand Up @@ -505,3 +507,26 @@
return await (store_path / ZGROUP_JSON).exists()
msg = f"Invalid zarr_format provided. Got {zarr_format}, expected 2 or 3" # type: ignore[unreachable]
raise ValueError(msg)


def get_node(store: Store, path: str, zarr_format: ZarrFormat) -> Array | Group:
"""
Get an Array or Group from a path in a Store.

Parameters
----------
store : Store
The store-like object to read from.
path : str
The path to the node to read.
zarr_format : {2, 3}
The zarr format of the node to read.

Returns
-------
Array | Group
"""
from zarr.core.group import _parse_async_node
from zarr.core.group import get_node as get_node_async

Check warning on line 530 in src/zarr/storage/_common.py

View check run for this annotation

Codecov / codecov/patch

src/zarr/storage/_common.py#L529-L530

Added lines #L529 - L530 were not covered by tests

return _parse_async_node(sync(get_node_async(store=store, path=path, zarr_format=zarr_format)))

Check warning on line 532 in src/zarr/storage/_common.py

View check run for this annotation

Codecov / codecov/patch

src/zarr/storage/_common.py#L532

Added line #L532 was not covered by tests
8 changes: 5 additions & 3 deletions tests/test_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -1534,7 +1534,7 @@ def test_create_nodes_concurrency_limit(store: MemoryStore) -> None:
(zarr.core.group.create_hierarchy, zarr.core.sync_group.create_hierarchy),
(zarr.core.group.create_nodes, zarr.core.sync_group.create_nodes),
(zarr.core.group.create_rooted_hierarchy, zarr.core.sync_group.create_rooted_hierarchy),
(zarr.core.group.get_node, zarr.core.sync_group.get_node),
(zarr.core.group.get_node, zarr.storage.get_node),
],
)
def test_consistent_signatures(
Expand Down Expand Up @@ -1594,7 +1594,9 @@ async def test_create_hierarchy(
else:
raise ValueError(f"Invalid impl: {impl}")
if not overwrite:
extra_group = sync_group.get_node(store=store, path="group/extra", zarr_format=zarr_format)
extra_group = zarr.storage.get_node(
store=store, path="group/extra", zarr_format=zarr_format
)
assert extra_group.metadata.attributes == {"path": "group/extra"}
else:
with pytest.raises(FileNotFoundError):
Expand Down Expand Up @@ -1713,7 +1715,7 @@ async def test_group_create_hierarchy(
assert all_members[k].metadata == v == extant_created[k].metadata
# ensure that we left the root group as-is
assert (
sync_group.get_node(store=store, path=group_path, zarr_format=zarr_format).attrs.asdict()
zarr.storage.get_node(store=store, path=group_path, zarr_format=zarr_format).attrs.asdict()
== root_attrs
)

Expand Down