Skip to content

NestedDirectoryStore.listdir returns invalid chunk keys #1334

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
braingram opened this issue Jan 26, 2023 · 0 comments · Fixed by #1335
Closed

NestedDirectoryStore.listdir returns invalid chunk keys #1334

braingram opened this issue Jan 26, 2023 · 0 comments · Fixed by #1335
Labels
bug Potential issues with the zarr-python library

Comments

@braingram
Copy link
Contributor

Zarr version

2.13.6 and current head 0bf0b3b

Numcodecs version

0.11.0

Python Version

3.10.6

Operating System

Mac

Installation

pip install zarr and built from source

Description

Calling listdir on NestedDirectoryStore returns keys with a '.' dimension separator even though the store does not support this separator.

I believe the issue is this line:

new_children.append(rel_path.replace(os.path.sep, '.'))

I will open a PR with a fix and update to the tests.

Steps to reproduce

import zarr

store = zarr.storage.NestedDirectoryStore('tmp')
z = zarr.zeros((10, 10), chunks=(5, 5), store=store)
z[:] = 1
keys = zarr.storage.listdir(store)
print(f"Store keys: {keys}")

Returns

Store keys: ['.zarray', '0.0', '0.1', '1.0', '1.1']

However attempts to get data for those keys returns None:

print(store.get('0.0'))

Returns

None

Additional output

No response

@braingram braingram added the bug Potential issues with the zarr-python library label Jan 26, 2023
braingram added a commit to braingram/zarr-python that referenced this issue Jan 26, 2023
NestedDirectoryStore which inherits from DirectoryStore
only supports '/' as a dimension separator. However
listdir uses the parent DirectoryStore.listdir which
produces keys with an incorrect separator '.'

Fixes zarr-developers#1334
rabernat pushed a commit that referenced this issue Jan 26, 2023
* use store dimension separtor in DirectoryStore.listdir

NestedDirectoryStore which inherits from DirectoryStore
only supports '/' as a dimension separator. However
listdir uses the parent DirectoryStore.listdir which
produces keys with an incorrect separator '.'

Fixes #1334

* update release note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant