Extension proposal: multiscale arrays v0.1

[joshmoore]

This issue has been migrated to an image.sc topic after the 2020-05-06 community discussion. Authors are still encouraged to make use of the specification in their own libraries. As the v3 extension mechanism matures, the specification will be updated and registered as appropriate. Feedback and request changes are welcome either on this repository or on image.sc.

---

As a first draft of support for the multiscale use-case (#23), this issue proposes an intermediate nomenclature for describing groups of Zarr arrays which are scaled down versions of one another, e.g.:

example/
├── 0    # Full-sized array
├── 1    # Scaled down 0, e.g. 0.5; for images, in the X&Y dimensions
├── 2    # Scaled down 1, ...
├── 3    # Scaled down 2, ...
└── 4    # Etc.

This layout was independently developed in a number of implementations and has since been implemented in others, including:

• https://github.com/napari/napari
• https://github.com/glencoesoftware/bioformats2raw
• https://github.com/ome/omero-ms-zarr
• https://github.com/ome/bioformats
• Yours here

Using a common metadata representation across implementations:

1. fosters a common vocabulary between existing implementations
2. enables other implementations to reliably detect multiscale arrays
3. permits the upgrade of v0.1 arrays to future versions of this or other extension
4. tests this extension for limitations against multiple use cases

A basic example of the metadata that is added to the containing Zarr group is seen here:

{
  “multiscales”: [
    {
      “datasets” : [
          {"path": "0"},
          {"path": "1"},
          {"path": "2"},
          {"path": "3"},
          {"path": "4"}
        ]
      “version” : “0.1”
    }
     // See the detailed example below for optional metadata
  ]
}

Process

An RFC process for Zarr does not yet exist. Additionally, the v3 spec is a work-in-progress. However, since the implementations listed above as well as others are already being developed, I'd propose that if a consensus can be reached here, this issue should be turned into an .rst file similar to those in the v3 branches (e.g. filters) and used as a temporary spec for defining arrays with the understanding that this a prototype intended to be amended and brought into the general extension mechanism as it develops.

I'd welcome any suggestions/feedback, but especially around:

• Better terms for "multiscale" and "series"
• The most useful enum values
• Is this already too complicated? (Limit to one series per group?) or on the flip side:
• Are there existing use cases that aren't supported? (Note: I'm aware of some examples like BDV's N5 format but I'd suggest they are higher-level than just "multiscale arrays".)

Deadline for a first round of comments: March 15, 2020
Deadline for a second round of comments: April 15, 2020

Detailed example

Color key (according to https://www.ietf.org/rfc/rfc2119.txt):

- MUST     : If these values are not present, the multiscale series will not be detected.
! SHOULD   : Missing values may cause issues in future versions.
+ MAY      : Optional values which can be readily omitted.
# UNPARSED : When updating between versions, no transformation will be performed on these values.

Color-coded example:

-{
-  "multiscales": [
-    {
!      "version": "0.1",
!      "name": "example",
-      "datasets": [
-        {"path": "0"},
-        {"path": "1"},
-        {"path": "2"}
-      ],
!      "type": "gaussian",
!      "metadata": {
+        "method":
#          "skiimage.transform.pyramid_gaussian",
+        "version":
#          "0.16.1",
+        "args":
#          [true],
+        "kwargs":
#          {"multichannel": true}
!      }
-    }
-  ]
-}

Explanation

• Multiple multiscale series of datasets can be present in a single group.
• By convention, the first multiscale should be chosen if all else is equal.
• Alternatively, a multiscale can be chosen by name or with slightly more effort, but the zarray metadata like chunk size.
• The paths to the arrays are ordered from largest to smallest.
• These paths could potentially point to datasets in other groups via “../foo/0” in the future. For now, the identifiers MUST be local to the annotated group.
• These values SHOULD (MUST?) come from the enumeration below.
• The metadata example is taken from https://scikit-image.org/docs/dev/api/skimage.transform.html#skimage.transform.pyramid_reduce

Type enumeration:

• gaussian, e.g. skimage.transform.pyramid_gaussian
• laplacian, e.g. skimage.transform.pyramid_laplacian
• reduce, e.g. skimage.transform.pyramid_laplacian
• pick, e.g. SimpleImageScaler's “top-left” strategy
• Suggestions welcome

Sample code

#!/usr/bin/env python
import argparse
import zarr
import numpy as np
from skimage import data
from skimage.transform import pyramid_gaussian, pyramid_laplacian

parser = argparse.ArgumentParser()
parser.add_argument("zarr_directory")
ns = parser.parse_args()


# 1. Setup of data and Zarr directory
base = np.tile(data.astronaut(), (2, 2, 1))

gaussian = list(
    pyramid_gaussian(base, downscale=2, max_layer=4, multichannel=True)
)

laplacian = list(
    pyramid_laplacian(base, downscale=2, max_layer=4, multichannel=True)
)

store = zarr.DirectoryStore(ns.zarr_directory)
grp = zarr.group(store)
grp.create_dataset("base", data=base)


# 2. Generate datasets
series_G = []
for g, dataset in enumerate(gaussian):
    if g == 0:
        path = "base"
    else:
        path = "G%s" % g
        grp.create_dataset(path, data=gaussian[g])
    series_G.append({"path": path})

series_L = []
for l, dataset in enumerate(laplacian):
    if l == 0:
        path = "base"
    else:
        path = "L%s" % l
        grp.create_dataset(path, data=laplacian[l])
    series_L.append({"path": path})


# 3. Generate metadata block
multiscales = []
for name, series in (("gaussian", series_G),
                     ("laplacian", series_L)):
    multiscale = {
      "version": "0.1",
      "name": name,
      "datasets": series,
      "type": name,
    }
    multiscales.append(multiscale)
grp.attrs["multiscales"] = multiscales

which results in a .zattrs file of the form:

{
    "multiscales": [
        {
            "datasets": [
                {
                    "path": "base"
                },
                {
                    "path": "G1"
                },
                {
                    "path": "G2"
                },
                {
                    "path": "G3"
                },
                {
                    "path": "G4"
                }
            ],
            "name": "gaussian",
            "type": "gaussian",
            "version": "0.1"
        },
        {
            "datasets": [
                {
                    "path": "base"
                },
                {
                    "path": "L1"
                },
                {
                    "path": "L2"
                },
                {
                    "path": "L3"
                },
                {
                    "path": "L4"
                }
            ],
            "name": "laplacian",
            "type": "laplacian",
            "version": "0.1"
        }
    ]
}

and the following on-disk layout:

/var/folders/z5/txc_jj6x5l5cm81r56ck1n9c0000gn/T/tmp77n1ga3r.zarr
├── G1
│   ├── 0.0.0
...
│   └── 3.1.1
├── G2
│   ├── 0.0.0
│   ├── 0.1.0
│   ├── 1.0.0
│   └── 1.1.0
├── G3
│   ├── 0.0.0
│   └── 1.0.0
├── G4
│   └── 0.0.0
├── L1
│   ├── 0.0.0
...
│   └── 3.1.1
├── L2
│   ├── 0.0.0
│   ├── 0.1.0
│   ├── 1.0.0
│   └── 1.1.0
├── L3
│   ├── 0.0.0
│   └── 1.0.0
├── L4
│   └── 0.0.0
└── base
    ├── 0.0.0
...
    └── 1.1.1

9 directories, 54 files

Revision	Source	Date	Description
6	External feedback on twitter and image.sc	2020-05-06	Remove "scale"; clarify ordering and naming
5	External bug report from @mtbc	2020-04-21	Fixed error in the simple example
4	#50 (comment)	2020-04-08	Changed "name" to "path"
3	Discussions up through #50 (comment)	2020-04-01	Updated naming schema
2	#50 (comment)	2020-03-07	Fixed typo
1	@joshmoore	2020-03-06	Original text from in person discussions

---

Thanks to @ryan-williams, @jakirkham, @freeman-lab, @petebankhead, @jni, @sofroniewn, @chris-allan, and anyone else whose GitHub account I've forgotten for the preliminary discussions.

[manzt]

I'm really happy see this. We also used a similar layout for storing pyramids initially to that proposed above and it's fantastic to see this formalized.

I'm curious about the decision to store the base array in the same group as the downsampled levels. We initially did the same, but then moved towards a structure separating the two:

└── example/
    ├── .zgroup
    ├── base
    │   ├── .zarray
    │   ├── .zattrs
    │   ├── 0.0.0
    │   └── ...etc
    └── sub-resolutions/
        ├── .zgroup
        ├── .zattrs
        ├── 01/
        │   ├── .zarray
        │   ├── 0.0.0
        │   └── ...etc
        └── 02/
            ├── .zarray
            ├── 0.0.0
            └── ...etc

as a more general "image" format in zarr. One could expect to find a "base" array and then check for the "sub-resolutions" group to determine if it is a pyramid or not. We thought this structure would allow for other types of data (e.g. segmentation) to be store along side the base array. Again, thanks for the work here in formalizing this!

[joshmoore]

Thanks, @manzt. Let's see if there are more votes for the deeper representation. It's certainly also what I was originally thinking about in #23. The downside is that one likely needs metadata on all the datasets pointing up and down the hierarchy in order to support detection of the sequence from any scale. It's the other major design layout I can think of. (If anyone has more, those would be very welcome.)

[sofroniewn]

@joshmoore amazing to see this kick off. A couple short comments

• If looking for alternate names I'd consider multiresolution, but multiscale definitely works for me. We have been using pyramid at in napari but are thinking of changing (see here Support for multiresolution (pyramid) layers besides Image layers napari/napari#1019 (comment) and we can try and go with whatever the majority likes).

• One thing that has come up for me and a list of "scales" is that when you have large volumetric timeseries, where you might create a pyramid for each timepoint, some of the axes are unscaled, so you really need to look at the shapes of the arrays to do the right thing. I see that the field is optional but I wonder how much is gained from it (I'm also not opposed though, and would probably find usage from it, but wanted to put out this caveat)

• Multiple series per group is probably good flexibility to have, say if you have two independent multiscale datasets you want to put in the same group, it lets the group abstraction remain separate from the multiscale details.

• The concept of having base + subresolutions like @manzt proposes is intriguing to me too. Ultimately for visualization purposes I want something like a single list of arrays so I guess I find that representation little simpler, but I can construct that from the later representation if I know the data is multiscale and maybe it is nice to keep that a little separate. I will think on it more, curious what others say.

[d-v-b]

Glad to see the discussion here. Some thoughts:

• Philosophically, I'd like to suggest a few constraints (both of which are satisfied by @joshmoore's proposal, but not by a lot of other existing multiscale image schemas): First, individual images should be portable -- wherever possible, images should not have metadata/attributes that indicates their role in a multiscale representation, so that they can be copied somewhere else and viewed on their own without losing context. Second, no magic dataset names like s0, s1, etc. The use of the list of datasets in @joshmoore's group attributes solves this problem.

• Personally I'm not a fan of putting the base image at a different level of the hierarchy, since most software i've seen assumes that all the different scale levels will all be elements in the same collection. @manzt you suggest that you adopted this structure in order to facilitate checking for a multiscale representation, but I think this is a job for group metadata, not hierarchy.

• For simplicity, I would propose a restriction of one multiscale representation per group. Groups are cheap; if you to represent 2 multiscale images, then make 2 groups. (This doesn't work for multiple multiscale representations that use the same base image, e.g. gaussian and laplacian pyramids). The use of the series group metadata in @joshmoore's proposal handles this nicely.

• A multiscale image is a collection of images. Accordingly, the "multiscaleness" should be a group attribute that lists the images in the collection, which is how @joshmoore does it in the draft prosposal. I would add some dataset-specific information to the group attributes: software that consumes multiscale images needs to know about how the spatial properties of each image, and on cloud storage it can be cumbersome to query each image individually; so for convenience this image metadata could also be in the group attributes that describe the multiscale representation. I think explicitly listing the transform attributes of each image is safer than just listing "scales", as long as the transform attributes of each image are small.

Here's example metadata that implements this concept. The specifics of the "transform attributes" don't really matter -- this could be an affine transform, or something fancier. But I think the basic idea of putting the spatial information of each dataset in the group attributes is solid.

// group attributes
{
  “multiscale”: {
    “version” : “0.1”,
        “datasets” : [“0” : {transform attributes of 0}, 
                      “1” : {transform attributes of 1}, 
                      “2” : {transform attributes of 2}, 
                      “3” : {transform attributes of 3}, 
                      "4" : {transform attributes of 4}]
    }
    // optional stuff
  }
}

// example transform attributes of dataset 0

"transform" : {
    "offset" : {"X" : 0, "Y" : 0, "Z" : 0},
    "scale" : {"X" : 1, "Y" : 1, "Z" : 1},
    "units" : {"X" : "nm", "Y" : "nm", "Z" : "nm"}
} 

// example spatial attributes of dataset 1

"transform" : {
    "offset" : {"X" : .5, "Y" : .5, "Z" : 0},
    "scale" : {"X" : 2, "Y" : 2, "Z" : 1},
    "units" : {"X" : "nm", "Y" : "nm", "Z" : "nm"}
} 

For posterity, I've written about this issue (as it pertains to the data our group works with) here

[manzt]

@sofroniewn The concept of having base + subresolutions like @manzt proposes is intriguing to me too. Ultimately for visualization purposes I want something like a single list of arrays so I guess I find that representation little simpler, but I can construct that from the later representation if I know the data is multiscale and maybe it is nice to keep that a little separate. I will think on it more, curious what others say.

I generally have the same feelings. I'm for the simplicity of the current proposal, and I wonder if my suggestion adds an extra layer of complexity unnecessarily.

@d-v-b For simplicity, I would propose a restriction of one multiscale representation per group. Groups are cheap; if you to represent 2 multiscale images, then make 2 groups.

Wouldn't this require copying the base image into a separate group? Perhaps I'm misunderstanding.

[d-v-b]

Wouldn't this require copying the base image into a separate group? Perhaps I'm misunderstanding.

The base image would be in the same group with the downscaled versions. So on the file system, it would look like this:

└── example/
    ├── .zgroup
    ├── base
    │   ├── .zarray
    │   ├── .zattrs
    │   ├── 0.0.0
    │   └── ...etc
    ├── base_downscaled
    │   ├── .zarray
    │   ├── .zattrs
    │   ├── 0.0.0
    │   └── ...etc
    ...etc

[manzt]

Apologies, I thought you were suggesting that separate groups should be created for different sampling of the same base image (e.g. gaussian and laplacian).

[d-v-b]

@manzt
this is actually my mistake -- I was not thinking at all about the use case where the same base image is used for multiple pyramids, and I agree that copying data is not ideal. I will remove / amend the "one multiscale representation per group" part of my proposal above.

[thewtex]

I would add some dataset-specific information to the group attributes: software that consumes multiscale images needs to know about how the spatial properties of each image, and on cloud storage it can be cumbersome to query each image individually;

Adding to the practical importance here: the spatial position of the first pixel is shifted in subresolutions, and the physical spacing between pixels changes also. This must be accounted for during visualization or analysis when other datasets, e.g. other images or segmentations, come into play. If this metadata is readily and independently available for every subresolution, i.e. scale factors do not need to be fetched and computations made, each subresolution image can be used independently, effortlessly, and without computational overhead.

One option is to build on the model implied by storing images in the Xarray project data structures, which has Zarr support. This enables storing metadata such as the position of the first pixel, the spacing between pixels, and identification of the array dimensions, e.g., x, y, t, so that data can be used and passed through processing pipelines and visualization tools. This is helpful because it enables distributed computing via Dask and machine learning [2] via the scikit-learn API. Xarray has broad community adoption, and it is gaining more traction lately. Of course, a model that is compatible with Xarray does not require Xarray to use the data. On the other hand, Xarray coords have more flexibility than what is required for pixels sampled on a uniform rectilinear grid, and this adds a little complexity to the layout.

Generated from this example, here is what it looks like:

.
├── level_1.zarr
│   ├── rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6
│   │   ├── 0.0.0
│   │   ├── 0.0.1
....
│   │   ├── 9.9.9
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── x
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── y
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── z
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── .zattrs
│   ├── .zgroup
│   └── .zmetadata
├── level_2.zarr
│   ├── rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6
│   │   ├── 0.0.0
│   │   ├── 0.0.1

│   │   ├── 8.9.9
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── x
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── y
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── z
│   │   ├── 0
│   │   ├── .zarray
│   │   └── .zattrs
│   ├── .zattrs
│   ├── .zgroup
│   └── .zmetadata
....
├── rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6
│   ├── 0.0.0
│   ├── 0.0.1
...
│   ├── 9.9.9
│   ├── .zarray
│   └── .zattrs
├── x
│   ├── 0
│   ├── .zarray
│   └── .zattrs
├── y
│   ├── 0
│   ├── .zarray
│   └── .zattrs
├── z
│   ├── 0
│   ├── .zarray
│   └── .zattrs
├── .zattrs
├── .zgroup
└── .zmetadata

34 directories, 62359 files

This is the layout generated by xarray.DataSet.to_zarr. It does not mean that Xarray has to be used to read and write. But, it would mean that Zarr images would be extremely easy to use via xarray. In this case, .zmetadata is generated on each subresolution so it can be used entirely independently. Due to how Xarray/Zarr handles coords, x, y, are one dimensional arrays. This results in every resolution having its own group.

The metadata looks like this:

{
    "metadata": {
        ".zattrs": {
            "_MULTISCALE_LEVELS": [
                "",
                "level_1.zarr",
                "level_2.zarr",
                "level_3.zarr",
                "level_4.zarr",
                "level_5.zarr",
                "level_6.zarr"
            ],
            "_SPATIAL_IMAGE": "rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6"
        },

Open for rest

        ".zgroup": {
            "zarr_format": 2
        },
        "level_1.zarr/.zattrs": {},
        "level_1.zarr/.zgroup": {
            "zarr_format": 2
        },
        "level_1.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zarray": {
            "chunks": [
                64,
                64,
                64
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "zstd",
                "id": "blosc",
                "shuffle": 0
            },
            "dtype": "|u1",
            "fill_value": null,
            "filters": null,
            "order": "C",
            "shape": [
                1080,
                1280,
                1280
            ],
            "zarr_format": 2
        },
        "level_1.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z",
                "y",
                "x"
            ],
            "direction": [
                [
                    1.0,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.0,
                    0.0
                ],
                [
                    0.0,
                    0.0,
                    1.0
                ]
            ],
            "units": "\u03bcm"
        },
        "level_1.zarr/x/.zarray": {
            "chunks": [
                1280
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                1280
            ],
            "zarr_format": 2
        },
        "level_1.zarr/x/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "x"
            ]
        },
        "level_1.zarr/y/.zarray": {
            "chunks": [
                1280
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                1280
            ],
            "zarr_format": 2
        },
        "level_1.zarr/y/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "y"
            ]
        },
        "level_1.zarr/z/.zarray": {
            "chunks": [
                1080
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                1080
            ],
            "zarr_format": 2
        },
        "level_1.zarr/z/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z"
            ]
        },
        "level_2.zarr/.zattrs": {},
        "level_2.zarr/.zgroup": {
            "zarr_format": 2
        },
        "level_2.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zarray": {
            "chunks": [
                64,
                64,
                64
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "zstd",
                "id": "blosc",
                "shuffle": 0
            },
            "dtype": "|u1",
            "fill_value": null,
            "filters": null,
            "order": "C",
            "shape": [
                540,
                640,
                640
            ],
            "zarr_format": 2
        },
        "level_2.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z",
                "y",
                "x"
            ],
            "direction": [
                [
                    1.0,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.0,
                    0.0
                ],
                [
                    0.0,
                    0.0,
                    1.0
                ]
            ],
            "units": "\u03bcm"
        },
        "level_2.zarr/x/.zarray": {
            "chunks": [
                640
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                640
            ],
            "zarr_format": 2
        },
        "level_2.zarr/x/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "x"
            ]
        },
        "level_2.zarr/y/.zarray": {
            "chunks": [
                640
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                640
            ],
            "zarr_format": 2
        },
        "level_2.zarr/y/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "y"
            ]
        },
        "level_2.zarr/z/.zarray": {
            "chunks": [
                540
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                540
            ],
            "zarr_format": 2
        },
        "level_2.zarr/z/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z"
            ]
        },
        "level_3.zarr/.zattrs": {},
        "level_3.zarr/.zgroup": {
            "zarr_format": 2
        },
        "level_3.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zarray": {
            "chunks": [
                64,
                64,
                64
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "zstd",
                "id": "blosc",
                "shuffle": 0
            },
            "dtype": "|u1",
            "fill_value": null,
            "filters": null,
            "order": "C",
            "shape": [
                270,
                320,
                320
            ],
            "zarr_format": 2
        },
        "level_3.zarr/rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z",
                "y",
                "x"
            ],
            "direction": [
                [
                    1.0,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.0,
                    0.0
                ],
                [
                    0.0,
                    0.0,
                    1.0
                ]
            ],
            "units": "\u03bcm"
        },
  [...]
        "rec20160318_191511_232p3_2cm_cont__4097im_1500ms_ML17keV_6/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z",
                "y",
                "x"
            ],
            "direction": [
                [
                    1.0,
                    0.0,
                    0.0
                ],
                [
                    0.0,
                    1.0,
                    0.0
                ],
                [
                    0.0,
                    0.0,
                    1.0
                ]
            ],
            "units": "\u03bcm"
        },
        "x/.zarray": {
            "chunks": [
                2560
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                2560
            ],
            "zarr_format": 2
        },
        "x/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "x"
            ]
        },
        "y/.zarray": {
            "chunks": [
                2560
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                2560
            ],
            "zarr_format": 2
        },
        "y/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "y"
            ]
        },
        "z/.zarray": {
            "chunks": [
                2160
            ],
            "compressor": {
                "blocksize": 0,
                "clevel": 5,
                "cname": "lz4",
                "id": "blosc",
                "shuffle": 1
            },
            "dtype": "<f8",
            "fill_value": "NaN",
            "filters": null,
            "order": "C",
            "shape": [
                2160
            ],
            "zarr_format": 2
        },
        "z/.zattrs": {
            "_ARRAY_DIMENSIONS": [
                "z"
            ]
        }
    },
    "zarr_consolidated_format": 1
}

Here _MULTISCALE_LEVELS prevents the need to hardcode the identifiers as suggested by @d-v-b @manzt , but it could be renamed to multiscale, etc. _ARRAY_DIMENSIONS is the key that Xarray uses in Zarr files to identify the dims.

This example is generated with itk, but it could also just as easily be generated with scikit-image, or dask-image via [1] (work in progress) or pyimagej.

[sofroniewn]

Thanks for the link to that example @thewtex! Conforming with xarray.DataSet.to_zarr where possible seems reasonable to me too.

@constantinpape, @bogovicj, @axtimwalde might also be interested in weighing in.

[jni]

👍 to flat vs hierarchical representation. Also 👍 to "multiscale".

I also like the constraint that the sub-datasets should be openable as zarr arrays by themselves. I think @thewtex's example satisfies this. Having said this, @thewtex, the xarray model looks too complex to me compared to @joshmoore's proposed spec. It would be great if it could be stripped down to its bare essentials. I agree that it's nice to have the pixel start coordinate handy, but it can also be computed after the fact, so it should be optional I think.

Last thing, which may be out of scope, but might not be: for visualisation, it is sometimes convenient to have the same array with different chunk sizes, e.g. orthogonal planes to all axes for a 3D image. I wonder if the same data/metadata layout standard can be used in these situations.

Oh and @joshmoore

anyone else who's GitHub account I've forgotten for the preliminary discussions

whose. Regret pinging me yet? =P