Coordinate arrays of type object can not be written, read, and then written again with zarr<3

[joshua-gould]

What happened?

Coordinate arrays of type object can not be written, read, and then written again.

Minimal Complete Verifiable Example

import xarray as xr
import numpy as np

data = xr.DataArray(data=np.zeros((2, 2)),
                    dims=['x', 'y'],
                    coords=dict(
                        y=np.array(['a', 'b'],
                                   dtype=object)))
data.to_zarr("test.zarr", mode="w")
data = xr.open_zarr("test.zarr")
data.to_zarr("test2.zarr", mode="w")
# File "numcodecs/vlen.pyx", line 104, in numcodecs.vlen.VLenUTF8.encode
# TypeError: expected unicode string, found 2

MVCE confirmation

• Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
• Complete example — the example is self-contained, including all data and the text of any traceback.
• Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
• New issue — a search of GitHub Issues suggests this is not a duplicate.
• Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Environment

xarray==2025.9.0

[ianhi]

Hi @joshua-gould can you please give a full traceback of the error and also the output of xr.show_versions()?

I tried replicating this and your code worked for me. So until we reproduce the error it's hard to help.

[joshua-gould]

Thank you for investigating. Version info and full stack trace are below.

xr.show_versions()

commit: None
python: 3.12.11 | packaged by conda-forge | (main, Jun 4 2025, 14:38:53) [Clang 18.1.8 ]
python-bits: 64
OS: Darwin
OS-release: 24.6.0
machine: arm64
processor: arm
byteorder: little
LC_ALL: None
LANG: None
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.14.6
libnetcdf: None
xarray: 2025.9.0
pandas: 2.3.2
numpy: 1.26.4
scipy: 1.16.1
netCDF4: None
pydap: None
h5netcdf: None
h5py: 3.14.0
zarr: 2.18.7
cftime: None
nc_time_axis: None
iris: None
bottleneck: None
dask: 2025.7.0
distributed: 2025.7.0
matplotlib: 3.10.3
cartopy: None
seaborn: 0.13.2
numbagg: None
fsspec: 2025.7.0
cupy: None
pint: 0.24.4
sparse: None
flox: None
numpy_groupies: None
setuptools: 80.9.0
pip: 25.2
conda: None
pytest: 8.4.1
mypy: None
IPython: 9.5.0
sphinx: None

Traceback (most recent call last):
  File "<ipython-input-46-957ecf8d147a>", line 11, in <module>
    data.to_zarr("test2.zarr", mode="w")
  File "python3.12/site-packages/xarray/core/dataset.py", line 2364, in to_zarr
    return to_zarr(  # type: ignore[call-overload,misc]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python3.12/site-packages/xarray/backends/api.py", line 2463, in to_zarr
    dump_to_store(dataset, zstore, writer, encoding=encoding)
  File "python3.12/site-packages/xarray/backends/api.py", line 2157, in dump_to_store
    store.store(variables, attrs, check_encoding, writer, unlimited_dims=unlimited_dims)
  File "python3.12/site-packages/xarray/backends/zarr.py", line 1056, in store
    self.set_variables(
  File "python3.12/site-packages/xarray/backends/zarr.py", line 1280, in set_variables
    writer.add(v.data, zarr_array, region)
  File "python3.12/site-packages/xarray/backends/common.py", line 397, in add
    target[region] = source
    ~~~~~~^^^^^^^^
  File "python3.12/site-packages/zarr/core.py", line 1451, in __setitem__
    self.set_basic_selection(pure_selection, value, fields=fields)
  File "python3.12/site-packages/zarr/core.py", line 1547, in set_basic_selection
    return self._set_basic_selection_nd(selection, value, fields=fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python3.12/site-packages/zarr/core.py", line 1937, in _set_basic_selection_nd
    self._set_selection(indexer, value, fields=fields)
  File "python3.12/site-packages/zarr/core.py", line 1990, in _set_selection
    self._chunk_setitem(chunk_coords, chunk_selection, chunk_value, fields=fields)
  File "python3.12/site-packages/zarr/core.py", line 2263, in _chunk_setitem
    self._chunk_setitem_nosync(chunk_coords, chunk_selection, value, fields=fields)
  File "python3.12/site-packages/zarr/core.py", line 2273, in _chunk_setitem_nosync
    self.chunk_store[ckey] = self._encode_chunk(cdata)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "python3.12/site-packages/zarr/core.py", line 2389, in _encode_chunk
    chunk = f.encode(chunk)
            ^^^^^^^^^^^^^^^
  File "numcodecs/vlen.pyx", line 104, in numcodecs.vlen.VLenUTF8.encode
TypeError: expected unicode string, found 2

[ianhi]

Thanks very much, this is helpful!

This is likely a bug in zarr rather than Xarray. The quickest solution for you is like to ugrade to zarr-python version 3 if possible. You can still write to zarr format 2 using zarr_format argument

[max-sixty]

tbc, this still isn't a MCVE — review the links in the issue template for more details

(thanks for managing @ianhi ! v helpful!)

[keewis]

you can also clear the zarr 2 filters encoding:

ds["y"].encoding.pop("filters", None)

after which you should be able to write it back to disk. This does support the hypothesis of this being a zarr (or numcodecs?) issue, though.

[joshua-gould]

The issue seems to be that the vlen-utf8 filter is added twice for the second write:

test.zarr/y/.zarray

{
    "chunks": [
        2
    ],
    "compressor": {
        "blocksize": 0,
        "clevel": 5,
        "cname": "lz4",
        "id": "blosc",
        "shuffle": 1
    },
    "dtype": "|O",
    "fill_value": null,
    "filters": [
        {
            "id": "vlen-utf8"
        }
    ],
    "order": "C",
    "shape": [
        2
    ],
    "zarr_format": 2
}

test2.zarr/y/.zarray

{
    "chunks": [
        2
    ],
    "compressor": {
        "blocksize": 0,
        "clevel": 5,
        "cname": "lz4",
        "id": "blosc",
        "shuffle": 1
    },
    "dtype": "|O",
    "fill_value": null,
    "filters": [
        {
            "id": "vlen-utf8"
        },
        {
            "id": "vlen-utf8"
        }
    ],
    "order": "C",
    "shape": [
        2
    ],
    "zarr_format": 2
}

[ianhi]

Here is a zarr only reproducer:

# /// script
# requires-python = ">=3.9"
# dependencies = [
#     "zarr<3",
#     "numcodecs",
#     "numpy",
# ]
# ///

import numpy as np
import zarr
from numcodecs import VLenUTF8

# Step 1: Create a string array - zarr v2 auto-adds VLenUTF8 filter
z1 = zarr.open_array("test1.zarr", mode="w", shape=(2,), dtype=str, chunks=(2,))
z1[:] = np.array(["a", "b"], dtype=object)
print(f"First array filters: {z1.filters}")

# Step 2: Create another array with VLenUTF8 explicitly in filters
# Zarr v2 will add another VLenUTF8, causing duplication
z2 = zarr.open_array(
    "test2.zarr", mode="w", shape=(2,), dtype=str, chunks=(2,), filters=[VLenUTF8()]
)
print(f"Second array filters: {z2.filters}")

# Step 3: Writing data fails due to double encoding
z2[:] = np.array(["a", "b"], dtype=object)

So it's possible xarray could be more thoughtful about what codecs it passes, but even still zarr should at minimum throw a different error