Possible `CAN'T PARSE: File ` bug with `check_cmip7_packing`

[mo-gill]

I have been using check_cmip7_packing successfully for many files and it has been working great. However, on one particular file it fails with a CAN'T PARSE exit code of 5.

I have checked this file via a few different methods and it appears to be normal, and have successfully run the check on other similar files that have worked fine.

After having a dig the problem appears to occur in pyfive. Possibly some sort of edge case?

>>> import pyfive
>>> f = pyfive.File("filename")
>>> f.consolidated_metadata
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<redacted>/lib/python3.10/site-packages/pyfive/high_level.py", line 272, in consolidated_metadata
    if isinstance(f[ds], Dataset):
  File "<redacted>/lib/python3.10/site-packages/pyfive/high_level.py", line 64, in __getitem__
    return self.__getitem_lazy_control(y, noindex=False)
  File "<redacted>/lib/python3.10/site-packages/pyfive/high_level.py", line 129, in __getitem_lazy_control
    return Dataset(obj_name, DatasetID(dataobjs, noindex=noindex), self)
  File "<redacted>/lib/python3.10/site-packages/pyfive/h5d.py", line 96, in __init__
    self._meta = DatasetMeta(dataobject)
  File "<redacted>/lib/python3.10/site-packages/pyfive/h5d.py", line 699, in __init__
    self.attributes = dataobject.get_attributes()
  File "<redacted>/lib/python3.10/site-packages/pyfive/dataobjects.py", line 155, in get_attributes
    more_attrs = self._get_attributes_from_attr_info(attrs, attr_info)
  File "<redacted>/lib/python3.10/site-packages/pyfive/dataobjects.py", line 181, in _get_attributes_from_attr_info
    name, value = self._parse_attribute_msg(data,0)
  File "<redacted>/lib/python3.10/site-packages/pyfive/dataobjects.py", line 197, in _parse_attribute_msg
    version = struct.unpack_from('<B', buffer, offset)[0]
struct.error: unpack_from requires a buffer of at least 1 bytes for unpacking 1 bytes at offset 0 (actual buffer size is 0)

We have hosted the file on JASMIN if you would be able to investigate this it would be much appreciated:
~cdds/tmp/pyfive_error/

I'm not sure which repo would be best to post this issue so will also make an issue referencing this one on https://github.com/NCAS-CMS/pyfive/issues

Thankyou!

[davidhassell]

Just an update for this issue - the cause in pyfive has been found (NCAS-CMS/pyfive#158, many thanks to @kmuehlbauer!), and a fix is in hand.

[kmuehlbauer]

Fix is applied in NCAS-CMS/pyfive#160 and seem to work for the file in question.

[davidhassell]

A pin for a new minimum version of pyfive that includes the fix - which is expected to be released by tomorrow (I think) - will be added to check_cmip7_packing.

We probably won't release a new check_cmip7_pack version until January 2026, but the existing version will work with the updated pyfive, of course.

[davidhassell]

OK - January has come early, and this is now fixed in v0.6 (via #23). It'll be in PyPi/conda very shortly ...