CF global attribute "Conventions": expose to users, and include CF standard name table version

[larsbarring]

✨ Feature Request

Currently, the global attribute Conventions is difficult to change. The default is hardcoded deep in the code:

iris/lib/iris/fileformats/netcdf/saver.py

Line 104 in 021d6ff

CF_CONVENTIONS_VERSION = "CF-1.7"

and changing the default is not as easy as one (at least I) would have hoped for, because it is "hidden" as iris.config.netcdf.conventions_override = True, which is difficult to find if I search for "global attribute Conventions" (or similar) in the documentation.

For our own use cases we would like to have better control of which CF version is actually written into the files, and moreover we would like to include additional conventions, like ACDD, Clix-meta version, as well as the CF Standard name Table version. Hence, this feature request has three main components:

• In iris.save allow users to pass the kwarg Conventions='....'.
• Change the tools/generate_std_names.py so that the lib/std_names.py includes the version number (and possibly related information such as creation date).
• New functionality (in iris.util?) to get the default Conventions string, and to get the Standard name table version number from lib/st_names.py.

Motivation

A more general motivation for giving the user (easier) access to setting the CF version is given by CF itself in Section 2.6.1:

When CF is listed with other conventions, this asserts the same full compliance with CF requirements and interpretations as if CF was the sole convention. It is the responsibility of the data-writer to ensure that all common metadata is used with consistent meaning between conventions.

[pp-mo]

Not to speak against the proposal, but I just thought it would be worth explaining here why we control this at all, and what it really means -- which I don't think is written down anywhere in user docs ...

In my view, at least, the only meaning that you can attach to the CF version which Iris writes in output files is "whatever we put in this file should be CF-compliant up to this CF version" :
Effectively, that means "the file does not include any CF encoding from CF versions later than this" -- and that's really all.

You can probably infer that an application interpreting all aspects of that CF version can understand everything that we have tried to state about the data.
But it doesn't (even) mean that the output is definitely valid under later CF versions -- though to date I think CF have totally avoided any backwards-invalidating changes.
Only that it may contain information which is not understood by earlier versions.

[larsbarring]

Yes, I understand this and do agree that this is a very reasonable Iris developer perspective. But from our side, as users of iris and building applications using the Iris api the perspective is a slightly different one. The files we produce never make use of all CF/Iris mechanisms, and most of them are fairly standard in most respects, which means that they are equally valid in later CF versions. But occasionally there are improvements and clarifications, and even corrections to the CF Conventions text that either is taken on board by Iris, or handled by us, which means that it is meaningful to indicate this in the files. Or that we have projects or users that want the data to follow some specific CF version.

The main motivation behind the request is however that we want to append additional conventions to the Conventions string, and then we have no easy/nice/clean way to get the Iris default string to start with. Add to this the other part of the request, that we would like to have access to the Standard name table version.

[pp-mo]

as users of iris and building applications using the Iris api the perspective is a slightly different one.

Thanks, understood !

I have been working in this area lately, in the context of #3325 and the project for that .
( which hopefully will make it into Iris 3.6 )
The way Iris handles Conventions attributes is certainly a bit odd + we probably want to fix some things.
I will take care how this might interact with that work.

[pp-mo]

#5423 records the table version in the header of iris.std_names, and a public global variable iris.std_names.CF_STANDARD_NAME_TABLE_VERSION.

[larsbarring]

See cf-convention/cf-conventions#500

[ESadek-MO]

@SciTools/peloton We need to discuss this further amongst ourselves, but in the meantime, have you thought about ncdata for post-save modification?