Unable to decode a date in nanoseconds

[fbriol]

What happened:

If the unit of a date, is a number of nanoseconds elapsed since a reference date, xarray cannot read this dataset and throws the exception KeyError: nanoseconds.

What you expected to happen:

As this unit is valid in CF model, it must be decoded. Today the units that are recognized go from day to microseconds in the module coding/time.py.

Minimal Complete Verifiable Example:

import numpy as np
import array as xr

xr.DataArray(
    np.arange(0, 1e10, 1e9).astype("int64"),
    attrs=dict(units="nanoseconds since 1970-01-01")).to_netcdf("/tmp/test.nc")
xr.open_dataset("/tmp/test.nc")

Environment:

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.8.3 | packaged by conda-forge | (default, Jun 1 2020, 17:43:00)
[GCC 7.5.0]
python-bits: 64
OS: Linux
OS-release: 3.10.0-957.12.2.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.10.5
libnetcdf: 4.7.4

xarray: 0.15.1
pandas: 1.0.4
numpy: 1.17.5
scipy: 1.4.1
netCDF4: 1.5.3
pydap: None
h5netcdf: 0.8.0
h5py: 2.10.0
Nio: None
zarr: 2.4.0
cftime: 1.1.3
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.18.1
distributed: 2.18.0
matplotlib: 3.2.1
cartopy: None
seaborn: None
numbagg: None
setuptools: 47.3.1.post20200616
pip: 20.1.1
conda: 4.8.3
pytest: 5.4.3
IPython: 7.15.0
sphinx: None

[dcherian]

Thanks @fbriol .

We would happily take a PR for this.

[shoyer]

We would absolutely accept a pull request for this, but I'm not 100% sure that nanoseconds are valid per CF conventions. CF conventions references udunits.dat, which as far as I can tell includes no units smaller than 1 second:
https://www.unidata.ucar.edu/software/udunits/udunits-1/udunits.txt

[jthielen]

We would absolutely accept a pull request for this, but I'm not 100% sure that nanoseconds are valid per CF conventions. CF conventions references udunits.dat, which as far as I can tell includes no units smaller than 1 second:
https://www.unidata.ucar.edu/software/udunits/udunits-1/udunits.txt

Nanoseconds (and seconds with any other SI prefix from yocto to yotta) are indeed valid in the CF conventions. For reference, there is this table in v1.8 of the conventions, as well as the SI prefix and SI base unit databases of UDUNITS.

[hmaarrfk]

i ran into this problem trying to round trip time to the nanosecond (even though i don't need it, sub micro second would be nice)

but unfrotunately, you run into the fact that cftime doesn't support nanoseconds
https://github.com/Unidata/cftime/blob/master/cftime/_cftime.pyx

Seems like they discussed a nanosecond issue a while back too
Unidata/cftime#77

Their ultimate point was that there was little point in having precision down to the nano second given that python datetime objects only have microseconds. I guess they are right.

[stale]

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

[spencerkclark]

This issue was addressed as part of #4684.