Corrupt NetCDF files for ABI-L2-DSRF data from June 2024

[jamesmhbarry]

I had a problem using goes_single_point_timerange - some of the NetCDF files downloaded do not open, but give an error:

OSError: [Errno -101] NetCDF: HDF error

Since I use "goes_timerange" to download the files, and then I use the single_point_timerange to extract data for different points on a day-by-day basis, if some of the files are corrupt, the whole day does not get extracted. To this end I modified the script as follows:

error_files = []
for f in df["file"].to_list():
    try:
        xr.open_dataset(str(config["timerange"]["save_dir"]) + "/" + f)
    except:
        error_files.append(f)
        
file_list = [f for f in df["file"].to_list() if f not in error_files]
preprocessed_ds = xr.open_mfdataset(
    [str(config["timerange"]["save_dir"]) + "/" + f for f in file_list],
    concat_dim="t",
    combine="nested",
    preprocess=partial_func,
)

This way, one opens each file first as a check, before calling open_mfdataset

I am not sure what the root cause of the HDF error is?