Jupyter Notebook for EOBS UQ1

[BeatrizLopesi]

Notebook description

Hi @malmans2,
We have been testing the Jupyter notebook UQ1 with the new toolbox and encountered an error that we cannot understand. When we try to compute the rooling mean inside the download function an error appears.
Here are the functions we are using:

Define request

collection_id = "insitu-gridded-observations-europe"
request = {
"variable": ["wind_speed"],
"grid_resolution": "0_25deg",
"period": "full_period",
"version": ["30_0e"],
"area": area,
}

Define transform function to reduce downloaded data

def dayofyear_reindex(ds, years_start, years_stop):
# 15-day rolling mean
ds_rolled = ds.rolling(time=15, center=True).mean()

# Extract periods
datasets = []
for year_start, year_stop in zip(years_start, years_stop):
    period = f"{year_start}-{year_stop}"
    ds_masked = ds_rolled.where(
        (ds_rolled["time"].dt.year >= year_start)
        & (ds_rolled["time"].dt.year <= year_stop),
        drop=True,
    )
    datasets.append(
        ds_masked.groupby("time.dayofyear").mean().expand_dims(period=[period])
    )
ds_dayofyear = xr.merge(datasets)

# Add season (pick any leap year)
season = xr.DataArray(
    pd.to_datetime(ds_dayofyear["dayofyear"].values - 1, unit="D", origin="2008"),
).dt.season
return ds_dayofyear.assign_coords(season=("dayofyear", season.values))

def spatial_weighted_mean(ds):
ds = ds.resample(time="1D").mean(keep_attrs=True)
return diagnostics.spatial_weighted_mean(ds)

Periods

dataarrays = []
for reduction in ("mean", "spread"):
print(f"{reduction=}")
da = download.download_and_transform(
collection_id,
request | {"product_type": f"ensemble_{reduction}"},
transform_func=dayofyear_reindex,
transform_func_kwargs={"years_start": years_start, "years_stop": years_stop},
)["fg"]
dataarrays.append(da.rename(reduction))
da.attrs["long_name"] += f" {reduction}"
ds_periods = xr.merge(dataarrays)

We want to apply that function to these periods:

Define climatology periods - ToI

years_start = [1951, 1961, 1971, 1981, 1991]
years_stop = [1980, 1990, 2000, 2010, 2020]
colors = ["deepskyblue", "green", "gold", "darkorange", "red"]

and this is the error: ValueError: Moving window (=15) must between 1 and 14, inclusive

For some reason, when I try to do this with only two periods (e.g. years_start = [1981, 1991], years_stop = [2010, 2020]) it works.

Do you have any ideia of what can be causing the problem?
Thank you for your help,
Beatriz Lopes

Notebook link or upload

C3S2_D520-1.5.3.2_Quality_Assessment_User_Questions_EOBS_UQ1_v1.13.zip

Anything else we need to know?

No response

Environment

[malmans2]

Hi @BeatrizLopesi,

It looks like a bug in one of the underlying libraries. See: pydata/xarray#10115

The workaround is easy though. Could you please try this?

# Define transform function to reduce downloaded data
def dayofyear_reindex(ds, years_start, years_stop):
    ds = ds.chunk(time=-1)  # See: https://github.com/pydata/xarray/issues/10115

    # 15-day rolling mean
    ds_rolled = ds.rolling(time=15, center=True).mean()

    # Extract periods
    datasets = []
    for year_start, year_stop in zip(years_start, years_stop):
        period = f"{year_start}-{year_stop}"
        ds_masked = ds_rolled.where(
            (ds_rolled["time"].dt.year >= year_start)
            & (ds_rolled["time"].dt.year <= year_stop),
            drop=True,
        )
        datasets.append(
            ds_masked.groupby("time.dayofyear").mean().expand_dims(period=[period])
        )
    ds_dayofyear = xr.merge(datasets)

    # Add season (pick any leap year)
    season = xr.DataArray(
        pd.to_datetime(ds_dayofyear["dayofyear"].values - 1, unit="D", origin="2008"),
    ).dt.season
    return ds_dayofyear.assign_coords(season=("dayofyear", season.values))

I only added this: ds = ds.chunk(time=-1)

[BeatrizLopesi]

Hi @malmans2,
That fixed the problem.
Thank you