Integration of risk trajectories within climada

[spjuhel]

This discussion relates to PR #1037 which aims at implementing RiskTrajectory objects to ease the evaluation of risk / impact in between two points in time.

Overview of the new functionalities, decisions taken, and remarks

The following goes deeper in the technical aspects than the notebook.

Snapshot class

Its purpose is to hold Exposures, Hazard and ImpactFuncSet objects for a specific datetime.date.
It is intended as a data holding class.

It may also be useful as a "container" class in the future?

There are almost no advanced functionalities at the moment. If you have ideas feel free to list them!

Applying measures

Snapshot.apply(meas) can be used to apply a Measure object, thus creating a new Snapshot object with relevant changes in Exposures, Hazard and ImpactFuncSet, as well as setting its measure attribute to meas.

Note that this is functional only with the updated Measures objects within feature/cb_refactoring.

RiskPeriod class

Its purpose is to compute the impact in between two given snapshots. It is intended as an internal computing class, and might even be made private.

Given two snapshots present and future, and a time frequency freq (say a year) we compute the impacts at a freq interval between present and future. This is done by computing four Impact objects corresponding to the four possible combination of Exposures and Hazard between present and future (Each time using the impact functions associated with the Exposures used). To these four impacts possible risk transfer is applied (not tested yet).

Then from these four impacts, two lists of impacts matrices are computed using linear interpolation (for each year for instance):

• The interpolation from present Exposures to future Exposures with present Hazard.
• The interpolation from present Exposures to future Exposures with future Hazard.

These two lists are used by RiskTrajectory to compute risk metrics (explained further ahead)

ImpactFuncSet handling

To allow different impact functions between present and future, the future Exposures can be assigned to different impact functions. The ImpactFuncSet however, needs to be the same for both, and basically contain all the impact functions. For user-friendliness (but that can be argued against), if different ImpactFuncSet are used, they are merged together, using the functions from future when conflicting functions (and warning the user).

Interpolation efficiency

interpolate_imp_mat should a priori be the main source for computation requirement, if you have ideas on how to improve its efficiency these are most welcomed!

RiskTrajectory class

It is the main class for this module, and builds on the previous two. It is instantiated with at least a list of Snapshot, from which it generates a list of RiskPeriod by iterating it pairwise, and possibly:

• A risk discount (DiscRate) object to be applied to compute net present (discounted) risk value at the different dates.
• A set of metrics to compute, by default all currently possible: ["aai", "rp"].
• A set of return periods for the "rp" metric
• The possibility to compute metrics on a group basis (for each group_id defined in the Exposures objects)
• The possibility to add risk transfer (not yet tested)

How are metrics computed

Basically all the "magic" is in the impact_mixer(): we compute the different metrics for each of the impact matrix from the two interpolation lists, and compute the linear interpolation of the metric from this. (I can go in more details if wanted).

The results are saved in a private tidy DataFrame (where one row == one value), which is only recomputed if relevant.

What is accessible

Most attribute are private, or read-only to avoid having an inconsistent object.

Mostly the metrics (either on a per date or per risk period basis) are accessible, via per_date_risk_metrics and total_risk_metrics. Note that these two apply the discount rate if it is not None, but that the underlying private methods accept a boolean argument for that.

[spjuhel]

Comments from @chahank from the PR (and responses to):

Small point, I would use datetime.datetime objects instead of datetime.data since sometimes hazards are used in the context of weather forecasts with sub-daily resolution.

This is non-trivial, and using datetime.date is a bit on purpose, as I don't think we want to interpolate impacts on a sub-daily resolution.

I would look for another name than freq. The word frequency is already used to describe the probability/frequency of hazard events. I think this would be quite confusing. Maybe intervals ?

Indeed, intervals is probably better.

I think it is a bit unclear to me which class in the end contains the data from the risk trajectories.

The Snapshots hold the points in time, the RiskPeriods compute and hold the interpolated impact matrices. The RiskTrajectorys compute and hold the risk metrics

How about per_datetime_risk_metrics as I think you propose once too? Maybe too complicated ...

I thought it could be too long, but was also in favour.

Add a check that group_id are identical for Exposures or handle that case (how?)

What is the reason for this? Can one not simply compute the impact metrics per group if needed and that is it? I am not sure to see the issues at a first glance (I am sure there are some :D)

I thought of a case where it can go wrong, but I realize now that it shouldn't be a problem.

[chahank]

The Snapshots hold the points in time, the RiskPeriods compute and hold the interpolated impact matrices. The RiskTrajectorys compute and hold the risk metrics

Have you considered separating data and computations into two classes? It is not necessary, but has proven useful for instance when making the ImpactCalc and Impact classes. It reduces a bit the sizes, and makes data handling a bit more consistent (object is not modified in place). But there are also reasons to keep them as one of course.

[chahank]

This is non-trivial, and using datetime.date is a bit on purpose, as I don't think we want to interpolate impacts on a sub-daily resolution.

Sorry to insist. But why is it non-trivial?

And why don't we want to interpolate impacts on a sub-daily resolution? What is special about days?

[spjuhel]

Sorry to insist. But why is it non-trivial?

I might have been wrong here, I thought the index for pandas were different, but they are the same, so it may be more straightforward than I thought.

And why don't we want to interpolate impacts on a sub-daily resolution? What is special about days?

Snapshot represent a state of hazard, exposure and vulnerability at a certain point in time. On a yearly or monthly basis, I see how it could be valuable to give a sense of the evolution of the risk in between those points. I am unsure it makes sense on a weekly or daily basis, but in any case the implementation stays the same.

I fail to see a case where you would want to interpolate the risk matrix in between two hours ?
For forecast, you should have the actual data ? What would be the benefit or use case of going to a finer time resolution via linear interpolation ?

I have in mind the case of a (forecasted or not) storm or TC going over say a city, do we really want to enable computing the impact between two snapshots of that event at different hours as the linear interpolation of the impact at each point ?

[chahank]

I understand. It is a good point that probably even anything below multi-year interpolation would find rare applications.

This would more come when one has a timeseries time hazard (such as hourly forecasts or climate model projections).

If you do not use datetime in snapshots, could you then still accomodate for time series in the RiskTrajectory class?

[spjuhel]

(ping @emanuel-schmid)

Regarding the ImpactFuncSet handling, in the end what I am now choosing to do is the following:
Snapshot keep their impfset attribute (I don't see any solution that would remain user-friendly otherwise, and it makes sense conceptually, as you define a vulnerability for a specific point in time).

Instead of merging them in RiskPeriod (pair of Snapshot, part of a trajectory) in a not elegant way, I am now collecting all of them and merging them in the RiskTrajectory (the whole trajectory, possibly multiple RiskPeriod), such that each Snapshot keeps its own impfset, and the RiskTrajectory one is their regroupment.

Furthermore, I am only raising an error, if there is a "really conflicting" impact function (i.e., one that is strictly different for the exact same hazard and id)

[spjuhel]

Actually we found an even better solution where each Snapshot has its own impfset and there is no restriction at all.

[spjuhel]

Notes (for myself mostly):

• Exposures need to have the same shape (Otherwise interpolation is not possible)
• Hazard need not, as we interpolate impact matrices only at constant hazard and changing exposure

As such, we cannot compute the "risk components" (Change in risk due to change in Hazard, change in Exposure and the interaction term (change in both), but only the change due to Exposure and change due to Hazard (With interaction).

[spjuhel]

Would it make more sense to linearly interpolate the log of the impact matrices, and then power it back when we assume the exposure to grow exponentially?

[chahank]

But we do not always assume an exponential growth right?

[spjuhel]

Not necessarily no. But that is what is currently in the example.

[chahank]

OK good, I would be careful to not make exponential growth the default as this is a quite questionable assumption for the future.

[spjuhel]

There is no "default", the way users change their exposure is up to them, we did not implement any helper functions.
But the current example uses exponential growth to derive future exposure from present one. We can (and probably will change that, though I don't know what to put instead)

[timschmi95]

Beta-testing feedback (first test)

This is based only on running the climada_trajectories.ipynb tutorial, and not yet a testing with my own data or an analysis of the code from the module.

First off, it runs through smoothly for me and I really like the Class structure with Snapshots being combined into RiskTrajectories :)

I have two main questions.

1. You use a slightly different impact function for the present-day and future snapshot in the first example, but the impact-function change is not shown in the figures. So does the module actually consider the difference in impact functions, and if yes, are the changes also linearly interpolated over time (for each hazard intensity value)? And if no, why does each snapshot have an impact function and not just one per RiskTrajectory?

2. To which category (hazard or exposure contribution) are combined changes counted?
   E.g. we have one gridcell with an exposure of 1, a single event with a hazard intensity corresponding to a 10% impact, and thus a present-day damage = 1 * 10% = 0.1 USD. Then in the future the exposure is 2, and the hazard a single event with an intensity corresponding to 20% impact, and thus: 2 * 20% = 0.4 USD total damage
   In this case the hazard contribution alone is a doubling (i.e. an additional 0.1USD) and so is the exposure contribution. The last 0.1 USD only comes from the combination.
   o How is this currently split between hazard and exposure contribution?
   o Should it be separately quantified or not?

And two small comments regarding plots:

1. For the waterfall plot: The 2 middle columns should be renamed, I would suggest without the Date. E.g “Exposure contribution” and “Hazard contribution” or similar.
2. For the plot with multiple snapshots (which is a really cool new feature btw): It's probably worth it to add the snapshot times as dotted vlines with a label or something similar, so it is visually clear that the risk is re-based.