Skip to content

StandardScaler implementation #871

Description

@mathause

The StandardScaler standardizes data by its mean and standard deviation, i.e. $z = (x - \mu) / \sigma$. In #838 @sarasita wrote a SklearnXarrayTransformer class that can wrap transformer classes from sklearn - including StandardScaler.

This approach has the advantage that we can wrap other transformer classes, however it is not a simple class. If we don't need (many) other transformers I think we are better off implementing it ourselves. This would make it faster, simpler, less code & easier to understand, and it's a more natural xarray- (numpy-) like way of computing.

The details can be discussed but it would look approximately like this:

import xarray as xr


class StandardScaler:

    def __init__(self):

        self.params_: None | xr.Dataset = None

    def _assert_fitted(self):

        if self.params_ is None:
            raise ValueError("Nof fitted")

    def fit(self, data: xr.DataArray, dim) -> None:

        mean = data.mean(dim=dim)
        std = data.std(dim=dim)

        self.params_ = xr.Dataset({"mean_": mean, "std_": std})

    def transform(self, data: xr.DataArray) -> xr.DataArray:

        self._assert_fitted()

        return (data - self.params_.mean_) / self.params_.std_

    def fit_transform(self, data: xr.DataArray, dim) -> xr.DataArray:

        self.fit(data, dim)
        return self.transform(data)

    def inverse_transform(self, data: xr.DataArray) -> xr.DataArray:

        self._assert_fitted()

        return data * self.params_.std_ + self.params_.mean_

cc @sarasita

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions