feat(skore): Create a "custom" metrics registry under each report

[thomass-dev]

Caution

Just a notebook for reflection for now.

Points to cover:

• redefine __call__ in each metric subclass to provide a more user-friendly API
  • equivalent of the current implementation with r2 calling _r2
• show the docstr from the metric r2 = R2(); help(r2)
• show the helper
  • standard metrics
  • custom metrics
• deal with CrossValidation (only metrics)

---

Closes https://github.com/probabl-ai/skore-hub/issues/940.

[github-actions]

Coverage Report for skore/

File	Stmts	Miss	Cover	Missing
skore/src/skore
__init__.py	24	0	100%
_config.py	31	0	100%
exceptions.py	4	4	0%	4, 15, 19, 23
skore/src/skore/_sklearn
__init__.py	6	0	100%
_base.py	199	14	92%	46, 59, 128, 131, 184, 187–188, 190–193, 226, 229–230
find_ml_task.py	61	0	100%
types.py	28	1	96%	30
skore/src/skore/_sklearn/_comparison
__init__.py	7	0	100%
feature_importance_accessor.py	35	2	94%	93, 117
metrics_accessor.py	179	3	98%	169, 249, 1212
report.py	108	0	100%
utils.py	57	0	100%
skore/src/skore/_sklearn/_cross_validation
__init__.py	9	0	100%
data_accessor.py	45	3	93%	134, 137, 140
feature_importance_accessor.py	24	0	100%
metrics_accessor.py	183	1	99%	242
report.py	136	1	99%	490
skore/src/skore/_sklearn/_estimator
__init__.py	9	0	100%
data_accessor.py	66	1	98%	82
feature_importance_accessor.py	168	2	98%	258–259
metrics_accessor.py	388	5	98%	331, 400, 404, 419, 454
report.py	166	2	98%	449–450
skore/src/skore/_sklearn/_plot
__init__.py	3	0	100%
base.py	106	6	94%	61–62, 247–249, 253
utils.py	77	0	100%
skore/src/skore/_sklearn/_plot/data
__init__.py	2	0	100%
table_report.py	185	1	99%	706
skore/src/skore/_sklearn/_plot/metrics
__init__.py	6	0	100%
confusion_matrix.py	94	0	100%
feature_importance_coefficients_display.py	71	21	70%	116–119, 121, 140, 146–152, 155, 161–165, 170–171
metrics_summary_display.py	8	0	100%
precision_recall_curve.py	175	2	98%	441–442
prediction_error.py	233	5	97%	179, 186, 423, 506, 706
roc_curve.py	314	9	97%	263, 455, 578, 583, 684, 689, 693, 762, 902
skore/src/skore/_sklearn/train_test_split
__init__.py	0	0	100%
train_test_split.py	58	0	100%
skore/src/skore/_sklearn/train_test_split/warning
__init__.py	8	0	100%
high_class_imbalance_too_few_examples_warning.py	19	1	94%	83
high_class_imbalance_warning.py	20	0	100%
random_state_unset_warning.py	10	0	100%
shuffle_true_warning.py	9	0	100%
stratify_is_set_warning.py	10	0	100%
time_based_column_warning.py	21	0	100%
train_test_split_warning.py	3	0	100%
skore/src/skore/_utils
__init__.py	6	2	66%	8, 13
_accessor.py	90	3	96%	34, 146, 190
_cache.py	37	0	100%
_environment.py	27	1	96%	40
_fixes.py	8	0	100%
_index.py	5	0	100%
_logger.py	22	4	81%	15–17, 19
_measure_time.py	10	0	100%
_parallel.py	38	3	92%	23, 33, 124
_patch.py	21	12	42%	30, 35–39, 42–43, 46–47, 58, 60
_progress_bar.py	46	0	100%
_repr_html.py	8	0	100%
_show_versions.py	38	0	100%
_testing.py	56	0	100%
skore/src/skore/project
__init__.py	2	0	100%
project.py	57	0	100%
summary.py	75	1	98%	120
widget.py	187	0	100%
TOTAL	4098	110	97%

Tests	Skipped	Failures	Errors	Time
1175	5 💤	0 ❌	0 🔥	4m 23s ⏱️

[github-actions]

Documentation preview @ 483d21e

[thomass-dev]

from sklearn.datasets import make_regression
from sklearn.linear_model import LinearRegression
from sklearn.metrics import get_scorer
from sklearn.model_selection import train_test_split

from skore._sklearn._metric.registry import NewEstimatorReport

X, y = make_regression(random_state=42)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)

report = NewEstimatorReport(
    LinearRegression(),
    X_train=X_train,
    y_train=y_train,
    X_test=X_test,
    y_test=y_test,
)

def my_business_metric(*args, **kwargs):
    return 1

report.metrics.append(get_scorer("explained_variance"))
report.metrics.append("neg_mean_squared_error")
report.metrics.append(my_business_metric)

print(report.metrics)
print(report.metrics.r2())
print(report.metrics.explained_variance_score())
print(report.metrics.mean_squared_error())
print(report.metrics.my_business_metric())