Skip to content

Commit 191e21f

Browse files
ashwinvaidya17jpcbertoldosamet-akcay
authored
Add PIMO (#2329)
* PIMO (#1726) * update Signed-off-by: jpcbertoldo <[email protected]> * test binclf curves numpy and numba and fixes Signed-off-by: jpcbertoldo <[email protected]> * correct som docstrings Signed-off-by: jpcbertoldo <[email protected]> * torch interface and tests Signed-off-by: jpcbertoldo <[email protected]> * torch interface and tests Signed-off-by: jpcbertoldo <[email protected]> * constants regrouped in dataclass as class vars Signed-off-by: jpcbertoldo <[email protected]> * result class was unneccesary for per_image_binclf_curve Signed-off-by: jpcbertoldo <[email protected]> * factorize function _get_threshs_minmax_linspace Signed-off-by: jpcbertoldo <[email protected]> * small docs fixes Signed-off-by: jpcbertoldo <[email protected]> * add pimo numpy version and test Signed-off-by: jpcbertoldo <[email protected]> * move validation Signed-off-by: jpcbertoldo <[email protected]> * add `shared_fpr_metric` option Signed-off-by: jpcbertoldo <[email protected]> * add pimo torch functional version and test Signed-off-by: jpcbertoldo <[email protected]> * add torchmetrics interface and test Signed-off-by: jpcbertoldo <[email protected]> * renames and put things in init Signed-off-by: jpcbertoldo <[email protected]> * validate inputs in result objects Signed-off-by: jpcbertoldo <[email protected]> * result objects to from dict and tests Signed-off-by: jpcbertoldo <[email protected]> * add save and load methods to result objects and test Signed-off-by: jpcbertoldo <[email protected]> * refactor validations and minor changes Signed-off-by: jpcbertoldo <[email protected]> * test result objects' properties Signed-off-by: jpcbertoldo <[email protected]> * minor refactors Signed-off-by: jpcbertoldo <[email protected]> * add missing docstrings Signed-off-by: jpcbertoldo <[email protected]> * minore vocabulary fix for consistency Signed-off-by: jpcbertoldo <[email protected]> * add per image scores statistics and test it Signed-off-by: jpcbertoldo <[email protected]> * refactor constants notation Signed-off-by: jpcbertoldo <[email protected]> * add stats tests and test it Signed-off-by: jpcbertoldo <[email protected]> * change the meaning of AUPIMO.num_thresh Signed-off-by: jpcbertoldo <[email protected]> * interface to format pairwise test results Signed-off-by: jpcbertoldo <[email protected]> * improve doc Signed-off-by: jpcbertoldo <[email protected]> * add optional `paths` to result objects and some minor fixes and refactors Signed-off-by: jpcbertoldo <[email protected]> * remove frozen from dataclasses and some done todos Signed-off-by: jpcbertoldo <[email protected]> * review headers Signed-off-by: jpcbertoldo <[email protected]> * doc modifs Signed-off-by: jpcbertoldo <[email protected]> * refactor `score_less_than_thresh` in `_binclf_one_curve_python` Signed-off-by: jpcbertoldo <[email protected]> * correct license comments Signed-off-by: jpcbertoldo <[email protected]> * fix doc Signed-off-by: jpcbertoldo <[email protected]> * numba as extra requirement Signed-off-by: jpcbertoldo <[email protected]> * refactor copyrights from jpcbertoldo Signed-off-by: jpcbertoldo <[email protected]> * remove from __future__ import annotations Signed-off-by: jpcbertoldo <[email protected]> * refactor validations names Signed-off-by: jpcbertoldo <[email protected]> * dedupe file path validation Signed-off-by: jpcbertoldo <[email protected]> * fix tests Signed-off-by: jpcbertoldo <[email protected]> * Add todo Signed-off-by: jpcbertoldo <[email protected]> * refactor enums Signed-off-by: jpcbertoldo <[email protected]> * only logger.warning Signed-off-by: jpcbertoldo <[email protected]> * refactor test imports Signed-off-by: jpcbertoldo <[email protected]> * refactor docs Signed-off-by: jpcbertoldo <[email protected]> * refactor some docs Signed-off-by: jpcbertoldo <[email protected]> * correct pre commit errors Signed-off-by: jpcbertoldo <[email protected]> * remove author tag Signed-off-by: jpcbertoldo <[email protected]> * add thrid party program Signed-off-by: jpcbertoldo <[email protected]> * Update src/anomalib/metrics/per_image/pimo.py * move HAS_NUMBA Signed-off-by: jpcbertoldo <[email protected]> * remove PIMOSharedFPRMetric Signed-off-by: jpcbertoldo <[email protected]> * make torchmetrics compute avg by dft Signed-off-by: jpcbertoldo <[email protected]> * pre-commit hooks corrections Signed-off-by: jpcbertoldo <[email protected]> * correct numpy.trapezoid Signed-off-by: jpcbertoldo <[email protected]> --------- Signed-off-by: jpcbertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]> * 🗑️ Remove numba (#2313) * remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * add third-party-programs.txt Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]> * 🗑️ Remove unused methods (#2315) * remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]> * PIMO: Port Numpy → Torch (#2316) * remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> * replace numpy with torch Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]> * 🔨Refactor methods across files (#2321) * remove numba Signed-off-by: Ashwin Vaidya <[email protected]> * fix pre-commit checks Signed-off-by: Ashwin Vaidya <[email protected]> * remove all unused methods Signed-off-by: Ashwin Vaidya <[email protected]> * replace numpy with torch Signed-off-by: Ashwin Vaidya <[email protected]> * refactor code Signed-off-by: Ashwin Vaidya <[email protected]> * refactor move functional inside update remove path from the metric * Add changes from comments Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]> * Remove model to model comparison (#2325) * rename to pimo Signed-off-by: Ashwin Vaidya <[email protected]> * minor refactor Signed-off-by: Ashwin Vaidya <[email protected]> * remove model to model comparison Signed-off-by: Ashwin Vaidya <[email protected]> * fix test Signed-off-by: Ashwin Vaidya <[email protected]> * PR comments Signed-off-by: Ashwin Vaidya <[email protected]> * Minor refactor Signed-off-by: Ashwin Vaidya <[email protected]> --------- Signed-off-by: Ashwin Vaidya <[email protected]> * PR comments Signed-off-by: Ashwin Vaidya <[email protected]> * Remove unused enums Signed-off-by: Ashwin Vaidya <[email protected]> * update doc strings Signed-off-by: Ashwin Vaidya <[email protected]> * update param names Signed-off-by: Ashwin Vaidya <[email protected]> * add aupimo basic usage tutorial notebook (#2330) * add aupimo basic usage tutorial notebook Signed-off-by: jpcbertoldo <[email protected]> * update scipy import Signed-off-by: jpcbertoldo <[email protected]> * add cite us Signed-off-by: jpcbertoldo <[email protected]> * minor Signed-off-by: jpcbertoldo <[email protected]> * modify texts and add illustration Signed-off-by: jpcbertoldo <[email protected]> * udpate working dir Signed-off-by: jpcbertoldo <[email protected]> --------- Signed-off-by: jpcbertoldo <[email protected]> --------- Signed-off-by: jpcbertoldo <[email protected]> Signed-off-by: Ashwin Vaidya <[email protected]> Co-authored-by: Joao P C Bertoldo <[email protected]> Co-authored-by: Samet Akcay <[email protected]>
1 parent f473df8 commit 191e21f

File tree

16 files changed

+3744
-1
lines changed

16 files changed

+3744
-1
lines changed

notebooks/700_metrics/701a_aupimo.ipynb

Lines changed: 549 additions & 0 deletions
Large diffs are not rendered by default.

notebooks/700_metrics/roc_pro_pimo.svg

Lines changed: 690 additions & 0 deletions
Loading

src/anomalib/data/utils/path.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,13 +142,20 @@ def contains_non_printable_characters(path: str | Path) -> bool:
142142
return not printable_pattern.match(str(path))
143143

144144

145-
def validate_path(path: str | Path, base_dir: str | Path | None = None, should_exist: bool = True) -> Path:
145+
def validate_path(
146+
path: str | Path,
147+
base_dir: str | Path | None = None,
148+
should_exist: bool = True,
149+
extensions: tuple[str, ...] | None = None,
150+
) -> Path:
146151
"""Validate the path.
147152
148153
Args:
149154
path (str | Path): Path to validate.
150155
base_dir (str | Path): Base directory to restrict file access.
151156
should_exist (bool): If True, do not raise an exception if the path does not exist.
157+
extensions (tuple[str, ...] | None): Accepted extensions for the path. An exception is raised if the
158+
path does not have one of the accepted extensions. If None, no check is performed. Defaults to None.
152159
153160
Returns:
154161
Path: Validated path.
@@ -213,6 +220,11 @@ def validate_path(path: str | Path, base_dir: str | Path | None = None, should_e
213220
msg = f"Read or execute permissions denied for the path: {path}"
214221
raise PermissionError(msg)
215222

223+
# Check if the path has one of the accepted extensions
224+
if extensions is not None and path.suffix not in extensions:
225+
msg = f"Path extension is not accepted. Accepted extensions: {extensions}. Path: {path}"
226+
raise ValueError(msg)
227+
216228
return path
217229

218230

src/anomalib/metrics/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
from .f1_max import F1Max
2020
from .f1_score import F1Score
2121
from .min_max import MinMax
22+
from .pimo import AUPIMO, PIMO
2223
from .precision_recall_curve import BinaryPrecisionRecallCurve
2324
from .pro import PRO
2425
from .threshold import F1AdaptiveThreshold, ManualThreshold
@@ -35,6 +36,8 @@
3536
"ManualThreshold",
3637
"MinMax",
3738
"PRO",
39+
"PIMO",
40+
"AUPIMO",
3841
]
3942

4043
logger = logging.getLogger(__name__)
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
"""Per-Image Metrics."""
2+
3+
# Original Code
4+
# https://github.com/jpcbertoldo/aupimo
5+
#
6+
# Modified
7+
# Copyright (C) 2024 Intel Corporation
8+
# SPDX-License-Identifier: Apache-2.0
9+
10+
from .binary_classification_curve import ThresholdMethod
11+
from .pimo import AUPIMO, PIMO, AUPIMOResult, PIMOResult
12+
13+
__all__ = [
14+
# constants
15+
"ThresholdMethod",
16+
# result classes
17+
"PIMOResult",
18+
"AUPIMOResult",
19+
# torchmetrics interfaces
20+
"PIMO",
21+
"AUPIMO",
22+
"StatsOutliersPolicy",
23+
]

0 commit comments

Comments
 (0)