🐛 Bug
RetrievalMAP and RetrievalMRR do not correctly validate invalid top_k values.
This behavior is inconsistent with other retrieval metrics such as RetrievalNormalizedDCG, which raises a clear ValueError for invalid top_k values.
For example, RetrievalMRR(top_k=-1) is accepted at initialization, but later fails inside torch.topk with a RuntimeError.
To Reproduce
RetrievalMRR(top_k=-1) should raise a clear ValueError, but it currently raises a RuntimeError during metric computation.
Code sample
from torch import tensor
from torchmetrics.retrieval import RetrievalMRR
indexes = tensor([0, 0, 0, 1, 1, 1, 1])
preds = tensor([0.2, 0.3, 0.5, 0.1, 0.3, 0.5, 0.2])
target = tensor([False, False, True, False, True, False, True])
mrr = RetrievalMRR(top_k=-1)
mrr(preds, target, indexes=indexes)
Current error:
RuntimeError: selected index k out of range
Expected behavior:
ValueError: `top_k` has to be a positive integer or None
Environment
- TorchMetrics version (if build from source, add commit SHA): 1.9.0
- Python & PyTorch Version (e.g., 1.0): 3.12.13
- Any other relevant information such as OS (e.g., Linux): Run on Google Colab
Additional context
I think the cause is the top_k validation condition in the following places:
The condition currently uses and, so invalid integer values such as top_k=-1 can pass validation
If this needs to be fixed, I would be happy to open a PR.
🐛 Bug
RetrievalMAPandRetrievalMRRdo not correctly validate invalid top_k values.This behavior is inconsistent with other retrieval metrics such as
RetrievalNormalizedDCG, which raises a clearValueErrorfor invalid top_k values.For example,
RetrievalMRR(top_k=-1)is accepted at initialization, but later fails insidetorch.topkwith aRuntimeError.To Reproduce
RetrievalMRR(top_k=-1)should raise a clearValueError, but it currently raises aRuntimeErrorduring metric computation.Code sample
Current error:
Expected behavior:
Environment
Additional context
I think the cause is the
top_kvalidation condition in the following places:RetrievalMRR: https://github.com/Lightning-AI/torchmetrics/blob/master/src/torchmetrics/retrieval/reciprocal_rank.py#L113retrieval_reciprocal_rank: https://github.com/Lightning-AI/torchmetrics/blob/master/src/torchmetrics/functional/retrieval/reciprocal_rank.py#L52RetrievalMAP: https://github.com/Lightning-AI/torchmetrics/blob/master/src/torchmetrics/retrieval/average_precision.py#L113retrieval_average_precision: https://github.com/Lightning-AI/torchmetrics/blob/master/src/torchmetrics/functional/retrieval/average_precision.py#L52The condition currently uses
and, so invalid integer values such astop_k=-1can pass validationIf this needs to be fixed, I would be happy to open a PR.