Skip to content

RetrievalMAP and RetrievalMRR do not validate invalid top_k values #3378

Description

@rintaro121

🐛 Bug

RetrievalMAP and RetrievalMRR do not correctly validate invalid top_k values.
This behavior is inconsistent with other retrieval metrics such as RetrievalNormalizedDCG, which raises a clear ValueError for invalid top_k values.

For example, RetrievalMRR(top_k=-1) is accepted at initialization, but later fails inside torch.topk with a RuntimeError.

To Reproduce

RetrievalMRR(top_k=-1) should raise a clear ValueError, but it currently raises a RuntimeError during metric computation.

Code sample
from torch import tensor
from torchmetrics.retrieval import RetrievalMRR

indexes = tensor([0, 0, 0, 1, 1, 1, 1])
preds = tensor([0.2, 0.3, 0.5, 0.1, 0.3, 0.5, 0.2])
target = tensor([False, False, True, False, True, False, True])

mrr = RetrievalMRR(top_k=-1)
mrr(preds, target, indexes=indexes)

Current error:

RuntimeError: selected index k out of range

Expected behavior:

ValueError: `top_k` has to be a positive integer or None
Environment
  • TorchMetrics version (if build from source, add commit SHA): 1.9.0
  • Python & PyTorch Version (e.g., 1.0): 3.12.13
  • Any other relevant information such as OS (e.g., Linux): Run on Google Colab

Additional context

I think the cause is the top_k validation condition in the following places:

The condition currently uses and, so invalid integer values such as top_k=-1 can pass validation

If this needs to be fixed, I would be happy to open a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug / fixSomething isn't workinghelp wantedExtra attention is needed

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions