Skip to content

Conversation

stop1one
Copy link

@stop1one stop1one commented Sep 22, 2025

Description

This PR fixes the calculation of mAR@K in MeanAverageRecall to comply with the COCO evaluation protocol.
Previously, the implementation selected the top-K predictions globally across all images, rather than per image.
According to the COCO evaluation protocol, mAR@K should be calculated by considering the top-K highest-confidence detections for each image.

This issue is tracked in issue #1966

To resolve this, I modified the _compute and _compute_average_recall_for_classes function to first filter the statistics by confidence score before concatenating them and calculate the confusion matrix.

No new dependencies are required for this change.

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

I tested the change by running the metric on a dataset with varying numbers of predictions per image and verified that, for each image, only the top-K predictions (by confidence) were used in the mAR@K calculation.

Any specific deployment considerations

No special deployment considerations are required.

Docs

  • Docs updated? What were the changes: N/A

@stop1one stop1one requested a review from SkalskiP as a code owner September 22, 2025 03:23
@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants