Skip to content

Bugfix/auto normalizing logits binary#3377

Open
JanFidor wants to merge 10 commits into
Lightning-AI:masterfrom
JanFidor:bugfix/auto-normalizing-logits-binary
Open

Bugfix/auto normalizing logits binary#3377
JanFidor wants to merge 10 commits into
Lightning-AI:masterfrom
JanFidor:bugfix/auto-normalizing-logits-binary

Conversation

@JanFidor

@JanFidor JanFidor commented May 14, 2026

Copy link
Copy Markdown

What does this PR do?

Fixes #1604 for binary metrics

Details
  • Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
PR review

The bug has been discussed a lot up to this point. I've decided to tackle it in parts, as merging the entire thing at one would be challenging. This PR fixes the behaviour in binary metrics. When we decide on the final nature of the fix and merge it, I will tackle the other ones.

Did you have fun?

Make sure you had fun coding 🙃

A LOT OF FUN, the day of the end of this vengeful bug draws nearer

Credit to @SkafteNicki who began the work on this bug and whose code this PR was heavily based on

@JanFidor

JanFidor commented Jun 1, 2026

Copy link
Copy Markdown
Author

Hi @SkafteNicki @justusschock @Borda did you have a moment to take a lot the PR? I totally get it if you're busy, but if possible would very much appreciate it if you let me know when you could get around to a review! Have a great week! ☀️

@JanFidor

Copy link
Copy Markdown
Author

Hi again @SkafteNicki @justusschock @Borda any chance you might be able to do a review for this fix anytime soon? I'd love to make suitable fixes for the other metrics as soon as we merge this one.

@JanFidor

JanFidor commented Jul 7, 2026

Copy link
Copy Markdown
Author

I checked the maintainer list over at https://lightning.ai/docs/torchmetrics/stable/governance.html , so I'm moving my question to the currently listed maintainers. Hi @rittik9 @baskrahmer @matsumotosan , could you checkout this PR? I firmly believe that It would be a great addition to the library, as this bug is quite difficult to discover by oneself and as such could have silent impact on peoples workflows. Have amazing days! ☀️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BinaryAccuracy() sometimes gives incorrect answers due to non-deterministic sigmoiding

1 participant