-
Notifications
You must be signed in to change notification settings - Fork 7.1k
Fix NMS and IoU overflows for fp16 #3383
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
4807308
to
f289fae
Compare
f289fae
to
673fda8
Compare
Codecov Report
@@ Coverage Diff @@
## master #3383 +/- ##
==========================================
+ Coverage 74.80% 74.82% +0.01%
==========================================
Files 105 105
Lines 9714 9719 +5
Branches 1561 1562 +1
==========================================
+ Hits 7267 7272 +5
Misses 1960 1960
Partials 487 487
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, thanks a lot!
Summary: * Replace type T with accumulator. * Upcast tensors of box ops to avoid overflow in multiplications. Reviewed By: NicolasHug Differential Revision: D26605323 fbshipit-source-id: 443e2714bdf0c916834340b180a6ddfb0f55cb2e
Fixes #3371