You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This PR fixes a bug in number.floor where finite numbers of magnitude $\ge 2^{63}$ were incorrectly returning saturated values (about $\pm 9.22 \times 10^{18}$) due to narrowing conversion to i64.
Problem
number.floor was computing trunc = ^.as-i64.as-number. The conversion as-i64 is backed by Double.longValue(). For finite doubles whose magnitude is $\ge 2^{63}$, Double.longValue() saturates to Long.MAX_VALUE or Long.MIN_VALUE per Java Language Specification narrowing rules, causing floor of values like 1e20 or -1e20 to return the saturated boundary value.
Solution
Since double-precision floating point numbers have 52 mantissa bits, any double value with a magnitude $\ge 2^{52}$ cannot represent a fractional part and is already an exact mathematical integer.
We update the floor implementation to check if the magnitude is $\ge 2^{52}$ (specifically $\ge 4503599627370496$ or $\le -4503599627370496$), and if so, return the number itself unchanged. This bypasses the narrowing conversion path entirely for values where the result is guaranteed to be equal to the input.
All benchmarks are within the acceptable range. No critical degradation detected (threshold is 100%). Please refer to the detailed report for more information.
Click to see the detailed report
Test
Base Score
PR Score
Change
% Change
Unit
Mode
benchmarks.XmirBench.xmirToEO
12628.043
12583.618
-44.425
-0.35%
ms/op
Average Time
✅ Performance gain: benchmarks.XmirBench.xmirToEO is faster by 44.425 ms/op (0.35%)
@arnabnandy7 It is not a good idea to name Git branches the way you named this one: "fix/number-floor-overflow". You've earned -6 points. Next time, better give your branch the same name as the number of the ticket that you are solving. In this case, a perfect name, for example, would be "5265". Your running score is -6; don't forget to check your Zerocracy account too).
@arnabnandy7 Thanks for the contribution! You've earned +12 points for this: +16 as a basis; -4 for too few (18) hits-of-code. Please, keep them coming. Your running score is +6; don't forget to check your Zerocracy account too).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes a bug in$\ge 2^{63}$ were incorrectly returning saturated values (about $\pm 9.22 \times 10^{18}$ ) due to narrowing conversion to
number.floorwhere finite numbers of magnitudei64.Problem
number.floorwas computingtrunc = ^.as-i64.as-number. The conversionas-i64is backed byDouble.longValue(). For finite doubles whose magnitude isDouble.longValue()saturates toLong.MAX_VALUEorLong.MIN_VALUEper Java Language Specification narrowing rules, causingfloorof values like1e20or-1e20to return the saturated boundary value.Solution
Since double-precision floating point numbers have 52 mantissa bits, any double value with a magnitude$\ge 2^{52}$ cannot represent a fractional part and is already an exact mathematical integer.
We update the$\ge 2^{52}$ (specifically $\ge 4503599627370496$ or $\le -4503599627370496$ ), and if so, return the number itself unchanged. This bypasses the narrowing conversion path entirely for values where the result is guaranteed to be equal to the input.
floorimplementation to check if the magnitude isRelated Issues
Closes #5260