Requested by Yegor Bugayenko · Slack thread
Summary
number.floor computes trunc = ^.as-i64.as-number (truncate toward zero) and then subtracts 1 when trunc.gt ^. as-i64 is backed by EOnumber$EOas_i64, which does Double.longValue(). For finite doubles whose magnitude is >= 2^63 (e.g. 1e20, 1e30), longValue() saturates to Long.MAX_VALUE / Long.MIN_VALUE, so trunc becomes ~9.22e18 instead of the original value. Since every double >= 2^52 is already an integer, floor should return the input unchanged, but instead floor(1e20) returns ~9.223e18.
Location
eo-runtime/src/main/eo/number.eo lines 119-126.
Buggy code
[] > floor
if. > @
^.is-finite.not
^
if.
trunc.gt ^
trunc.minus 1
trunc
^.as-i64.as-number > trunc
Why it's wrong
floor must return the largest integer <= x; for any finite double with |x| >= 2^52 the value is itself an exact integer, so floor(x) == x. Reproduced in Java: x=1e20 -> Double.longValue()=9223372036854775807 -> trunc=9.22337e18; trunc.gt x is false, so eo-floor=9.22337e18 while Math.floor(1e20)=1e20 (mismatch). Same for 9.3e18, 1e30, and the negative counterparts. The is-finite guard does not cover these (they are finite), so the saturated path is taken.
Verifier confirmation: Confirmed independently. number.floor (number.eo:119-127) computes trunc = ^.as-i64.as-number and returns trunc (minus 1 only when trunc.gt ^). as-i64 is implemented by EOnumber$EOas_i64.java:28 as Double.longValue(), which by JLS narrowing-conversion rules saturates to Long.MAX_VALUE / Long.MIN_VALUE for any finite double whose magnitude is >= 2^63. Every double with |x| >= 2^52 is already an exact integer, so floor(x) should equal x, but for |x| >= 2^63 (e.g. 1e20, 9.3e18, 1e30) floor instead returns a value capped at about ±9.223e18. I reproduced this exactly in Java: x=1e20 -> longValue()=9223372036854775807 -> trunc=9.22337e18; trunc.gt x is false, so the eo result is 9.22337e18 while Math.floor(1e20)=1e20 (mismatch), with identical mismatches for -1e20, 9.3e18, -9.3e18, and 1e30. The is-finite.not guard only short-circuits nan/inf, so these finite values take the buggy saturated path; the trunc.gt ^ correction never fires for the positive saturated case. This is a genuine correctness bug, not intended behavior.
Expected behavior
floor(1e20) == 1e20 (and generally floor(x)==x for any finite x with |x| >= 2^52). Currently it returns a saturated ~9.22e18-magnitude value.
Suggested fix
Treat finite values whose magnitude is too large to be a non-integer (|x| >= 2^52, or more simply outside the i64-representable range) as already-integer and return the input unchanged, rather than routing them through as-i64. E.g. when ^.as-bytes-based magnitude check shows the value cannot have a fractional part, return ^; only apply the as-i64 truncate-and-adjust for values within i64 range.
Requested by Yegor Bugayenko · Slack thread
Summary
number.floor computes
trunc = ^.as-i64.as-number(truncate toward zero) and then subtracts 1 whentrunc.gt ^. as-i64 is backed by EOnumber$EOas_i64, which doesDouble.longValue(). For finite doubles whose magnitude is >= 2^63 (e.g. 1e20, 1e30), longValue() saturates to Long.MAX_VALUE / Long.MIN_VALUE, so trunc becomes ~9.22e18 instead of the original value. Since every double >= 2^52 is already an integer, floor should return the input unchanged, but instead floor(1e20) returns ~9.223e18.Location
eo-runtime/src/main/eo/number.eolines 119-126.Buggy code
Why it's wrong
floor must return the largest integer <= x; for any finite double with |x| >= 2^52 the value is itself an exact integer, so floor(x) == x. Reproduced in Java: x=1e20 -> Double.longValue()=9223372036854775807 -> trunc=9.22337e18; trunc.gt x is false, so eo-floor=9.22337e18 while Math.floor(1e20)=1e20 (mismatch). Same for 9.3e18, 1e30, and the negative counterparts. The is-finite guard does not cover these (they are finite), so the saturated path is taken.
Verifier confirmation: Confirmed independently. number.floor (number.eo:119-127) computes trunc = ^.as-i64.as-number and returns trunc (minus 1 only when trunc.gt ^). as-i64 is implemented by EOnumber$EOas_i64.java:28 as Double.longValue(), which by JLS narrowing-conversion rules saturates to Long.MAX_VALUE / Long.MIN_VALUE for any finite double whose magnitude is >= 2^63. Every double with |x| >= 2^52 is already an exact integer, so floor(x) should equal x, but for |x| >= 2^63 (e.g. 1e20, 9.3e18, 1e30) floor instead returns a value capped at about ±9.223e18. I reproduced this exactly in Java: x=1e20 -> longValue()=9223372036854775807 -> trunc=9.22337e18; trunc.gt x is false, so the eo result is 9.22337e18 while Math.floor(1e20)=1e20 (mismatch), with identical mismatches for -1e20, 9.3e18, -9.3e18, and 1e30. The is-finite.not guard only short-circuits nan/inf, so these finite values take the buggy saturated path; the trunc.gt ^ correction never fires for the positive saturated case. This is a genuine correctness bug, not intended behavior.
Expected behavior
floor(1e20) == 1e20 (and generally floor(x)==x for any finite x with |x| >= 2^52). Currently it returns a saturated ~9.22e18-magnitude value.
Suggested fix
Treat finite values whose magnitude is too large to be a non-integer (|x| >= 2^52, or more simply outside the i64-representable range) as already-integer and return the input unchanged, rather than routing them through as-i64. E.g. when
^.as-bytes-based magnitude check shows the value cannot have a fractional part, return ^; only apply the as-i64 truncate-and-adjust for values within i64 range.