Skip to content

Wrong results on CPU for const @ x followed by a reduction (YNN fusion), still present on 0.12.1 nightly #42651

Open

Description

@AdamOrmondroyd

Description

On CPU, a jitted matmul whose left operand is a closed-over constant, followed by a sum, returns wrong results for matmul outputs of 4096 or more elements. For example, 16×16 @ 16×255 (4080 elements) is correct, but 16×16 @ 16×256 (4096) is wrong. Disabling YNN fusion gives the correct answer.

This looks related to #39103, #41414 and #42549, but it is not fixed by them. The #39103 reproducer passes on this machine from 0.11.0 onwards, and the #41414 and #42549 reproducers pass on the nightly, yet this case still fails on 0.12.0.1 and the nightly.

import jax, jax.numpy as jnp, numpy as np

rng = np.random.default_rng(0)
a = jnp.asarray(rng.standard_normal((13, 13)), jnp.float32)  # closed over
b = rng.standard_normal((316, 13)).astype(np.float32)        # 13*316 = 4108 >= 4096

y = jax.jit(lambda b: (a @ b.T).sum(axis=0))(b)
ref = (np.asarray(a, np.float64) @ b.T.astype(np.float64)).sum(axis=0)
print(np.abs(np.asarray(y) - ref).max())  # 39.5, expected ~1e-6
case 315 rows (4095 elements) 316 rows (4108 elements)
a closed over (constant) 3.6e-6 39.5
a passed as an argument 3.6e-6 3.6e-6
a closed over, XLA_FLAGS=--xla_cpu_experimental_ynn_fusion_type= 3.6e-6 3.6e-6

What is needed to trigger it:

  • The left operand of the matmul is a compile-time constant. With the constant on the right (x @ C), results are correct.
  • The matmul is followed by a reduction. Summing over axis 0, over axis 1 or over everything all fail. The matmul alone, without a reduction, is correct.
  • The output has at least 4096 elements. This is necessary but not sufficient. Some shapes pass above it, e.g. a single-row constant (1×4 @ 4×8192), a 1- or 2-column result (4096×4 @ 4×2) and 1024×4 @ 4×4. Some shapes that fail on 0.12.0.1 pass on the nightly (e.g. 64×8 @ 8×64). The 13×13 case fails on both.

Whether the right operand is transposed makes no difference.

case (13×13 constant C, YNN fusion on) 4095 output elements 4108 13000
(C @ x).sum(axis=0) ok wrong wrong
(C @ x).sum(axis=1) ok wrong wrong
(C @ x).sum() ok wrong wrong
C @ x (no reduction) ok ok ok
(x @ C).sum(...), any axis ok ok ok

It reproduces on an M4 Pro but not on an M1 Pro. YNN fusion is on by default on both machines, since the #41414 repro fails on both with fusion on and passes with it off.

chip jax YNN fusion this issue #41414 repro
M1 Pro 0.12.0.1 on (default) ok wrong
M1 Pro 0.12.0.1 off ok ok
M1 Pro 0.12.1 nightly on (default) ok ok
M4 Pro 0.12.0.1 on (default) wrong wrong
M4 Pro 0.12.0.1 off ok ok
M4 Pro 0.12.1 nightly on (default) wrong ok
M4 Pro 0.12.1 nightly off ok ok

This is the form jax.vmap produces for lambda x: (L @ x).sum() with a closed-over L:

c:f32[13,500] = dot_general[dimension_numbers=(([1], [1]), ([], []))] a b
d:f32[500] = reduce_sum[axes=(0,)] c

We hit this in a vmapped Gaussian log-likelihood (v = L @ (x - mean); -(v*v).sum()/2, with L a fixed 13×13 inverse-Cholesky factor) evaluated over batches of 500 points. It silently shifted a nested-sampling Bayesian evidence from 240.7 to 247.1.

System info (python version, jaxlib version, accelerator, etc.)

Fails:  Apple M4 Pro, Darwin 25.6.0, JAX_PLATFORMS=cpu, XLA_FLAGS unset
        Python 3.14.10 + numpy 2.5.5: jax/jaxlib 0.10.2, 0.11.0, 0.11.1, 0.11.2, 0.12.0, 0.12.0.1,
                                      0.12.1.dev20261115 (jaxlib 0.12.1.dev20261113)
        Python 3.13.14 + numpy 2.5.4: jax/jaxlib 0.10.2, 0.11.0, 0.11.1, 0.11.2, 0.12.0, 0.12.0.1,
                                      0.12.1.dev20261115 and dev20261116 (jaxlib 0.12.1.dev20261113)
Passes: Apple M1 Pro, Darwin 25.6.0, JAX_PLATFORMS=cpu, XLA_FLAGS unset
        Python 3.14.10 + numpy 2.5.5: jax/jaxlib 0.12.0.1, 0.12.1 nightly

Activity

  1. AdamOrmondroyd commented on Nov 16, 2026

    @AdamOrmondroyd
    Author

    A colleague keeps going on about 16-bit precision, so I thought why not test it there. Same repro, M4 Pro, jax 0.12.0.1, error against float64:

    dtype fusion 4095 elems (13×13) 4108 (13×13) 4080 (16×16) 4096 (16×16)
    float16 on 0.012 0.016 0.017 0.018
    float16 off 0.012 0.016 0.017 0.018
    bfloat16 on 0.20 41.5 0.13 75.8
    bfloat16 off 0.061 0.064 0.12 0.12
    float64 on 0 7.1e-15 0 1.4e-14

    So bfloat16 fails at the same threshold, while float16 and float64 are unaffected for these shapes. I only ran float64 with fusion on. I didn't try the nightly for the 16-bit types.

  2. self-assigned this
    on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions