Repository navigation
Description
Description
On CPU, a jitted matmul whose left operand is a closed-over constant, followed by a sum, returns wrong results for matmul outputs of 4096 or more elements. For example, 16×16 @ 16×255 (4080 elements) is correct, but 16×16 @ 16×256 (4096) is wrong. Disabling YNN fusion gives the correct answer.
This looks related to #39103, #41414 and #42549, but it is not fixed by them. The #39103 reproducer passes on this machine from 0.11.0 onwards, and the #41414 and #42549 reproducers pass on the nightly, yet this case still fails on 0.12.0.1 and the nightly.
import jax, jax.numpy as jnp, numpy as np
rng = np.random.default_rng(0)
a = jnp.asarray(rng.standard_normal((13, 13)), jnp.float32) # closed over
b = rng.standard_normal((316, 13)).astype(np.float32) # 13*316 = 4108 >= 4096
y = jax.jit(lambda b: (a @ b.T).sum(axis=0))(b)
ref = (np.asarray(a, np.float64) @ b.T.astype(np.float64)).sum(axis=0)
print(np.abs(np.asarray(y) - ref).max()) # 39.5, expected ~1e-6| case | 315 rows (4095 elements) | 316 rows (4108 elements) |
|---|---|---|
a closed over (constant) |
3.6e-6 | 39.5 |
a passed as an argument |
3.6e-6 | 3.6e-6 |
a closed over, XLA_FLAGS=--xla_cpu_experimental_ynn_fusion_type= |
3.6e-6 | 3.6e-6 |
What is needed to trigger it:
- The left operand of the matmul is a compile-time constant. With the constant on the right (
x @ C), results are correct. - The matmul is followed by a reduction. Summing over axis 0, over axis 1 or over everything all fail. The matmul alone, without a reduction, is correct.
- The output has at least 4096 elements. This is necessary but not sufficient. Some shapes pass above it, e.g. a single-row constant (1×4 @ 4×8192), a 1- or 2-column result (4096×4 @ 4×2) and 1024×4 @ 4×4. Some shapes that fail on 0.12.0.1 pass on the nightly (e.g. 64×8 @ 8×64). The 13×13 case fails on both.
Whether the right operand is transposed makes no difference.
case (13×13 constant C, YNN fusion on) |
4095 output elements | 4108 | 13000 |
|---|---|---|---|
(C @ x).sum(axis=0) |
ok | wrong | wrong |
(C @ x).sum(axis=1) |
ok | wrong | wrong |
(C @ x).sum() |
ok | wrong | wrong |
C @ x (no reduction) |
ok | ok | ok |
(x @ C).sum(...), any axis |
ok | ok | ok |
It reproduces on an M4 Pro but not on an M1 Pro. YNN fusion is on by default on both machines, since the #41414 repro fails on both with fusion on and passes with it off.
| chip | jax | YNN fusion | this issue | #41414 repro |
|---|---|---|---|---|
| M1 Pro | 0.12.0.1 | on (default) | ok | wrong |
| M1 Pro | 0.12.0.1 | off | ok | ok |
| M1 Pro | 0.12.1 nightly | on (default) | ok | ok |
| M4 Pro | 0.12.0.1 | on (default) | wrong | wrong |
| M4 Pro | 0.12.0.1 | off | ok | ok |
| M4 Pro | 0.12.1 nightly | on (default) | wrong | ok |
| M4 Pro | 0.12.1 nightly | off | ok | ok |
This is the form jax.vmap produces for lambda x: (L @ x).sum() with a closed-over L:
c:f32[13,500] = dot_general[dimension_numbers=(([1], [1]), ([], []))] a b
d:f32[500] = reduce_sum[axes=(0,)] c
We hit this in a vmapped Gaussian log-likelihood (v = L @ (x - mean); -(v*v).sum()/2, with L a fixed 13×13 inverse-Cholesky factor) evaluated over batches of 500 points. It silently shifted a nested-sampling Bayesian evidence from 240.7 to 247.1.
System info (python version, jaxlib version, accelerator, etc.)
Fails: Apple M4 Pro, Darwin 25.6.0, JAX_PLATFORMS=cpu, XLA_FLAGS unset
Python 3.14.10 + numpy 2.5.5: jax/jaxlib 0.10.2, 0.11.0, 0.11.1, 0.11.2, 0.12.0, 0.12.0.1,
0.12.1.dev20261115 (jaxlib 0.12.1.dev20261113)
Python 3.13.14 + numpy 2.5.4: jax/jaxlib 0.10.2, 0.11.0, 0.11.1, 0.11.2, 0.12.0, 0.12.0.1,
0.12.1.dev20261115 and dev20261116 (jaxlib 0.12.1.dev20261113)
Passes: Apple M1 Pro, Darwin 25.6.0, JAX_PLATFORMS=cpu, XLA_FLAGS unset
Python 3.14.10 + numpy 2.5.5: jax/jaxlib 0.12.0.1, 0.12.1 nightly
Activity
AdamOrmondroyd commentedon Nov 16, 2026
A colleague keeps going on about 16-bit precision, so I thought why not test it there. Same repro, M4 Pro, jax 0.12.0.1, error against float64:
So bfloat16 fails at the same threshold, while float16 and float64 are unaffected for these shapes. I only ran float64 with fusion on. I didn't try the nightly for the 16-bit types.
Give folded YNNPACK constants the strides their producers require