Repository navigation
Open
Labels
GPUIssues that apply to both CUDA and ROCM GPUsIssues that apply to both CUDA and ROCM GPUsXLAbugSomething isn't workingSomething isn't working
Description
Description
Concatenating a float64 vector with its sum takes ~0.27 ms per call, whereas adding a jax.lax.optimization_barrier around the sum reduces this to ~0.03 ms.
This reproduces in a fresh environment installed using pip install "jax[cuda12]==0.12.0.1", using a RTX 5090 GPU.
This was found while investigating a JAX-MD simulation slowdown, where the same simulation took ~25 ms/step with JAX 0.11.1 and ~71 ms/step with 0.12.0.1. A barrier around the relevant charge-equilibration sum restored the 25 ms/step.
The following minimal example reproduces the inefficient fusion in both versions of JAX, but does not show the version-to-version regression:
from timeit import timeit
import jax
import jax.numpy as jnp
jax.config.update("jax_enable_x64", True)
jax.config.update("jax_platforms", "cuda")
@jax.jit
def fused(x):
return jnp.concatenate((x, x.sum(keepdims=True)))
@jax.jit
def barrier(x):
total = jax.lax.optimization_barrier(x.sum(keepdims=True))
return jnp.concatenate((x, total))
x = jnp.ones(10_000)
for fn in (fused, barrier):
fn(x).block_until_ready() # compile before timing
seconds = timeit(lambda: fn(x).block_until_ready(), number=100)
print(f"{fn.__name__}: {seconds / 100 * 1000:.3f} ms/call")
# RTX 5090, JAX/jaxlib 0.12.0.1:
# fused: ~0.27 ms/call
# barrier: ~0.03 ms/callPossibly related historical reports: #18424 and openxla/xla#7152.
System info (python version, jaxlib version, accelerator, etc.)
jax: 0.12.0.1
jaxlib: 0.12.0.1
numpy: 2.5.4
python: 3.12.16 | packaged by Anaconda, Inc. | (main, Oct 26 2026, 20:21:38) [GCC 14.3.0]
device info: NVIDIA GeForce RTX 5090-1, 1 local devices"
process_count: 1
platform: uname_result(system='Linux', node='hidden', release='5.14.0-611.54.3.el9_7.x86_64', version='#1 SMP PREEMPT_DYNAMIC Thu May 7 16:31:24 EDT 2026', machine='x86_64')
XLA_PYTHON_CLIENT_PREALLOCATE=false
JAX_PLATFORMS=cuda
$ nvidia-smi
Thu Nov 12 14:05:30 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 570.172.08 Driver Version: 570.172.08 CUDA Version: 12.8 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce RTX 5090 On | 00000000:D1:00.0 Off | N/A |
| 0% 39C P1 19W / 600W | 1038MiB / 32607MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GeForce RTX 5090 On | 00000000:F1:00.0 On | N/A |
| 0% 46C P8 36W / 600W | 943MiB / 32607MiB | 1% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
Metadata
Metadata
Assignees
Labels
GPUIssues that apply to both CUDA and ROCM GPUsIssues that apply to both CUDA and ROCM GPUsXLAbugSomething isn't workingSomething isn't working
Activity