Repository navigation
SIMD AOT 10-20x slower. #63217
Copy link
Copy link
Closed
Description
I'm optimizing a bit array implementation in Dart with SIMD. JIT is much faster, as expected. However, AOT is 10-20x slower. My expectation was that the performance gains would carry over to AOT.
Output JIT:
words=2048 (65536 bits) iters=200000
or scalar: 0.896 us/iter (sink=23552)
or simd : 0.419 us/iter (sink=23552)
-> simd/scalar = 0.47x
and_not scalar: 1.101 us/iter (sink=9216)
and_not simd : 0.328 us/iter (sink=9216)
-> simd/scalar = 0.30x
Output AOT:
words=2048 (65536 bits) iters=200000
or scalar: 1.090 us/iter (sink=23552)
or simd : 12.494 us/iter (sink=23552)
-> simd/scalar = 11.46x
and_not scalar: 0.997 us/iter (sink=9216)
and_not simd : 23.282 us/iter (sink=9216)
-> simd/scalar = 23.36x
Repro:
import 'dart:typed_data';
const int words = 2048;
const int iters = 200000;
void main() {
print('words=$words (${words * 32} bits) iters=$iters');
final orS = bench('or scalar', orScalar);
final orV = bench('or simd ', orSimd);
print(' -> simd/scalar = ${(orV / orS).toStringAsFixed(2)}x\n');
final anS = bench('and_not scalar', andNotScalar);
final anV = bench('and_not simd ', andNotSimd);
print(' -> simd/scalar = ${(anV / anS).toStringAsFixed(2)}x');
}
double bench(String label, void Function(Uint32List, Uint32List) op) {
final a = Uint32List(words);
final b = Uint32List(words);
for (int i = 0; i < words; i++) {
a[i] = 0xA5A5A5A5 ^ i;
b[i] = 0x5A5A5A5A ^ (i * 31);
}
for (int w = 0; w < 50; w++) op(a, b); // warmup
final sw = Stopwatch()..start();
for (int k = 0; k < iters; k++) op(a, b);
sw.stop();
int sink = 0;
for (int i = 0; i < words; i++) sink ^= a[i]; // prevent DCE
final usPerIter = sw.elapsedMicroseconds / iters;
print('$label: ${usPerIter.toStringAsFixed(3)} us/iter (sink=$sink)');
return usPerIter;
}
void orSimd(Uint32List a, Uint32List b) {
final la = Int32x4List.view(a.buffer, a.offsetInBytes, a.length >> 2);
final lb = Int32x4List.view(b.buffer, b.offsetInBytes, b.length >> 2);
for (int j = 0; j < la.length; j++) {
la[j] = la[j] | lb[j];
}
}
void orScalar(Uint32List a, Uint32List b) {
int i = 0;
final n = a.length;
for (; i + 7 < n; i += 8) {
a[i] |= b[i];
a[i + 1] |= b[i + 1];
a[i + 2] |= b[i + 2];
a[i + 3] |= b[i + 3];
a[i + 4] |= b[i + 4];
a[i + 5] |= b[i + 5];
a[i + 6] |= b[i + 6];
a[i + 7] |= b[i + 7];
}
for (; i < n; i++) a[i] |= b[i];
}
void andNotSimd(Uint32List a, Uint32List b) {
// Int32x4 has no bitwise NOT, so use the identity a & ~b == a ^ (a & b).
final la = Int32x4List.view(a.buffer, a.offsetInBytes, a.length >> 2);
final lb = Int32x4List.view(b.buffer, b.offsetInBytes, b.length >> 2);
for (int j = 0; j < la.length; j++) {
final x = la[j];
la[j] = x ^ (x & lb[j]);
}
}
void andNotScalar(Uint32List a, Uint32List b) {
int i = 0;
final n = a.length;
for (; i + 7 < n; i += 8) {
a[i] &= ~b[i];
a[i + 1] &= ~b[i + 1];
a[i + 2] &= ~b[i + 2];
a[i + 3] &= ~b[i + 3];
a[i + 4] &= ~b[i + 4];
a[i + 5] &= ~b[i + 5];
a[i + 6] &= ~b[i + 6];
a[i + 7] &= ~b[i + 7];
}
for (; i < n; i++) a[i] &= ~b[i];
}Observed on Dart SDK 3.10.7 & 3.13.0-edge.5748206f2fbca39ffccd155c93b186f19bc28260 on macos_arm64 (Apple Silicon, macOS 26.3.1)
There's also the question of what happens when SIMD is not available? (i.e. Dart running on a chip that doesn't support the right SIMD primitives) Since there's no reliable way to detect that, the fallback should ideally be in the ballpark of the scalar solution, so I essentially see two issues here:
- why is AOT not as fast as JIT?
- if the fallback in this case is a simulated scalar path, why is it so much slower than the handwritten scalar version?
(cc @mraleph)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
mraleph commentedon Apr 21, 2026
I am pretty sure it is a duplicate to #53662 (comment) - we are missing rules to correctly specialize
Int32x4.operator+andInt32x4.operator&in AOT mode. So you end up with a code that is worse then simple scalar equivalent because it is full of boxing and calls.We mostly assume the lowest common denominator when compiling (we assume SSE2 on Intel and I think NEON on ARM), there are no fallback paths. If you try to run AOT compiled Dart code on something that does not support all features which were assumed during compilation it will crash.
TBH for a long time I just want to rip current SIMD support out and replace it with a "user-space" intrinsic system.
[vm/compiler] Specialize Int32x4 operators in AOT.
modulovalue commentedon Apr 22, 2026
Fix landed in https://dart-review.googlesource.com/c/sdk/+/497000 so closing.