Skip to content

SIMD AOT 10-20x slower. #63217

Closed
Closed

Description

@modulovalue

I'm optimizing a bit array implementation in Dart with SIMD. JIT is much faster, as expected. However, AOT is 10-20x slower. My expectation was that the performance gains would carry over to AOT.

Output JIT:

words=2048 (65536 bits)  iters=200000
or      scalar: 0.896 us/iter  (sink=23552)
or      simd  : 0.419 us/iter  (sink=23552)
  -> simd/scalar = 0.47x

and_not scalar: 1.101 us/iter  (sink=9216)
and_not simd  : 0.328 us/iter  (sink=9216)
  -> simd/scalar = 0.30x

Output AOT:

words=2048 (65536 bits)  iters=200000
or      scalar: 1.090 us/iter  (sink=23552)
or      simd  : 12.494 us/iter  (sink=23552)
  -> simd/scalar = 11.46x

and_not scalar: 0.997 us/iter  (sink=9216)
and_not simd  : 23.282 us/iter  (sink=9216)
  -> simd/scalar = 23.36x

Repro:

import 'dart:typed_data';

const int words = 2048;
const int iters = 200000;

void main() {
  print('words=$words (${words * 32} bits)  iters=$iters');
  final orS = bench('or      scalar', orScalar);
  final orV = bench('or      simd  ', orSimd);
  print('  -> simd/scalar = ${(orV / orS).toStringAsFixed(2)}x\n');
  final anS = bench('and_not scalar', andNotScalar);
  final anV = bench('and_not simd  ', andNotSimd);
  print('  -> simd/scalar = ${(anV / anS).toStringAsFixed(2)}x');
}

double bench(String label, void Function(Uint32List, Uint32List) op) {
  final a = Uint32List(words);
  final b = Uint32List(words);
  for (int i = 0; i < words; i++) {
    a[i] = 0xA5A5A5A5 ^ i;
    b[i] = 0x5A5A5A5A ^ (i * 31);
  }
  for (int w = 0; w < 50; w++) op(a, b); // warmup
  final sw = Stopwatch()..start();
  for (int k = 0; k < iters; k++) op(a, b);
  sw.stop();
  int sink = 0;
  for (int i = 0; i < words; i++) sink ^= a[i]; // prevent DCE
  final usPerIter = sw.elapsedMicroseconds / iters;
  print('$label: ${usPerIter.toStringAsFixed(3)} us/iter  (sink=$sink)');
  return usPerIter;
}

void orSimd(Uint32List a, Uint32List b) {
  final la = Int32x4List.view(a.buffer, a.offsetInBytes, a.length >> 2);
  final lb = Int32x4List.view(b.buffer, b.offsetInBytes, b.length >> 2);
  for (int j = 0; j < la.length; j++) {
    la[j] = la[j] | lb[j];
  }
}

void orScalar(Uint32List a, Uint32List b) {
  int i = 0;
  final n = a.length;
  for (; i + 7 < n; i += 8) {
    a[i] |= b[i];
    a[i + 1] |= b[i + 1];
    a[i + 2] |= b[i + 2];
    a[i + 3] |= b[i + 3];
    a[i + 4] |= b[i + 4];
    a[i + 5] |= b[i + 5];
    a[i + 6] |= b[i + 6];
    a[i + 7] |= b[i + 7];
  }
  for (; i < n; i++) a[i] |= b[i];
}

void andNotSimd(Uint32List a, Uint32List b) {
  // Int32x4 has no bitwise NOT, so use the identity a & ~b == a ^ (a & b).
  final la = Int32x4List.view(a.buffer, a.offsetInBytes, a.length >> 2);
  final lb = Int32x4List.view(b.buffer, b.offsetInBytes, b.length >> 2);
  for (int j = 0; j < la.length; j++) {
    final x = la[j];
    la[j] = x ^ (x & lb[j]);
  }
}

void andNotScalar(Uint32List a, Uint32List b) {
  int i = 0;
  final n = a.length;
  for (; i + 7 < n; i += 8) {
    a[i] &= ~b[i];
    a[i + 1] &= ~b[i + 1];
    a[i + 2] &= ~b[i + 2];
    a[i + 3] &= ~b[i + 3];
    a[i + 4] &= ~b[i + 4];
    a[i + 5] &= ~b[i + 5];
    a[i + 6] &= ~b[i + 6];
    a[i + 7] &= ~b[i + 7];
  }
  for (; i < n; i++) a[i] &= ~b[i];
}

Observed on Dart SDK 3.10.7 & 3.13.0-edge.5748206f2fbca39ffccd155c93b186f19bc28260 on macos_arm64 (Apple Silicon, macOS 26.3.1)

There's also the question of what happens when SIMD is not available? (i.e. Dart running on a chip that doesn't support the right SIMD primitives) Since there's no reliable way to detect that, the fallback should ideally be in the ballpark of the scalar solution, so I essentially see two issues here:

  • why is AOT not as fast as JIT?
  • if the fallback in this case is a simulated scalar path, why is it so much slower than the handwritten scalar version?

(cc @mraleph)

Activity

  1. mraleph commented on Apr 21, 2026

    @mraleph
    Member

    why is AOT not as fast as JIT?

    I am pretty sure it is a duplicate to #53662 (comment) - we are missing rules to correctly specialize Int32x4.operator+ and Int32x4.operator& in AOT mode. So you end up with a code that is worse then simple scalar equivalent because it is full of boxing and calls.

    There's also the question of what happens when SIMD is not available?

    We mostly assume the lowest common denominator when compiling (we assume SSE2 on Intel and I think NEON on ARM), there are no fallback paths. If you try to run AOT compiled Dart code on something that does not support all features which were assumed during compilation it will crash.

    TBH for a long time I just want to rip current SIMD support out and replace it with a "user-space" intrinsic system.

  2. modulovalue commented on Apr 22, 2026

    @modulovalue
    ContributorAuthor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions