Implement shared native memory multithreading

[mraleph]

The Shared Memory Multithreading for Dart proposal outlines a number of modifications to the Dart native runtime and core libraries which would unlock interoperability use cases which currently require developers to writing additional native code.

These modifications include:

• Introduction of shared fields - static fields which are shared between all isolates in a group. In shared native memory multithreading only fields containing trivially shareable objects are allowed to be marked as shared.
• Introduction of shared isolates - each isolate group has a dedicated isolate which does not contain any isolated state. An attempt to run code in this isolate which accesses non-shared field will throw an exception.
• API extensions to dart:ffi:
  • NativeCallable.shared

There might be other changes to core libraries which would help the cause but is not necessarily required:

• Helpers for atomic reads and writes in dart:ffi.
• API extensions to dart:isolate
  • Isolate.shared
  • Isolate.postTask.
  • Isolate.runSync.
• Introduction of dart:concurrent.
  • AtomicRef, AtomicInt32, AtomicInt64
  • Mutex and Condition.

These changes do not fundamentally alter Dart's isolate model - pure Dart code running in different isolates remains isolated, but introduce more straightforward way to bridge concurrency chasm between native world (which is structured around threads) and Dart (which is structured around isolates).

[mraleph]

@leafpetersen @lrhn From my perspective this subset of changes does not really fundamentally change Dart language capabilities (pure Dart code remains isolated) so I would like to move forward, implement and ship this not guarded by any flag so that our interop efforts could rely on it. Any opposition to this plan?

cc @aam @a-siva @mkustermann @mit-mit

[lrhn]

If you think the feature will be useful as written, then I have no problem with trying try to implement it.
It might be the only way to see if it's viable.

I'm not promising to change the implementation of platform libraries to be able to run without access to non-constant static state, or to make all such fields shared (which likely isn't viable anyway). That means that an unknown-size part of the platform libraries cannot be used in a shared isolate. I do fear that the that part might contain some essential APIs.

If all the shared isolate code does is computation on input data to produce output data, then maybe that's not a problem. Maybe it's just a matter of managing expectations.

[ClementBeal]

I'm very excited by this feature! I believe it would speed up a lot of code u

To visualize what would be the development of code with the shared memory, I wrote this quick sort with the shared fields and shared isolates.

Edit : Not correct. See #56841 (comment)

/**
 * Quick sort
 */

import 'dart:isolate';
import 'dart:math';
import 'dart:typed_data';

Future<void> quickSort(Int64List array, int low, int high) async {
  if (low < high) {
    int partitionIndex = partition(array, low, high);

    await Future.wait(
      [
        Isolate.runShared(
          () => quickSort(array, low, partitionIndex - 1),
        ),
        Isolate.runShared(
          () => quickSort(array, partitionIndex + 1, high),
        ),
      ],
    );
  }
}

int partition(Int64List array, int low, int high) {
  int pivot = array[high];
  int i = (low - 1);

  for (var j = low; j < high; j++) {
    if (array[j] <= pivot) {
      swap(array, ++i, j);
    }
  }

  swap(array, i + 1, high);

  return i + 1;
}

void swap(Int64List array, int i, int j) {
  final tmp = array[i];
  array[i] = array[j];
  array[j] = tmp;
}

void main() async {
  final random = Random();
  final List<int> numbers = List.generate(10000, (i) => random.nextInt(999999));
  final shared Int64List toSort = Int64List.fromList(numbers);

  Isolate.runShared(() {
    quickSort(toSort, 0, numbers.length - 1);
  });
}

I wonder if the shared keyword or the pragma (I'm not sure about it) will be passed to the function parameters like this Future<void> quickSort(shared Int64List array, int low, int high) async.

[mraleph]

@ClementBeal

To visualize what would be the development of code with the shared memory, I wrote this quick sort with the shared fields and shared isolates.

I just want to point out a couple of things.

First of all, the type of code you have written can already be written without any issues: you can allocate memory for Int64List outside of the Dart heap and then share that between multiple isolates. Then process that concurrently.

Second, the code as written has a number of issues:

• The proposal does not introduce shared on local variables. So it will be simply final Int64List toSort.
• The closure you have written captures numbers which means runShared will actually throw an ArgumentError because that is not allowed.
• The closure you have written uses async which will not work in shared isolate.

So this code is not really the best example for this feature.

[ClementBeal]

Thanks you for your explanation. I understand better now

[venbrinoDev]

Hi @mraleph,

I’ve been working with isolates and came across an issue — well, not really an issue, but more of a question regarding how Dart handles shared memory. Specifically, I have a socket.io instance that’s spawned by a native block of code, and I was looking for a way to access the same instance across the main isolate.

Would using Shared Isolates or a shared static field help in this case? Or is my design approach flawed?

The goal is to be able to access the same SocketIo instance across different isolates. For example:

shared SocketIo instance = io();

That way, I could access this instance from any isolate. Any advice or guidance on this would be greatly appreciated!

[mraleph]

@venbrinoDev whether or not we will allow sharing arbitrary mutable objects between isolate is still not fully decided. The first step we are planning to make - shared native memory - will not help you, because Socket¹ is a full fledged mutable Dart object, it can't be made trivially shareable.

Here is a question: why do you have two isolates both connecting the server? what is the second isolate doing?

If you really need it, then you need to design this in a different way: one isolate creates and maintains the connection and all other isolates need to communicate with that isolate using ports. One isolate becomes kinda like a proxy for a connection. (Unfortunately Dart isolates don't facilitate this sort of thinking because they are designed as a coarse grained fork of the whole program).

Footnotes

1. I assume you are using this package: https://github.com/rikulo/socket.io-client-dart ↩

[nguyenhothanhtam0709]

Are there currently any working on this?

[mraleph]

@aam is working on this.

[nguyenhothanhtam0709]

@mraleph , thanks for your response. Hope to see this feature in Dart soon.