Library feature for isolate-shared variables.

[lrhn]

The "shared native-memory multithreading" proposal includes "shared variables", which are static/top-level variables which share the same storage location across all isolates in an isolate group.

Instead of making this an annotation on the variable, it could be made a library feature, a class with a getter and setter which read from an isolate-shared storage location. (As someone suggested in a recent meeting.)

By doing so, more operations can be defined on the shared variable, operations that are only needed for shared variables.

Example of what such a class could look like:

/// An isolate-shared storage location.
///
/// An instance of [Shared] represents a single isolate-group shared
/// storage location.
///
/// A `const` allocated [Shared] value represents the same location
/// in each isolate of the isolate group, and preserves its identity
/// when sent to other isolates.
/// A new allocated [Shared] value creates a new shared storage location,
/// and can also be sent to other isolates in the same isolate group.
final class Shared<T> {
  /// Creates a new isolate-shared storage location.
  ///
  /// If created using `const`, each invocation has a separate identity,
  /// and two `const Shared<int>(4)` expressions are not canonicalized.
  /// The instance preserves its identity.
  ///
  /// Separate isolates in the same isolate group can refer to the same
  /// constant instance, and read and shared write the stored value
  ///
  /// If created as non-constant, each object represents a new
  /// shared storage location.
  /// The [Shared] object can be sent to other isolates
  /// in the same isolate group.
  const Shared(T initialValue);
  
  /// Shared value.
  external T value;
  
  /// Exchanges the [newValue] and the [value].
  ///
  /// Reads the current value and writes a new value as an atomic operation.
  ///
  /// Returns the read value.
  ///
  /// The operation is atomic.
  external T update(T newValue);

  /// Copies the value from [other].
  ///
  /// Returns the new value.
  ///
  /// The operation is atomic.
  external T copyFrom(Shared<T> other);

  /// Swaps the values of this varible and [other].
  ///
  /// The operation is atomic.
  external void swap(Shared<T> other);
}

extension SharedInt on Shared<int> {
  /// Increments value by one and returns the new value.
  ///
  /// The operation is atomic.
  external int increment();

  /// Decrements value by one and returns the new value.
  ///
  /// The operation is atomic.
  external int artomicDecrement();
  
  /// Increments the value only if it's equal to [value].
  ///
  /// Returns the new or unchanged value.
  ///
  /// The operation is atomic.
  external int compareIncrement(int value);

  /// Decrements the value only if it's equal to [value].
  ///
  /// Returns the new or unchanged value.
  ///
  /// The operation is atomic.
  external int compareIncrement(int value);
}

extension SharedBool on Shared<bool> {
  /// Toggles the boolean value.
  ///
  /// Returns the new value.
  ///
  /// The operation is atomic.
  external bool atomicToggle();
}

It avoids depending on annotations, and it makes it very explicit when you are reading a shared variable.

It does mean that the migration from unshared variable to shared gets more comlicated:

int _globalCounter = 0;

becomes

const _globalCounterVar = Shared<int>();
int get _globalCounter => _globalCounterVar.value;
get _globalCounter(int value) { _globalCounterVar.value = value; }

instead of just adding an annotation.

It also allows specifying operations specifically on such variables.

[Reprevise]

Just to be 100% clear: you're talking about changing shared int sharedGlobal = 0 into int sharedGlobal = Shared(0), correct?. Sorry I just don't normally refer to keywords as annotations.

[lrhn]

The shared would not be a keyword, since shared variables is a VM-specific feature, not a language feature, it would be @shared int sharedGlobal = 0; (IMO).

It would probably be const Shared<int> sharedGlobal = Shared<int>(0);.

And then it's also possible to have other kinds of variables, like maybe a per-isolate variable that can be accessed in a shared isolate, const IsolateVar<int> nonSharedGlobal = IsolateVar<int>(42); or const IsolateVar<int> nonSharedGlobal = IsolateVar<int>.late(_compute); int _compute() => someComputation();, which could compute the result only once, and then share it among all isolates.

[Reprevise]

@lrhn I was looking at this code sample from the proposal. If it's not going to be a keyword, then I'd much rather have it be a class than an annotation.

int global = 0;

shared int sharedGlobal = 0; // This looks like a keyword

void main() async {
  global = 42;
  sharedGlobal = 42;
  await Isolate.runShared(() {
    print(global);
    global = 24;

    print(sharedGlobal);  // => 42
    sharedGlobal = 24;
  });
  print(global);  // => 42
  print(sharedGlobal);  // => 24
}

[mraleph]

There are some things I like about this proposal (e.g. every shared field automatically gets atomic APIs) and some things I don't like. Being shared or not-shared is an attribute of a location not the attribute of a value stored in that location. Similar to how final is an attribute of a location. Because of that it feels that @shared (or @pragma('vm:shared')) should be on the same level as final.

We would need FinalShared<T> and FinalLateShared<T> to express all things which are possible with variables.

pragma('vm:shared') is also required on captured variables to make closure shareable. So we would also need to use Shared<T> there.

final x = Shared<int>(10);
(() { use(x.value); }) 

It gets awkward. Also it is relatively easy to make a mistake and write final x = const Shared<int>(10) here.

Finally, this part of the proposal:

  /// If created using `const`, each invocation has a separate identity,
  /// and two `const Shared<int>(4)` expressions are not canonicalized.
  /// The instance preserves its identity.

Requires some sort of language feature. I guess it could be tied to source location (similar to what widget transformer is doing - so maybe two birds can be killed here with one stone here).

[lrhn]

The non-canonicalization based on arguments does imply some compiler hack. It doesn't have to be anything more complicated than a hidden ID argument that is unique to each const invocation location.

Imagine a constructor of the form:

class Shared<V> {
  external V value;
  const Shared(V value) : this._(value, __constCallerId__);
  external const Shared._(V value, Object? id);
}

where __constCallerId__ is a magical potentially constant expression which evaluates to a different value for each const invocation of the constructor, including each different const invocation.

Then each instance of Shared corresponds to one shared storage location.
We can definitely have final and late shared variables.

final class Shared<V> {
  external V value;
  /// Shared mutable variable initialized to [value].
  external const Shared(V value);
  /// Shared mutable variable with optional initializer.
  ///
  /// Throws if [value] is read before written when no [initializer] is provided.
  /// Otherwise evaluates [initializer] on first read and assigns the
  /// result before returning it to [value].
  external const Shared.late([V Function()? initializer]);
}
final class FinalShared<V> implements Shared {
  /// Shared unmodifiable value.
  external FinalShared(V value);
  /// Shared unmodifiable lazily-initialized value.
  external FinalShared.late(V Function() initializer);
}