[vm] Introduce pragma vm:deeply-immutable

This CL introduces a way to mark all instances of a class as deeply
immutable.

In order to statically verify that all instances of a deeply immutable
class are immutable, a deeply immutable classes must have the following
properties:

1. All instance fields must
   1. have a deeply immutable type,
   2. be final, and
   3. be non-late.
2. The class must be `final` or `sealed`. This ensures no
   non-deeply-immutable subtypes are added by external code.
3. All subtypes must be deeply immutable. This ensures 1.1 can be
   trusted.
4. The super type must be deeply immutable (except for Object).

Note that instances of some classes in the VM are deeply immutable
while their class cannot be marked immutable.

* SendPort, Capability, RegExp, and StackTrace are not `final` and
  can be implemented by external code.
* UnmodifiableTypedDataViews do not have a public type. (It was
  recently deprecated.)

See runtime/docs/deeply_immutable.md for more details.

Use case:

This enables attaching a `Dart_FinalizableHandle` to a deeply immutable
object and the deeply immutable object with other isolates in the same
isolate group.

(Note that `NativeFinalizer`s live in an isolate, and not an isolate
group. So this should currently _not_ be used with `NativeFinalizer`s.
See #55062 for making a
`NativeFinalizer.shared(` that would live in an isolate group instead
of in an isolate.)

Implementation details:

Before this CL, the `ImmutableBit` in the object header was only ever
set to true for predefined class ids (and for const objects). After
this CL, the bit can also be set to true for non const instances of
user-defined classes. The object allocation and initialization code has
been changed to deal with this new case. The immutability of a class is
saved in the class state bits. On object allocation and initialization
the immutability bit is read from the class for non-predefined class
ids.

TEST=runtime/tests/vm/dart/isolates/fast_object_copy2_test.dart
TEST=runtime/vm/isolate_reload_test.cc
TEST=tests/lib/isolate/deeply_immutable_*

Bug: #55120
Bug: #54885
Change-Id: Ib97fe589cb4f81673cb928c93e3093838d82132d
Cq-Include-Trybots: luci.dart.try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-kernel-linux-debug-x64-try,vm-kernel-precomp-linux-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354902
Commit-Queue: Daco Harkes <[email protected]>
Reviewed-by: Martin Kustermann <[email protected]>

Author: dcharkes

pkg/_fe_analyzer_shared/lib/src/messages/codes_generated.dart

@@ -5994,6 +5994,73 @@ const MessageCode messageFfiCreateOfStructOrUnion = const MessageCode(
       r"""Subclasses of 'Struct' and 'Union' are backed by native memory, and can't be instantiated by a generative constructor. Try allocating it via allocation, or load from a 'Pointer'.""",
 );
 
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const Code<Null> codeFfiDeeplyImmutableClassesMustBeFinalOrSealed =
+    messageFfiDeeplyImmutableClassesMustBeFinalOrSealed;
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const MessageCode messageFfiDeeplyImmutableClassesMustBeFinalOrSealed =
+    const MessageCode(
+  "FfiDeeplyImmutableClassesMustBeFinalOrSealed",
+  problemMessage: r"""Deeply immutable classes must be final or sealed.""",
+  correctionMessage: r"""Try marking this class as final or sealed.""",
+);
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const Code<Null> codeFfiDeeplyImmutableFieldsModifiers =
+    messageFfiDeeplyImmutableFieldsModifiers;
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const MessageCode messageFfiDeeplyImmutableFieldsModifiers = const MessageCode(
+  "FfiDeeplyImmutableFieldsModifiers",
+  problemMessage:
+      r"""Deeply immutable classes must only have final non-late instance fields.""",
+  correctionMessage:
+      r"""Add the 'final' modifier to this field, and remove 'late' modifier from this field.""",
+);
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const Code<Null> codeFfiDeeplyImmutableFieldsMustBeDeeplyImmutable =
+    messageFfiDeeplyImmutableFieldsMustBeDeeplyImmutable;
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const MessageCode messageFfiDeeplyImmutableFieldsMustBeDeeplyImmutable =
+    const MessageCode(
+  "FfiDeeplyImmutableFieldsMustBeDeeplyImmutable",
+  problemMessage:
+      r"""Deeply immutable classes must only have deeply immutable instance fields. Deeply immutable types include 'int', 'double', 'bool', 'String', 'Pointer', 'Float32x4', 'Float64x2', 'Int32x4', and classes annotated with `@pragma('vm:deeply-immutable')`.""",
+  correctionMessage:
+      r"""Try changing the type of this field to a deeply immutable type or mark the type of this field as deeply immutable.""",
+);
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const Code<Null> codeFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable =
+    messageFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable;
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const MessageCode messageFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable =
+    const MessageCode(
+  "FfiDeeplyImmutableSubtypesMustBeDeeplyImmutable",
+  problemMessage:
+      r"""Subtypes of deeply immutable classes must be deeply immutable.""",
+  correctionMessage:
+      r"""Try marking this class deeply immutable by adding `@pragma('vm:deeply-immutable')`.""",
+);
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const Code<Null> codeFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable =
+    messageFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable;
+
+// DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
+const MessageCode messageFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable =
+    const MessageCode(
+  "FfiDeeplyImmutableSupertypeMustBeDeeplyImmutable",
+  problemMessage:
+      r"""The super type of deeply immutable classes must be deeply immutable.""",
+  correctionMessage:
+      r"""Try marking the super class deeply immutable by adding `@pragma('vm:deeply-immutable')`.""",
+);
+
 // DO NOT EDIT. THIS FILE IS GENERATED. SEE TOP OF FILE.
 const Code<Null> codeFfiDefaultAssetDuplicate = messageFfiDefaultAssetDuplicate;
 

pkg/front_end/lib/src/api_unstable/vm.dart

@@ -48,6 +48,11 @@ export '../fasta/codes/fasta_codes.dart'
         messageFfiAbiSpecificIntegerMappingInvalid,
         messageFfiAddressOfMustBeNative,
         messageFfiCreateOfStructOrUnion,
+        messageFfiDeeplyImmutableClassesMustBeFinalOrSealed,
+        messageFfiDeeplyImmutableFieldsModifiers,
+        messageFfiDeeplyImmutableFieldsMustBeDeeplyImmutable,
+        messageFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable,
+        messageFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable,
         messageFfiDefaultAssetDuplicate,
         messageFfiExceptionalReturnNull,
         messageFfiExpectedConstant,

pkg/front_end/messages.status

@@ -382,6 +382,11 @@ FfiAbiSpecificIntegerMappingInvalid/analyzerCode: Fail
 FfiCompoundImplementsFinalizable/analyzerCode: Fail
 FfiCreateOfStructOrUnion/analyzerCode: Fail
 FfiDartTypeMismatch/analyzerCode: Fail
+FfiDeeplyImmutableClassesMustBeFinalOrSealed/analyzerCode: Fail
+FfiDeeplyImmutableFieldsModifiers/analyzerCode: Fail
+FfiDeeplyImmutableFieldsMustBeDeeplyImmutable/analyzerCode: Fail
+FfiDeeplyImmutableSubtypesMustBeDeeplyImmutable/analyzerCode: Fail
+FfiDeeplyImmutableSupertypeMustBeDeeplyImmutable/analyzerCode: Fail
 FfiEmptyStruct/analyzerCode: Fail
 FfiExceptionalReturnNull/analyzerCode: Fail
 FfiExpectedConstant/analyzerCode: Fail

pkg/front_end/messages.yaml

@@ -5165,6 +5165,36 @@ FfiDartTypeMismatch:
   problemMessage: "Expected '#type' to be a subtype of '#type2'."
   external: test/ffi_test.dart
 
+FfiDeeplyImmutableClassesMustBeFinalOrSealed:
+  # Used by dart:ffi
+  problemMessage: 'Deeply immutable classes must be final or sealed.'
+  correctionMessage: 'Try marking this class as final or sealed.'
+  external: test/ffi_test.dart
+
+FfiDeeplyImmutableFieldsMustBeDeeplyImmutable:
+  # Used by dart:ffi
+  problemMessage: "Deeply immutable classes must only have deeply immutable instance fields. Deeply immutable types include 'int', 'double', 'bool', 'String', 'Pointer', 'Float32x4', 'Float64x2', 'Int32x4', and classes annotated with `@pragma('vm:deeply-immutable')`."
+  correctionMessage: 'Try changing the type of this field to a deeply immutable type or mark the type of this field as deeply immutable.'
+  external: test/ffi_test.dart
+
+FfiDeeplyImmutableFieldsModifiers:
+  # Used by dart:ffi
+  problemMessage: 'Deeply immutable classes must only have final non-late instance fields.'
+  correctionMessage: "Add the 'final' modifier to this field, and remove 'late' modifier from this field."
+  external: test/ffi_test.dart
+
+FfiDeeplyImmutableSubtypesMustBeDeeplyImmutable:
+  # Used by dart:ffi
+  problemMessage: 'Subtypes of deeply immutable classes must be deeply immutable.'
+  correctionMessage: "Try marking this class deeply immutable by adding `@pragma('vm:deeply-immutable')`."
+  external: test/ffi_test.dart
+
+FfiDeeplyImmutableSupertypeMustBeDeeplyImmutable:
+  # Used by dart:ffi
+  problemMessage: 'The super type of deeply immutable classes must be deeply immutable.'
+  correctionMessage: "Try marking the super class deeply immutable by adding `@pragma('vm:deeply-immutable')`."
+  external: test/ffi_test.dart
+
 FfiDefaultAssetDuplicate:
   # Used by dart:ffi
   problemMessage: "There may be at most one @DefaultAsset annotation on a library."

pkg/front_end/test/spell_checking_list_messages.txt

@@ -45,6 +45,7 @@ dart:js_interop
 dart:js_interop_unsafe
 dart_runner
 dartbug.com
+deeply
 defaultasset
 dname
 e.g
@@ -55,6 +56,8 @@ extensiontype
 f
 ffi
 finality
+float32x
+float64x
 flutter_runner
 function.tojs
 futureor
@@ -63,6 +66,7 @@ guarded
 guides
 h
 https
+int32x
 interact
 interop
 intervening
@@ -79,6 +83,7 @@ loadlibrary
 macro
 member(s)
 migrate
+modifier
 mocking
 n
 name.#name
@@ -106,6 +111,7 @@ patterns
 placing
 pointer`s
 pragma
+pragma('vm:deeply
 preexisting
 pubspec.yaml
 r

pkg/vm/lib/modular/target/vm.dart

@@ -11,6 +11,7 @@ import 'package:kernel/target/changed_structure_notifier.dart';
 import 'package:kernel/target/targets.dart';
 
 import '../transformations/call_site_annotator.dart' as callSiteAnnotator;
+import '../transformations/deeply_immutable.dart' as deeply_immutable;
 import '../transformations/lowering.dart' as lowering
     show transformLibraries, transformProcedure;
 import '../transformations/mixin_full_resolution.dart' as transformMixins
@@ -151,6 +152,13 @@ class VmTarget extends Target {
       ReferenceFromIndex? referenceFromIndex,
       {void Function(String msg)? logger,
       ChangedStructureNotifier? changedStructureNotifier}) {
+    deeply_immutable.validateLibraries(
+      libraries,
+      coreTypes,
+      diagnosticReporter,
+    );
+    logger?.call("Validated deeply immutable");
+
     transformMixins.transformLibraries(
         this, coreTypes, hierarchy, libraries, referenceFromIndex);
     logger?.call("Transformed mixin applications");

pkg/vm/lib/modular/transformations/deeply_immutable.dart

@@ -0,0 +1,156 @@
+// Copyright (c) 2024, the Dart project authors.  Please see the AUTHORS file
+// for details. All rights reserved. Use of this source code is governed by a
+// BSD-style license that can be found in the LICENSE file.
+
+import 'package:front_end/src/fasta/codes/fasta_codes.dart'
+    show
+        messageFfiDeeplyImmutableClassesMustBeFinalOrSealed,
+        messageFfiDeeplyImmutableFieldsModifiers,
+        messageFfiDeeplyImmutableFieldsMustBeDeeplyImmutable,
+        messageFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable,
+        messageFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable;
+import 'package:kernel/ast.dart';
+import 'package:kernel/core_types.dart';
+import 'package:kernel/target/targets.dart' show DiagnosticReporter;
+
+void validateLibraries(
+  List<Library> libraries,
+  CoreTypes coreTypes,
+  DiagnosticReporter diagnosticReporter,
+) {
+  final validator = DeeplyImmutableValidator(
+    coreTypes,
+    diagnosticReporter,
+  );
+  for (final library in libraries) {
+    validator.visitLibrary(library);
+  }
+}
+
+/// Implements the `vm:deeply-immutable` semantics.
+class DeeplyImmutableValidator {
+  static const vmDeeplyImmutable = "vm:deeply-immutable";
+
+  final CoreTypes coreTypes;
+  final DiagnosticReporter diagnosticReporter;
+  final Class pragmaClass;
+  final Field pragmaName;
+
+  DeeplyImmutableValidator(
+    this.coreTypes,
+    this.diagnosticReporter,
+  )   : pragmaClass = coreTypes.pragmaClass,
+        pragmaName = coreTypes.pragmaName;
+
+  void visitLibrary(Library library) {
+    for (final cls in library.classes) {
+      visitClass(cls);
+    }
+  }
+
+  void visitClass(Class node) {
+    _validateDeeplyImmutable(node);
+  }
+
+  void _validateDeeplyImmutable(Class node) {
+    if (!_isDeeplyImmutableClass(node)) {
+      // If class is not marked deeply immutable, check that none of the super
+      // types is marked deeply immutable.
+      final classes = [
+        if (node.superclass != null) node.superclass!,
+        for (final superType in node.implementedTypes) superType.classNode,
+        if (node.mixedInClass != null) node.mixedInClass!,
+      ];
+      for (final superClass in classes) {
+        if (_isDeeplyImmutableClass(superClass)) {
+          diagnosticReporter.report(
+            messageFfiDeeplyImmutableSubtypesMustBeDeeplyImmutable,
+            node.fileOffset,
+            node.name.length,
+            node.location!.file,
+          );
+        }
+      }
+      return;
+    }
+
+    final superClass = node.superclass;
+    if (superClass != null && superClass != coreTypes.objectClass) {
+      if (!_isDeeplyImmutableClass(superClass)) {
+        diagnosticReporter.report(
+          messageFfiDeeplyImmutableSupertypeMustBeDeeplyImmutable,
+          node.fileOffset,
+          node.name.length,
+          node.location!.file,
+        );
+      }
+    }
+
+    // Don't allow implementing, extending or mixing in deeply immutable classes
+    // in other libraries. Adding a `vm:deeply-immutable` pragma to a class that
+    // might be implemented, extended or mixed in would break subtypes that are
+    // not marked deeply immutable. (We could consider relaxing this and
+    // allowing breaking subtypes upon adding the pragma.)
+    if (!(node.isFinal || node.isSealed)) {
+      diagnosticReporter.report(
+        messageFfiDeeplyImmutableClassesMustBeFinalOrSealed,
+        node.fileOffset,
+        node.name.length,
+        node.location!.file,
+      );
+    }
+
+    // All instance fields should be non-late final and deeply immutable.
+    for (final field in node.fields) {
+      if (field.isStatic) {
+        // Static fields are not part of instances.
+        continue;
+      }
+      if (!_isDeeplyImmutableDartType(field.type)) {
+        diagnosticReporter.report(
+          messageFfiDeeplyImmutableFieldsMustBeDeeplyImmutable,
+          field.fileOffset,
+          field.name.text.length,
+          field.location!.file,
+        );
+      }
+      if (!field.isFinal || field.isLate) {
+        diagnosticReporter.report(
+          messageFfiDeeplyImmutableFieldsModifiers,
+          field.fileOffset,
+          field.name.text.length,
+          field.location!.file,
+        );
+      }
+    }
+  }
+
+  bool _isDeeplyImmutableDartType(DartType dartType) {
+    if (dartType is NullType) {
+      return true;
+    }
+    if (dartType is InterfaceType) {
+      final classNode = dartType.classNode;
+      return _isDeeplyImmutableClass(classNode);
+    }
+    if (dartType is TypeParameterType) {
+      return _isDeeplyImmutableDartType(dartType.bound);
+    }
+    return false;
+  }
+
+  bool _isDeeplyImmutableClass(Class node) {
+    for (final annotation in node.annotations) {
+      if (annotation is ConstantExpression) {
+        final constant = annotation.constant;
+        if (constant is InstanceConstant &&
+            constant.classNode == pragmaClass &&
+            constant.fieldValues[pragmaName.fieldReference] ==
+                StringConstant(vmDeeplyImmutable)) {
+          return true;
+        }
+      }
+    }
+    return false;
+  }
+}