[vm] `NativeCallable.isolateLocal` should enter owned isolate

[liamappelbe]

If NativeCallable.isolateLocal is invoked on a thread that is not entered into any isolate, but is the owner of the target isolate (according to the new isolate ownership API), we can enter that isolate, invoke the callback, then exit it again. Currently this case just FATALs.

Context: dart-lang/native#1908 (comment)

[biggs0125]

@liamappelbe Are you planning on taking a look at this at some point?

[liamappelbe]

Yeah, I've got a CL in development now

[knopp]

Very happy to see this being addressed. Not having to write a shim to manually enter the isolate every time we need a delayed callback on platform thread would be a major improvement.

[knopp]

FYI, this is currently also problem for ObjC bindings:

  final delegate = NSWindowDelegate$Builder.implement(
    windowWillMove_: (_) {
      print('Window will move');
    },
  );
  final window = NSWindow();
  window.styleMask =
      NSWindowStyleMask.NSWindowStyleMaskTitled |
      NSWindowStyleMask.NSWindowStyleMaskClosable |
      NSWindowStyleMask.NSWindowStyleMaskResizable |
      NSWindowStyleMask.NSWindowStyleMaskMiniaturizable;
  window.setFrame(makeRect(100, 100, 800, 600), display: true);
  window.delegate = delegate;
  window.makeKeyAndOrderFront(null);

This will crash with

../../../flutter/third_party/dart/runtime/vm/runtime_entry.cc: 5034: error: Cannot invoke native callback outside an isolate.
version=3.11.0-270.0.dev (dev) (Mon Dec 22 16:03:08 2025 -0800) on "macos_arm64"
pid=67997, thread=259, isolate_group=(nil)(0x0), isolate=(nil)(0x0)
os=macos, arch=arm64, comp=no, sim=no
isolate_instructions=0, vm_instructions=1029b69a0
fp=16f801880, sp=16f801840, pc=10272eea8
  pc 0x000000010272eea8 fp 0x000000016f801880 dart::Profiler::DumpStackTrace(bool)+0x30
  pc 0x00000001025cd320 fp 0x000000016f8018a0 dart::Assert::Fail(char const*, ...) const+0x24
  pc 0x000000010274d324 fp 0x000000016f801930 DLRT_GetFfiCallbackMetadata+0x320
  pc 0x0000000116a43f10 fp 0x000000016f801948 Unknown symbol
  pc 0x000000011683f97c fp 0x000000016f801a30 _NativeLibrary_protocolTrampoline_18v1jvf+0x84
  pc 0x000000011683f97c fp 0x000000016f801aa0 _NativeLibrary_protocolTrampoline_18v1jvf+0x84
  pc 0x000000019b4e346c fp 0x000000016f801af0 __CFNOTIFICATIONCENTER_IS_CALLING_OUT_TO_AN_OBSERVER__+0x94
  pc 0x000000019b572b28 fp 0x000000016f801b30 ___CFXRegistrationPost_block_invoke+0x5c
  pc 0x000000019b572a6c fp 0x000000016f801c10 _CFXRegistrationPost+0x1b4
  pc 0x000000019b4b28b8 fp 0x000000016f8025d0 _CFXNotificationPost+0x2e4
  pc 0x000000019ca6c680 fp 0x000000016f802600 -[NSNotificationCenter postNotificationName:object:userInfo:]+0x58
  pc 0x000000019f607608 fp 0x000000016f802710 __43-[NSApplication _handleAppKitDefinedEvent:]_block_invoke+0x78
  pc 0x000000019f605e08 fp 0x000000016f802740 ___NSAppInstallWindowWillDragEventMonitor_block_invoke+0x94

Even though it is invoked on correct thread owned by the main isolate.

cc @dcharkes

[knopp]

Btw. same issue for package win32. I would want to be able to run win32 applications such as this with flutter zero (a stripped-down flutter runtime), but the thread message loop is pumped outside of the isolate so most windows messages would simply crash.