Support for external Strings.

[modulovalue]

I was delighted to find Dart_NewExternalXString in dart_api.h. My understanding is that these APIs allow Strings to be backed by native memory.

I've tried using them from within Dart, but I can't manage to get the finalizer machinery to work. My attempt can be found here: dart-lang/sdk#50452.

I was wondering, have the maintainers of package:ffi considered to support e.g. Dart_NewExternalUTF16String for providing a view over native memory as a String?

For my use cases I'd like to share native memory across isolates, and the ability to create Strings from native memory in some zero-cost way would make that much more practical by making it significantly more efficient.

[dcharkes]

I've replied in the other issue.

I was wondering, have the maintainers of package:ffi considered to support e.g. Dart_NewExternalUTF16String for providing a view over native memory as a String?

Yes. However, there are two issues currently:

1. package:ffi does not require compiling any C/C++ code. And passing in a Dart_HandleFinalizer to Dart_NewExternalUTF16String requires defining a C function that does the freeing. In the future, bundling and build C code with Dart packages should become easier, so then we could consider it.
2. In order for Dart_NewExternalXString to be available on Flutter, we should add them to dart_api_dl.h as Dart_NewExternalXString_DL. (Side note: adding a dependency on dart_api_dl.h would make package:ffi's C code by locked in to the dart_version.h, which requires us to make a package:ffi major version bump every time we change that API. This shouldn't be a problem.)

Cross linking related Dart SDK bug:

• [vm/ffi] Investigate copy-free Strings sdk#39787

We haven't done anything about this yet, because we're not aware of any use cases in which the copying is prohibitive. @modulovalue What is your use case? If you can get the Dart_NewExternalXString to work locally, could you report the performance difference?

[modulovalue]

Would it perhaps be an option to expose the ability to create external strings from within NativeApi at dart:ffi similar to how it was done with native finalizers through dart-lang/sdk#35770 e.g.

library dart.ffi

// ...

abstract class NativeApi {
  // ...
  external static String utf16String<T>(
    Pointer<Uint16> storage,
    int size,
    T finalizationToken,
    void Function(T) callback,
  );

  external static String latin1String<T>(
    Pointer<Uint8> storage,
    int size,
    T finalizationToken,
    void Function(T) callback,
  );
  // ...
}

// ...

However, this might not be the best option because Ropes couldn't be optimally implemented with just a contiguous array of memory. (V8, for example, uses its version of Ropes and heuristics to achieve more performant strings on the web.)

---

If you can get the Dart_NewExternalXString to work locally, could you report the performance difference?

I will try to do that.

What is your use case?

There's quite a bit of context that would need to be considered around my use case. I'll try to collect it in one issue and present it as a coherent unit that way.

(In short, I'd like to beat the Web in performance and memory overhead w.r.t. Strings, which is not possible may not be possible today.)