Missing Dart_InitializeApiDL symbol in plugin using package:objective_c

[alevlako]

I wrote and tested my code in Swift, made a wrapper for it in Objective C, everything works up to this point. Now I need to call this code from Dart. I turned to the ffigen package and for several days I could not achieve a working result. Then I decided to check the simplest option - and I found that even base Swift example here does not work (
I have the macOs 13.7 installed.

I get the following error message when I try to run the example after all the preparation described in the readme (for default example pubspec.yaml):
swift_api_bindings.dart:26780:12: Error: Can't declare a member that conflicts with an inherited one.
bool get isProxy {
^^^^^^^
../../../objective_c/lib/src/objective_c_bindings_generated.dart:6785:8: Context: This is the inherited member.
bool isProxy() {
^^^^^^^
Or for dependencies: objective_c: ^3.0.0, ffigen: ^15.0.0:
swift_api_bindings.dart:25294:12: Error: Can't declare a member that conflicts with an inherited one.
bool get isProxy {
^^^^^^^
../../../../../../../.pub-cache/hosted/pub.dev/objective_c-3.0.0/lib/src/objective_c_bindings_generated.dart:6785:8: Context: This is the inherited member.
bool isProxy() {
^^^^^^^

[liamappelbe]

Looks like the swift example is hitting #1220

[liamappelbe]

@alevlako Do want to file a separate bug to discuss the issues you're having with your own Swift bindings? The first thing to check is whether you can see the APIs you expect in the generated ObjC header.

[alevlako]

@liamappelbe I see in generated swift_api.h following code:

SWIFT_CLASS("_TtC12swift_module10SwiftClass")
@interface SwiftClass : NSObject
- (NSString * _Nonnull)sayHello SWIFT_WARN_UNUSED_RESULT;
@property (nonatomic) NSInteger someField;
- (nonnull instancetype)init OBJC_DESIGNATED_INITIALIZER;
@end

I commented out getter isProxy in generated file swift_api_bindings.dart and was able to receive the error in this example that I get in my project:

Exception has occurred.
ArgumentError (Invalid argument(s): Couldn't resolve native function 'Dart_InitializeApiDL' in 'objective_c.framework/objective_c' : No asset with id 'objective_c.framework/objective_c' found. No available native assets. Attempted to fallback to process lookup. dlsym(RTLD_DEFAULT, Dart_InitializeApiDL): symbol not found.
)

The exception occurs here (in last line of the code, in generated file swift_api_bindings.dart ):

class SwiftClass extends objc.NSObject {
  SwiftClass._(ffi.Pointer<objc.ObjCObject> pointer,
      {bool retain = false, bool release = false})
      : super.castFromPointer(pointer, retain: retain, release: release);

I am using the actual versions of dependencies:
dependencies:
ffi: ^2.1.3
objective_c: ^3.0.0

[liamappelbe]

Ok, let's set aside the swift example for now (I know what the issue is there, and I'll fix that separately). Let's just focus on your project.

Couldn't resolve native function 'Dart_InitializeApiDL' isn't related to swift bindings specifically. It means that whatever dylibs are being loaded don't contain that symbol.

Can you tell me more about your project? Are you writing a flutter app or a dart command line tool? What does your project do? What are you using swift interop for?

From that error, my guess is you're writing a dart tool, not a flutter app. That's possible, but has some limitations (certain Apple APIs won't work in a pure dart tool, such as UI APIs).

[alevlako]

@liamappelbe thank you for your interest in my problem.

The ultimate goal of my project is to create a flutter plugin package for all desktop operating systems. The Windows part I finished whithin limits of pure dart package, using only WinAPI. For Mac and Linux I need work with ffi for access to native libs. I already realized that pure dart package will not enough for this purpose. In particular, in MacOS, my code accesses the Carbon library, or if I want to be more accurate its part HIToolbox/TextInputSources.h.

At this stage, I can call my swift library in the Flutter application, which runs from Xcode or which is launched in macOS after release build in VSCode. The application crashes when I launch code in debug mode in VSCode because of carbon call.
And I have not yet been able to include my library wtitten in swift into the flutter plugin, not into the application that should use this plugin.

[liamappelbe]

Ok, if you're writing an ObjC/Swift plugin, then this shouldn't be too hard. The missing Dart_InitializeApiDL symbol is supposed to come from the package:objective_c plugin. Are you including that as a dependency? If you have package:objective_c as a dep, it's supposed to "just work", but this is still a relatively new plugin, so there might be a bug there.

[liamappelbe]

The examples are fixed, so this bug is now tracking @alevlako's specific issue, where the missing symbol error is showing up in a flutter plugin.

@alevlako The next step to debug this is to make sure that your plugin depends on the package:objective_c plugin, then build your plugin, then find your plugin's dylib and use nm to list its symbols. I'm wondering if Dart_InitializeApiDL is missing from the plugin (which means the error is in the build), or if the symbol is there but not being loaded (which means the plugin dylib is not being loaded correctly at runtime). Something like nm path/to/plugin.dylib | grep Dart_InitializeApiDL should work.

[alevlako]

@liamappelbe thank you for your assistance, I really managed to advance thanks to your help.
I create ffi pugin for macos: flutter create --template=plugin_ffi --platforms=macos my_ffi_plugin
I remove src folder from plugin and add 2 classes (.h and .m) into folder macos/Classes. My Objective C code calls to functions of different frameworks and do it as expected, without any error.
The code also handles calling function TISCopyCurrentKeyboardLayoutInputSource() of Carbon framework. But when code calling a function TISGetInputSourceProperty, the plugin crashes. I am attach a screenshot of Xcode in the desribed moment. Objective C code tested in pure macos inviroment without Flatter wrapping and it works. Does this case require opening a new issue?

[liamappelbe]

Does TISGetInputSourceProperty need to be called on the main thread (aka platform thread)? Thread restricted APIs can be problematic for FFI since Dart isolates don't always run on the main/platform thread. A future version of flutter will merge the UI thread and main thread so this is less likely to be a problem.

In the meantime you can use runOnPlatformThread which is like Isolate.run, but runs the callback on the main/platform thread. So whatever Dart code is initiating this FFI call, move it into runOnPlatformThread.

[alevlako]

@liamappelbe Thanks to your hint, the application does not crash, but also does not do the useful work for which it is created. Looks like I need to try using the method channel instead of ffi (

[liamappelbe]

@alevlako I forgot that runOnPlatformThread is only supported on iOS and Android, sorry. Still, method channels shouldn't be necessary.

For MacOS I'd recommend writing a little bit of ObjC code that dispatches that call to the main thread. Since your callback didn't run, we still don't actually know if your original crash was caused by the call needing to be done on the main thread. So before you get too deep into writing that native code, try writing a little test function to check if dispatching to the main thread fixes the crash:

void callCarbonOnMainThread() {
  dispatch_async(dispatch_get_main_queue(), ^{
    // Call carbon.
    // Log/print the result to confirm that it works.
  });
}

If that works, then we can talk about how to send the result back to Dart land.

[alevlako]

@liamappelbe great, it works!
How do you recommend returning the result to Dart side?
What do you suggest if I need to return not a simple string, but a list of custom objects? I will need to create some codec/wrapper/parser for my objects?

[liamappelbe]

@alevlako It's actually pretty simple. Just pass a listener block to callCarbonOnMainThread. Assuming you want the objects in an NSArray, it'll look something like this:

// Objective-C
void callCarbonOnMainThread(void (^completionHandler)(NSArray *)) {
  dispatch_async(dispatch_get_main_queue(), ^{
    // Call carbon.
    completionHandler(result);
  });
}

Add callCarbonOnMainThread to your ffigen config, if you haven't already, and it'll generate bindings for that type of block, probably named something like ObjCBlock_ffiVoid_NSArray. Then you can create a listener for this block from a Dart function. The listener will be invoked in whatever isolate you create it in, so you can use that to receive your result:

// Dart
final completionHandler = ObjCBlock_ffiVoid_NSArray.listener((NSArray result) {
  callCarbonResult = result;
});

callCarbonOnMainThread(completionHandler);

On some older versions of ffigen we've had bugs to do with memory management of blocks. So if you see a crash when the completion handler is invoked, try holding a reference to it on the Dart side (eg stick it in a global variable). Hopefully that bug won't affect you though.

[alevlako]

@liamappelbe Unfortunately, the proposed solution produces an error even without any implementation, at the call protocol level. What am I missing?

//  SimpleCalls.h
- (NSString * _Nonnull)carbon_call;
- (void)call_carbon_on_main_thread:(void (^_Nonnull)(NSString * _Nonnull))completion_handler;

//  SimpleCalls.m
- (NSString * _Nonnull) carbon_call {
    dispatch_async(dispatch_get_main_queue(), ^{
        NSLog(@"A few carbon calls are working here as expected");
    });
    return @"carbon_call only prints to console";
}
- (void)call_carbon_on_main_thread:(void (^ _Nonnull __strong)(NSString * _Nonnull __strong))completion_handler {
    NSLog(@"-----> call_carbon_on_main_thread");
}

Now I generate the dart file with ffigen (the package version is up to date: ^16.0.0)

//   lib/ffigen_app.dart
late final _macos_bindings;

void initLib() {
  DynamicLibrary.open('$_libName.framework/$_libName');
  _macos_bindings = SimpleCalls.new1();
}

String callCarbon() {
  return _macos_bindings.carbon_call().toString();
}

Future<String> callCarbonAsync() async {
  final completer = Completer<String>();
  final completionHandler = ObjCBlock_ffiVoid_NSString.listener(
      (NSString result) => completer.complete(result.toString()));
  _macos_bindings.call_carbon_on_main_thread(completionHandler);
  return completer.future;
}

Method callCarbon prints mesage to console.
But I am receive the following exception for method callCarbonAsync:

[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Couldn't resolve native function '_FfigenAppBindings_wrapListenerBlock_1jdvcbf' in 'package:ffigen_app/ffigen_app_bindings_generated.dart' : No asset with id 'package:ffigen_app/ffigen_app_bindings_generated.dart' found. No available native assets. Attempted to fallback to process lookup. dlsym(RTLD_DEFAULT, _FfigenAppBindings_wrapListenerBlock_1jdvcbf): symbol not found.

[liamappelbe]

@alevlako That error means you're not linking the ffigen generated ObjC code into your app. Newer versions of ffigen generate a little bit of ObjC trampoline code for each listener block (to fix that memory management issue I mentioned). It'll be sitting next to the generated dart file (same name, with .m at the end). You need to compile that file into your app, along with all your other ObjC code.

We should probably make that error more descriptive.

[alevlako]

@liamappelbe FYI, I tested all my methods that call the Carbon library in the ffigen swift example. They work as intended without using dispatch_async(dispatch_get_main_queue() or DispatchQueue.global(qos: .background).async

[liamappelbe]

@liamappelbe FYI, I tested all my methods that call the Carbon library in the ffigen swift example. They work as intended without using dispatch_async(dispatch_get_main_queue() or DispatchQueue.global(qos: .background).async

I assume these are thread locked APIs? Yeah, very recent versions of Flutter have started merging the UI thread and the platform thread, so that makes calling these sorts of APIs simpler.

[alevlako]

@liamappelbe No, it's only possible in the mentioned example (native/pkgs/ffigen/swift). When I try to use ffigen & objective_c packages in the standard plugin_ffi example of the Flutter, I still can't do Carbon calls without these methods. At least, I haven't succeeded yet. I wrote one working function in Objective C. Estimating the time I'll have to spend to rewrite my swift code to Objective C, I'm inclined to abandon the idea of ​​using ffi in favor of method channel. Also, the dart documentation describes creating a .dylib library, not a .framework from swift code.
In principle, the method channel performance is quite sufficient for my code. But I have a slight regret that I was not able to bring the code using ffi to a working version on MacOS. I did not encounter such problems in Windows, everything works there.

[liamappelbe]

It depends what version of Flutter you're using. I think the thread merging I mentioned is only available in Flutter 3.29, and only on iOS/Android. Not sure what the situation is on macOS.

If you're on an older version of Flutter you can use runOnPlatformThread to run a Dart function on the platform thread. You should be able to invoke thread locked APIs in there. Again that's only on iOS/Android as far as I know.