XCFramework: Unknown library type with extension 'so' in library 'linux-aarch64'

[vsarunas]

Trying to compile on Linux with 6.3:

swift build --build-system swiftbuild --experimental-xcframeworks-on-linux

Gives an error:

error: Failed to load XCFramework at '.build/artifacts/ordo-sdk/OrdoSDK-OrdoPublic/OrdoPublic.xcframework': Unknown library type with extension 'so' in library 'linux-aarch64'.

It seems need to do add additional support in swift-build to handle Linux and dynamic libraries for Linux?
@owenv , is it just a matter of changing several ifdef definitions depending on which platform the code is running on and picking the correct name? I can help out with this.

Running this test fails slightly differently:

swift test --filter FunctionalTests.SwiftPMTests/libraryEvolutionLinuxXCFramework --build-system swiftbuild --experimental-xcframeworks-on-linux

stderr: error: 'swiftframework': Invalid manifest (compiled with: ["/usr/local/share/toolchains/6.3.0/usr/bin/swiftc", "-vfsoverlay", "/tmp/TemporaryDirectory.6DK79V/vfs.yaml", "-L", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-swift-version", "6", "-I", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-package-description-version", "6.0.0", "-module-cache-path", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "/tmp/Miscellaneous_LibraryEvolutionLinuxXCF.AzjUMO/SwiftFramework/Package.swift", "-o", "/tmp/TemporaryDirectory.lR8kV8/swiftframework-manifest"])
/tmp/TemporaryDirectory.lR8kV8/swiftframework-manifest: error while loading shared libraries: /libPackageDescription.so: cannot open shared object file: No such file or directory
error: 'swiftframework': Invalid manifest (compiled with: ["/usr/local/share/toolchains/6.3.0/usr/bin/swiftc", "-vfsoverlay", "/tmp/TemporaryDirectory.MNO5NI/vfs.yaml", "-L", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-swift-version", "6", "-I", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-package-description-version", "6.0.0", "-module-cache-path", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "/tmp/Miscellaneous_LibraryEvolutionLinuxXCF.AzjUMO/SwiftFramework/Package.swift", "-o", "/tmp/TemporaryDirectory.LS5RlY/swiftframework-manifest"])
/tmp/TemporaryDirectory.LS5RlY/swiftframework-manifest: error while loading shared libraries: /libPackageDescription.so: cannot open shared object file: No such file or directory

✘ Test libraryEvolutionLinuxXCFramework() recorded an issue at LibraryEvolutionXCFLinuxTests.swift:18:6: Caught error: terminated(1): /export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux/swift-build --package-path /tmp/Miscellaneous_LibraryEvolutionLinuxXCF.AzjUMO/SwiftFramework --configuration debug --build-system native --scratch-path /tmp/TemporaryDirectory.csevYp/.build output:
    error: 'swiftframework': Invalid manifest (compiled with: ["/usr/local/share/toolchains/6.3.0/usr/bin/swiftc", "-vfsoverlay", "/tmp/TemporaryDirectory.6DK79V/vfs.yaml", "-L", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-swift-version", "6", "-I", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-package-description-version", "6.0.0", "-module-cache-path", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "/tmp/Miscellaneous_LibraryEvolutionLinuxXCF.AzjUMO/SwiftFramework/Package.swift", "-o", "/tmp/TemporaryDirectory.lR8kV8/swiftframework-manifest"])
    /tmp/TemporaryDirectory.lR8kV8/swiftframework-manifest: error while loading shared libraries: /libPackageDescription.so: cannot open shared object file: No such file or directory
    error: 'swiftframework': Invalid manifest (compiled with: ["/usr/local/share/toolchains/6.3.0/usr/bin/swiftc", "-vfsoverlay", "/tmp/TemporaryDirectory.MNO5NI/vfs.yaml", "-L", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-lPackageDescription", "-Xlinker", "-rpath", "-Xlinker", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-swift-version", "6", "-I", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "-package-description-version", "6.0.0", "-module-cache-path", "/export/home/sarunas/swift-package-manager/.build/out/Products/Debug-linux", "/tmp/Miscellaneous_LibraryEvolutionLinuxXCF.AzjUMO/SwiftFramework/Package.swift", "-o", "/tmp/TemporaryDirectory.LS5RlY/swiftframework-manifest"])
    /tmp/TemporaryDirectory.LS5RlY/swiftframework-manifest: error while loading shared libraries: /libPackageDescription.so: cannot open shared object file: No such file or directory
✘ Test libraryEvolutionLinuxXCFramework() failed after 0.950 seconds with 1 issue.
✘ Suite SwiftPMTests failed after 0.951 seconds with 1 issue.

Can start locally Linux via:

container run -ti  -v ./swift-package-manager:/swift-package-manager  swiftlang/swift:nightly-main-noble bash
apt-get update
apt-get install libsqlite3-dev libncurses6 libncurses5-dev libncursesw5-dev -y

[owenv]

Given this is an experimental option with no stability guarantees and it hasn't made progress towards stabilization, I'm hesitant to support this as XCFrameworks are a Darwin-only format and require the destination platform to have a stable Swift ABI.

[hassila]

We are more than keen on making sure we can support this experimental feature, if you go back to the original discussion this was really intended to be a pressure valve to allow for using library evolution with a stock toolchain on linux too - we really would love to be able to use the new build infrastructure too and not be stuck on the legacy one that is being phased out. We had a number of pitches and discussions going back and the pressure valve of allowing for this was the least intrusive and most pragmatic approach really. How can we help make this happen? Also ping @dschaefer2 - who probably remembers our discussions at the swift server conference, maybe you have some other feedback? Although I think you summarised it good here: swiftlang/swift-package-manager#7239 (comment)

This was originally discussed here too:
swiftlang/swift-package-manager#7239

[vsarunas]

@owenv - changes look to be fairly minimal #1310 , could you please have a look?

[dschaefer2]

Reusing XCFrameworks for shared libraries on Linux is not really the direction I'd like to see this go. I am currently working on a proposal for a replacement for binary artifacts that should allow you to take any binary distribution (i.e. not just artifact bundles), and allow you to specify libraries and executables in them and do all that from the package manifest. Binary artifact bundles have a lot of weird heuristics that are hidden from the manifest that really should be under more control of the developer.

[jakepetroules]

Agree, XCFrameworks were always intended to be specific to Apple platforms.

[hassila]

So given the background in the related case and all previous discussions- how can we avoid the situation that legacy SPM build system is dropped and we have no way to provide library evolution enabled libraries on Linux again?

Or will the legacy build system be supported until there is support for shared libraries with library evolution enabled as binary distribution in some other format?

[dschaefer2]

The Native build system will be around for a while. I'm working on a proposal that should address this and it should land before we remove it.

[hassila]

I'm working on a proposal that should address this and it should land before we remove it.

Ok, looking forward - we'd be super happy to migrate onto fully supported and intended infrastructure of course.

[vsarunas]

Closed via #1310