Skip to content

[Build Script Helper] Add support for non-Darwin cross-compilation hosts, starting with Android #743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 7, 2021
Merged

[Build Script Helper] Add support for non-Darwin cross-compilation hosts, starting with Android #743

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions Utilities/build-script-helper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import subprocess
import sys
import errno
import re

if platform.system() == 'Darwin':
shared_lib_ext = '.dylib'
Expand Down Expand Up @@ -96,7 +97,11 @@ def get_swiftpm_options(args):
os.path.join(args.toolchain, 'lib', 'swift', 'Block'),
]

if 'ANDROID_DATA' in os.environ:
if args.cross_compile_hosts:
swiftpm_args += ['--destination', args.cross_compile_config]

if 'ANDROID_DATA' in os.environ or (args.cross_compile_hosts and re.match(
'android-', args.cross_compile_hosts[0])):
swiftpm_args += [
'-Xlinker', '-rpath', '-Xlinker', '$ORIGIN/../lib/swift/android',
# SwiftPM will otherwise try to compile against GNU strerror_r on
Expand Down Expand Up @@ -180,8 +185,13 @@ def handle_invocation(args):
if args.sysroot:
env['SDKROOT'] = args.sysroot

env['SWIFT_EXEC'] = '%sc' % (swift_exec)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only change in this pull that will affect native compilation too, by applying this environment variable when invoking swift-build when installing on non-Darwin platforms. I had to add this when cross-compiling for Android with Swift 5.5 or trunk on a linux host with a Swift 5.4 package installed, as SPM would then try to use that system Swift 5.4 and error because of the Swift stdlib version mismatch.

This fixes that bug in the native build too.


if args.action == 'build':
build_using_cmake(args, toolchain_bin, args.build_path, targets)
if args.cross_compile_hosts and not re.match('-macosx', args.cross_compile_hosts[0]):
swiftpm('build', swift_exec, swiftpm_args, env)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since SPM is used for the install step below anyway, I see no reason to cross-compile with CMake first, which would require passing in a separate list of Swift flags to cross-compile with CMake.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really like to unify these one day, but for now, I agree, this is simpler.

else:
build_using_cmake(args, toolchain_bin, args.build_path, targets)

elif args.action == 'clean':
print('Cleaning ' + args.build_path)
Expand All @@ -191,7 +201,6 @@ def handle_invocation(args):
tool_path = os.path.join(toolchain_bin, tool)
if os.path.exists(tool_path):
env['SWIFT_DRIVER_' + tool.upper().replace('-','_') + '_EXEC'] = '%s' % (tool_path)
env['SWIFT_EXEC'] = '%sc' % (swift_exec)
test_args = swiftpm_args
test_args += ['-Xswiftc', '-enable-testing']
if should_test_parallel():
Expand All @@ -208,16 +217,17 @@ def handle_invocation(args):
else:
bin_path = swiftpm_bin_path(swift_exec, swiftpm_args, env)
swiftpm('build', swift_exec, swiftpm_args, env)
non_darwin_install(bin_path, args.toolchain, args.verbose)
non_darwin_install(args, bin_path)
else:
assert False, 'unknown action \'{}\''.format(args.action)

# Installation flow for non-darwin platforms, only copies over swift-driver and swift-help
# TODO: Unify CMake-based installation flow used on Darwin with this
def non_darwin_install(swiftpm_bin_path, toolchain, verbose):
toolchain_bin = os.path.join(toolchain, 'bin')
for exe in executables_to_install:
install_binary(exe, swiftpm_bin_path, toolchain_bin, verbose)
def non_darwin_install(args, swiftpm_bin_path):
for prefix in args.install_prefixes:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should keep working as it always has when a --prefix list isn't passed in, since this list is default-initialized to the toolchain path anyway and build-script never passed --prefix in.

prefix_bin = os.path.join(prefix, 'bin')
for exe in executables_to_install:
install_binary(exe, swiftpm_bin_path, prefix_bin, args.verbose)

def install(args, build_dir, targets):
# Construct and install universal swift-driver, swift-help executables
Expand Down Expand Up @@ -584,6 +594,10 @@ def add_common_args(parser):
nargs='*',
help='List of cross compile hosts targets.',
default=[])
parser.add_argument(
'--cross-compile-config',
metavar='PATH',
help="A JSON SPM config file with Swift flags for cross-compilation")
parser.add_argument('--ninja-bin', metavar='PATH', help='ninja binary to use for testing')
parser.add_argument('--cmake-bin', metavar='PATH', help='cmake binary to use for building')
parser.add_argument('--build-path', metavar='PATH', default='.build', help='build in the given path')
Expand Down Expand Up @@ -626,6 +640,8 @@ def add_common_args(parser):
args.cross_compile_hosts = [build_target + macos_deployment_target, 'arm64-apple-macosx%s' % macos_deployment_target]
elif (build_target == 'arm64-apple-macosx' and 'macosx-x86_64' in args.cross_compile_hosts):
args.cross_compile_hosts = [build_target + macos_deployment_target, 'x86_64-apple-macosx%s' % macos_deployment_target]
elif args.cross_compile_hosts and re.match('android-', args.cross_compile_hosts[0]):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gates this new cross-compilation path for Android only, since it has only been tested for that non-Darwin platform.

Copy link
Member Author

@finagolfin finagolfin Jul 5, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, it only checks the first element of the hosts with args.cross_compile_hosts[0], as swiftlang/swift#36917 only passes in one at a time. I have proposed changing that flag to a single --cross-compile-host instead.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that you're changing the format of the strings in cross_compile_hosts for Darwin in the lines above, whereas I'm keeping the passed-in format for non-Darwin. Since the triple format you use on Darwin is only needed for the CMake build and Darwin install(), both of which are unused with non-Darwin hosts in this pull, that works fine for now with these non-Darwin hosts.

print('Cross-compiling for %s' % args.cross_compile_hosts[0])
elif args.cross_compile_hosts:
error("cannot cross-compile for %s" % cross_compile_hosts)

Expand Down