This repository was archived by the owner on Jul 1, 2023. It is now read-only.
build: enable building X10 support with SPM #1176
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dan-zheng
approved these changes
Jan 5, 2021
| @@ -50,6 +50,10 @@ let package = Package( | |||
| "PythonKit", | |||
| "CTensorFlow", | |||
| .product(name: "Numerics", package: "swift-numerics"), | |||
| ], | |||
| swiftSettings: [ | |||
| .define("USING_X10_BACKEND"), | |||
There was a problem hiding this comment.
From offline discussion: it seems like we can remove the USING_X10_BACKEND flag and checks soon, since that is the only path forward for enabling SwiftPM builds for tensorflow/swift-apis.
| ], | ||
| swiftSettings: [ | ||
| .define("USING_X10_BACKEND"), | ||
| .define("DEFAULT_BACKEND_EAGER"), |
There was a problem hiding this comment.
I wonder if it's possible to eventually enable non-DEFAULT_BACKEND_EAGER builds via SwiftPM?
There was a problem hiding this comment.
It is possible, but requires an API break; I'd want such a switch to be planned (and communicated) prior to executing.
Enable building the Swift interfaces for X10 with SPM. This effectively hard-enables X10 support in TensorFlow as SPM does not provide configuration. In order to accomplish this, we need to create symbolic links to sources in order to merge the `x10` sources into the TensorFlow module. It is not possible to simply move the sources to reflect this structure due to out-of-tree use cases.
The CX10 module setup is currently very complicated. The module is built twice - once via bazel and incorporated into the TensorFlow library. The second build is built via Swift for the modules to import the interfaces into Swift. In order to build the swift bindings via SPM, we need to extract the sources. However, we cannot reorganise the CX10 subdirectory to enable SPM to filter out just the modules. As a result, we create a `CX10Modules` module that is only built when building with SPM. CMake builds will ignore this directory, building the modules from the CX10 directory directly as it can handle the subdirectory layout.
|
Intentionally merging commits separately for separation of concerns. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable building the Swift interfaces for X10 with SPM. This effectively
hard-enables X10 support in TensorFlow as SPM does not provide
configuration.
In order to accomplish this, we need to create symbolic links to sources
in order to merge the
x10sources into the TensorFlow module. It isnot possible to simply move the sources to reflect this structure due to
out-of-tree use cases.