Skip to content

[build-script] Add the installation prefix to the toolchain path #30565

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 20, 2020

Conversation

finagolfin
Copy link
Member

In several places, it was assumed to be /usr and hard-coded to that. This makes sure the installation prefix actually passed in is used.

I just noticed this when packaging a recent Swift 5.2 snapshot for Android, as I use /data/data/com.termux/files/usr as the install prefix there, not the usual /usr. There are three additional instances where (install_)toolchain_path() is passed without the install prefix, so those repos are likely hard-coding /usr themselves and will need to be fixed.

If someone gives me the go-ahead on this pull, I will submit pulls for those other repos or contact their developers. @aciidb0mb3r, you added the SwiftPM part of this pull, which is what I actually hit, so let me know what you think. @ahoppen, you added some of these.

In several places, it was assumed to be /usr and hard-coded to that. This makes sure
the installation prefix actually passed in is used.
@finagolfin
Copy link
Member Author

After looking into this more, I realized that toolchain_path() would need to be fixed on Darwin too. It looks like darwin_toolchain_prefix() just drops the last path component and /usr is added everywhere else, so I simply centralized that Darwin /usr addition in toolchain_path() instead. Along with swiftlang/indexstore-db#75, swiftlang/swift-stress-tester#123, and swiftlang/swift-syntax#207, this should be fixed now.

@@ -284,5 +284,7 @@ def toolchain_path(install_destdir, install_prefix):
built_toolchain_path = install_destdir
if platform.system() == 'Darwin':
# The prefix is an absolute path, so concatenate without os.path.
built_toolchain_path += darwin_toolchain_prefix(install_prefix)
built_toolchain_path += darwin_toolchain_prefix(install_prefix) + "/usr"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if calling this a "toolchain path" is good terminology, since on Darwin that would mean the .xctoolchain directory. That's also what the documentation string for this method says. After this change, it's always an absolute form of the install_prefix. I'm not sure what to call that 🤔

By the way, darwin_toolchain_prefix removes usr from install_prefix, and then this adds it back.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if calling this a "toolchain path" is good terminology

I have no idea why this is needed on Darwin or what the common paths are there, I just tried to keep the behavior the same.

By the way, darwin_toolchain_prefix removes usr from install_prefix, and then this adds it back.

Yes, as I noted in my previous comment, that is what was being done everywhere already, so rather than adding it back in a dozen different places, I simply added it back once. 😄

I figured it might be because install_prefix on Darwin doesn't necessarily end with /usr, so darwin_toolchain_prefix() just removes that last component and the rest of the code replaces it with /usr. But that's just a guess, given my professed ignorance of the Darwin toolchain.

Just let me know what should be done, as this pull actually starts using install_prefix on non-Darwin platforms, rather than assuming it's always /usr.

@finagolfin
Copy link
Member Author

I missed one, swiftlang/sourcekit-lsp#257 also submitted now.

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 41ef65c

@finagolfin
Copy link
Member Author

The mac/Windows failures seem spurious, linux works.

@benlangmuir
Copy link
Contributor

@finagolfin
Copy link
Member Author

Alright, looks like everything passes on the CI, barring the spurious Windows failure. Feel free to change the docs or let me know what you want changed for the comments.

@shahmishal shahmishal requested a review from edymtt March 24, 2020 22:33
@shahmishal
Copy link
Member

@edymtt Can you please review this?

@shahmishal
Copy link
Member

@swift-ci
Copy link
Contributor

Linux Toolchain (Ubuntu 16.04)
Download Toolchain
Git Sha - 41ef65c

Install command
tar zxf swift-PR-30565-353-ubuntu16.04.tar.gz
More info

@swift-ci
Copy link
Contributor

macOS Toolchain
Download Toolchain
Git Sha - 41ef65c

Install command
tar -zxf swift-PR-30565-493-osx.tar.gz --directory ~/

@finagolfin
Copy link
Member Author

Ping, would be good to get this in before the 5.3 branch, as setting --install-prefix on non-Darwin platforms worked up till this release. Now that SwiftPM shifted to this build-script invocation and requires the toolchain directory be set up before building since last year, SwiftPM won't build anymore if a different installation prefix than /usr is set, whereas it worked before.

@shahmishal
Copy link
Member

cc: @najacque

@finagolfin
Copy link
Member Author

I had to use a portion of this patch to get SwiftPM to build in the package script for Swift 5.2 that just got accepted into the Termux app for Android. @drexin, this pull may also be worth merging into the 5.2 branch, or at least the parts of it that apply.

@finagolfin
Copy link
Member Author

@benlangmuir, you approved some of these simple pulls, mind reviewing the rest?

@shahmishal
Copy link
Member

@swift-ci test windows

@finagolfin
Copy link
Member Author

Passes all CI, ready to go?

@benlangmuir
Copy link
Contributor

@buttaface sorry things are moving slowly here. This hasn't been forgotten, but the folks who have the expertise to review this have been busy. @shahmishal are you still planning to look at this soon?

@shahmishal shahmishal self-requested a review April 20, 2020 03:23
@shahmishal
Copy link
Member

Thanks @buttaface!

@finagolfin
Copy link
Member Author

Thanks for reviewing and merging, one less patch I have to keep locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants