Skip to content

[Driver] Work around lld 13+ issue with --gc-sections for ELF by adding -z nostart-stop-gc #60544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 19, 2022

Conversation

finagolfin
Copy link
Member

Fixes #60406

Ran the linux x86_64 compiler validation suite on this pull with -use-ld=lld -Xlinker --gc-sections, then without stripping as in the linked issue, and only saw the 20 unrelated failures just like @drodriguez had.

I would have added this test to Driver/linker.swift, but I see those tests have been disabled for more than two years now, #32818. @nate-chandler, seems like those should be re-enabled by now?

@finagolfin
Copy link
Member Author

@3405691582, would you try this fix on OpenBSD?

@3405691582
Copy link
Member

This doesn't seem to solve the issue with lib_InternalSwiftSyntaxParser.so for me.

@finagolfin
Copy link
Member Author

This doesn't seem to solve the issue with lib_InternalSwiftSyntaxParser.so for me.

Hmm, do you build with the early swift driver? That reminds me, I'll need to patch that too.

@3405691582
Copy link
Member

Hmm, do you build with the early swift driver?

No, I haven't built with that in a little while (I probably should remedy that, though).

@finagolfin
Copy link
Member Author

finagolfin commented Aug 14, 2022

No, I haven't built with that in a little while (I probably should remedy that, though).

Oh, I just realized, that failing command, does it add -use-ld=lld or is it just assumed that lld is the system linker? If the latter, the Swift compiler will not know that it's using lld. If you paste the failing command and the output it produces with -v appended, we can figure it out.

@finagolfin
Copy link
Member Author

Oh, duh, got it: I just checked and the compiler build doesn't link this library with the Swift compiler, it uses a clang++ command for the final link step. As such, this pull won't help with that library, which was never my aim anyway, and you will need to keep using whatever workaround you had for that library before or pass this lld flag for that library separately.

@finagolfin
Copy link
Member Author

@compnerd, would you review?

Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙄 lld just really enjoys breaking the guaranteed unix semantics. This seems like a good idea to add. Thanks!

@compnerd
Copy link
Member

@swift-ci please test

@compnerd
Copy link
Member

Hmm, did we already do this in the Swift driver? If not, we should ensure that we get it in there as well.

@finagolfin
Copy link
Member Author

I submitted it there too, swiftlang/swift-driver#1153, feel free to review that one too.

@keith
Copy link
Member

keith commented Aug 15, 2022

Does this cover the case where you don't pick lld but it's the inferred linker? Isn't that the case on android with new NDKs?

I still feel like we need to get to the bottom of the section preservation 😞

@finagolfin
Copy link
Member Author

Does this cover the case where you don't pick lld but it's the inferred linker?

Depends what you mean, if you mean inferred by the Swift compiler as the default linker for that platform, then yes. However, as I noted on the issue I opened, I don't know how to handle the scenario where someone changes the system ld to link to lld instead, as opposed to the old binutils ld it usually points at on linux, so I don't bother with that.

Isn't that the case on android with new NDKs?

It is inferred by the Swift compiler, so that should work with this pull.

I still feel like we need to get to the bottom of the section preservation

I'm all for someone looking into that and us trying other approaches, this is just the only one that works so far, as demonstrated in the test run data I posted in #60406. I asked for feedback on alternatives there, and didn't get any.

@finagolfin
Copy link
Member Author

finagolfin commented Aug 16, 2022

@keith, please merge if you are okay with this approach for now, so we have a fix in the compiler for this lld issue with stripping ELF binaries. You can always investigate #60357 more later if you like- I don't care enough about this problem to dig deeper on those other possible fixes- and swap this out for an updated version of that pull later on, that hopefully fixes those additional tests that fail with that SHF_GNU_RETAIN approach.

In the meantime, I'd like to get this merged in trunk and 5.7 and re-enable your SPM pulls for dead-stripping, hopefully before 5.7 releases. We can always change this fix later if you find a better way.

@finagolfin
Copy link
Member Author

@keith, what do you think? If it's too late to get this into 5.7 before it releases, I'm in no hurry and can wait on this.

@finagolfin
Copy link
Member Author

@artemcm, mind merging this and the swift-driver version, so we have a fix for this lld issue for now, and whenever @keith gets back to us with a working alternative, we can rework as necessary? In the meantime, I'd like to get this into 5.7 and re-enable his ELF dead-stripping pulls for SPM.

@keith keith merged commit cc0e122 into swiftlang:main Aug 19, 2022
@keith
Copy link
Member

keith commented Aug 19, 2022

I don't know if it's too late for 5.7 or not, I'd say open the PR and hopefully the release manager can take a look

@keith
Copy link
Member

keith commented Aug 19, 2022

Seems like a good fix until we can track down the core issue

finagolfin added a commit to finagolfin/swift-package-manager that referenced this pull request Aug 19, 2022
It should work with lld now that the lld flag '-z nostart-stop-gc' was added with
swiftlang/swift#60544 and swiftlang/swift-driver#1153.
finagolfin added a commit to finagolfin/swift that referenced this pull request Aug 19, 2022
…ng -z nostart-stop-gc (swiftlang#60544)

Fixes swiftlang#60406 

Ran the linux x86_64 compiler validation suite on this pull with `-use-ld=lld -Xlinker --gc-sections`, then without stripping as in the linked issue, and only saw the 20 unrelated failures just like @drodriguez had.
@finagolfin finagolfin deleted the lld-gc branch August 19, 2022 20:33
neonichu pushed a commit to swiftlang/swift-package-manager that referenced this pull request Aug 19, 2022
It should work with lld now that the lld flag '-z nostart-stop-gc' was added with
swiftlang/swift#60544 and swiftlang/swift-driver#1153.
finagolfin added a commit to finagolfin/swift that referenced this pull request Aug 24, 2022
meg-gupta pushed a commit to meg-gupta/swift that referenced this pull request Aug 27, 2022
…ng -z nostart-stop-gc (swiftlang#60544)

Fixes swiftlang#60406 

Ran the linux x86_64 compiler validation suite on this pull with `-use-ld=lld -Xlinker --gc-sections`, then without stripping as in the linked issue, and only saw the 20 unrelated failures just like @drodriguez had.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lld 13+ stopped working with the linker flag --gc-sections and Swift on ELF platforms
4 participants