rustc: Handle modules in "fat" LTO more robustly - #63956
Merged
Merged
Conversation
Contributor
|
r? @eddyb (rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
This comment has been minimized.
This comment has been minimized.
When performing a "fat" LTO the compiler has a whole mess of codegen units that it links together. To do this it needs to select one module as a "base" module and then link everything else into this module. Previously LTO passes assume that there's at least one module in-memory to link into, but nowadays that's not always true! With incremental compilation modules may actually largely be cached and it may be possible that there's no in-memory modules to work with. This commit updates the logic of the LTO backend to handle modules a bit more uniformly during a fat LTO. This commit immediately splits them into two lists, one serialized and one in-memory. The in-memory list is then searched for the largest module and failing that we simply deserialize the first serialized module and link into that. This refactoring avoids juggling three lists, two of which are serialized modules and one of which is half serialized and half in-memory. Closes rust-lang#63349
alexcrichton
force-pushed
the
fix-lto-all-cached
branch
from
August 27, 2019 20:51
464657b to
1a4330d
Compare
Member
|
Thanks, @alexcrichton! I suspected that it's a relatively simple problem like that. Great that you came up with a small test case! @bors r+ |
Collaborator
|
📌 Commit 1a4330d has been approved by |
bors
added
S-waiting-on-bors
RalfJung
reviewed
Aug 29, 2019
| // Sort out all our lists of incoming modules into two lists. | ||
| // | ||
| // * `serialized_modules` (also and argument to this function) contains all | ||
| // modules that are serialized in-memory. |
Member
There was a problem hiding this comment.
So serialized modules are also in-memory? That makes serialized_modules vs in_memory not mutually exclusive, so maybe serialized_modules vs parsed_modules or so would make more sense?
As someone not at all familiar with this code, this is rather confusing.
Centril
added a commit
to Centril/rust
that referenced
this pull request
Aug 29, 2019
…michaelwoerister rustc: Handle modules in "fat" LTO more robustly When performing a "fat" LTO the compiler has a whole mess of codegen units that it links together. To do this it needs to select one module as a "base" module and then link everything else into this module. Previously LTO passes assume that there's at least one module in-memory to link into, but nowadays that's not always true! With incremental compilation modules may actually largely be cached and it may be possible that there's no in-memory modules to work with. This commit updates the logic of the LTO backend to handle modules a bit more uniformly during a fat LTO. This commit immediately splits them into two lists, one serialized and one in-memory. The in-memory list is then searched for the largest module and failing that we simply deserialize the first serialized module and link into that. This refactoring avoids juggling three lists, two of which are serialized modules and one of which is half serialized and half in-memory. Closes rust-lang#63349
Collaborator
Member
|
@bors r- retry |
bors
added
S-waiting-on-author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When performing a "fat" LTO the compiler has a whole mess of codegen
units that it links together. To do this it needs to select one module
as a "base" module and then link everything else into this module.
Previously LTO passes assume that there's at least one module in-memory
to link into, but nowadays that's not always true! With incremental
compilation modules may actually largely be cached and it may be
possible that there's no in-memory modules to work with.
This commit updates the logic of the LTO backend to handle modules a bit
more uniformly during a fat LTO. This commit immediately splits them
into two lists, one serialized and one in-memory. The in-memory list is
then searched for the largest module and failing that we simply
deserialize the first serialized module and link into that. This
refactoring avoids juggling three lists, two of which are serialized
modules and one of which is half serialized and half in-memory.
Closes #63349