Enforce in bootstrap that build must have stage at least 1 #142581

Kobzol · 2025-06-16T15:38:35Z

This PR is a step towards https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Proposal.20to.20cleanup.20stages.20and.20steps.20after.20the.20redesign/with/523586917. It's very hard or me to make self-contained changes to bootstrap at this moment, so this PR kind of does several things:

(first two commits) Try to reduce the usage of Std::new in bootstrap, and replace it with a Builder::std method (similar to Builder::compiler). This is mostly to remove executions of the Step step for stage 0, which doesn't make a lot of sense; I'd like to ideally have the invariant that when a step is invoked, it actually builds or does something. Eventually, I'd like for everything to go through Builder::std. (Note: I'm not totally married to this idea, if you don't like it, we can remove it from this PR. I mostly did it right now to remove stage 0 std steps from snapshot tests, which shouldn't be there, but we can also filter them out in a different way)
Make sure that when you pass x build compiler, only the Assemble root level step will be invoked, and not the Rustc step. Before, both were invoked, which actually ran Rustc twice, once with all crates filled, and once with no crates (but both actually represent the same situation). Since the Rustc::make_run step actually requests a compile that is one stage below it, this actually made build compiler --stage 0 work, which we don't want to have anymore.
Enforce a bootstrap-global invariant that all build commands are always on stage >=1. If you try to build anything on stage 0, it will print a warning and exit bootstrap. This follows the intuition from the new staging rules after the stage redesign; artifacts that are "stage 0" come outside of bootstrap, and we can't really build something for which we don't have source (although we can still test it, but that's for another PR).

Now the logic for build should be quite simple. For pretty much everything except for Std, you first use the stage0 compiler to build stage 1. Then you can build a stage 2 using the previously built stage 1 (and then you can continue to stage 3 etc.). And that's it. The default build stage for everything is 1 (modulo download-ci-rustc, but that's a separate can of worms).

The snapshot test infra isn't super nice at the moment, as one of next steps I want to create some simple Builder pattern that will allow us to configure the bootstrap invocations in a more "forward-compatible" way (e.g. now it's not possible to modify the config passed to configure_with_args).

There are some things not yet fully resolved for build stage 0:

Cargo is still a ModeRustc tool, even though it doesn't really have to be, it is buildable with the stage0 compiler
bootstrap tools (opt-dist, build-manifest etc.) are still called stage0 tools, and in the bootstrap output it says something like "stage 0 rustc builds stage 0 opt-dist". Which is a bit weird, but functionally there's no difference, it's just a slightly inconsistent output. We still haven't decided if we should make these tools ignore staging altogether (which is IMO the right choice) or if we want to allow building stage 1/2/3/... bootstrap tools.

r? @jieyouxu

rustbot · 2025-06-16T15:38:39Z

jieyouxu is not on the review rotation at the moment.
They may take a while to respond.

Kobzol · 2025-06-16T15:59:12Z

This actually interacts with cross-compilation, as I mentioned here:

./x build --host 'i686-unknown-linux-gnu'
thread 'main' panicked at src/bootstrap/src/core/builder/mod.rs:1344:17:
It is not possible to build the standard library for `i686-unknown-linux-gnu` using the stage0 compiler.
You have to build a stage1 compiler for `x86_64-unknown-linux-gnu` first, and then use it to build a standard library for `i686-unknown-linux-gnu`.

You need to pass --stage 2 now to make this work. Yet another trade-off between "bootstrap keeping explicit stages for commands" vs "bootstrap should just make stuff work somehow"...

rustbot · 2025-06-17T07:12:20Z

This PR modifies src/bootstrap/defaults.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

…ompiler` path It was confusing that the `Rustc` step was invoked twice, once with all crates and once with no crates (even though both of these mean the same thing).

rust-log-analyzer · 2025-06-17T10:53:19Z

The job aarch64-gnu-llvm-19-2 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- core::builder::tests::alias_and_path_for_library stdout ----

thread 'core::builder::tests::alias_and_path_for_library' panicked at src/bootstrap/src/core/builder/tests.rs:227:5:
assertion `left == right` failed
  left: [Std { target: i686-unknown-haiku, compiler: Compiler { stage: 1, host: i686-unknown-haiku, forced_compiler: false }, crates: [], force_recompile: false, extra_rust_args: [], is_for_mir_opt_tests: false }]
 right: [Std { target: i686-unknown-haiku, compiler: Compiler { stage: 0, host: i686-unknown-haiku, forced_compiler: false }, crates: [], force_recompile: false, extra_rust_args: [], is_for_mir_opt_tests: false }, Std { target: i686-unknown-haiku, compiler: Compiler { stage: 1, host: i686-unknown-haiku, forced_compiler: false }, crates: [], force_recompile: false, extra_rust_args: [], is_for_mir_opt_tests: false }]
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- core::builder::tests::defaults::build_cross_compile stdout ----

thread 'core::builder::tests::defaults::build_cross_compile' panicked at src/bootstrap/src/core/builder/mod.rs:1348:17:
It is not possible to build the standard library for `i686-unknown-hurd-gnu` using the stage0 compiler.
You have to build a stage1 compiler for `i686-unknown-haiku` first, and then use it to build a standard library for `i686-unknown-hurd-gnu`.


---- core::builder::tests::defaults::build_default stdout ----

thread 'core::builder::tests::defaults::build_default' panicked at src/bootstrap/src/core/builder/tests.rs:321:9:
---



---- core::builder::tests::dist::build_all stdout ----
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-gcc": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-gcc": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)

thread 'core::builder::tests::dist::build_all' panicked at src/bootstrap/src/core/builder/tests.rs:736:9:
assertion failed: `(left == right)`

Diff < left / right > :
---



---- core::builder::tests::dist::build_with_empty_host stdout ----
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-gcc": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-gcc": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)
cargo:warning=Compiler family detection failed due to error: ToolNotFound: failed to find tool "i486--netbsdelf-g++": No such file or directory (os error 2)

thread 'core::builder::tests::dist::build_with_empty_host' panicked at src/bootstrap/src/core/builder/tests.rs:793:9:
assertion failed: `(left == right)`

Diff < left / right > :

jieyouxu · 2025-06-18T10:14:27Z

src/bootstrap/src/core/config/config.rs

+        // Now check that the selected stage makes sense, and if not, print a warning and end
+        match (config.stage, &config.cmd) {
+            (0, Subcommand::Build) => {
+                eprintln!("WARNING: cannot build anything on stage 0. Use at least stage 1.");
+                exit!(1)
+            }
+            _ => {}
+        }


Question: if we warn on the no-op that is ./x build --stage=0, what about ./x check --stage=0? I guess for the check case, we can't do that yet without untying bootstrap tool checking from staging?

Also, does this reject e.g. ./x build src/tools/opt-dist --stage=0?

EDIT: indeed,

#[test] #[should_panic] fn build_bootstrap_tool_stage_0() { insta::assert_snapshot!(get_steps("opt-dist", "build", Some(0)), @""); }

Check is a whole another beast, I'm working on a PR for it, but the individual bootstrap kinds (build/check/etc.) are mostly separate in the stage handling, so I will do it piece by piece.

Btw, I'm planning for check --stage 0 to also be an error.

jieyouxu · 2025-06-18T10:22:14Z

src/bootstrap/src/core/build_steps/compile.rs

+        // If only `compiler` was passed, do not run this step.
+        // Instead the `Assemble` step will take care of compiling Rustc.
+        if run.builder.paths == vec![PathBuf::from("compiler")] {
+            return;
+        }


Remark: this kinda feels iffy, in that I feel like the path filtering/handling logic that lead to

Rustc step was invoked twice, once with all crates and once with no crates (even though both of these mean the same thing).

makes this kinda weird.

I would love Rustc and Assemble to be just one step (to avoid things like this), but I guess that it's sort of an optimizatoin, Rustc allows building only selected crates, and doesn't involve building a ton of other things that you might not want for a quick build.

Yeah, that does make sense.

jieyouxu · 2025-06-18T10:24:32Z

I'm generally in favor of imposing more discipline (or more order, for that matter) on build staging as is done in this PR. Not super sure about rejecting ./x build <bootstrap_tool> --stage=0. Would like a second opinion.

Maybe r? @Mark-Simulacrum

Kobzol · 2025-06-18T11:23:25Z

Not super sure about rejecting ./x build <bootstrap_tool> --stage=0. Would like a second opinion.

That depends on how we decide #t-infra/bootstrap > Proposal to cleanup stages and steps after the redesign. The bootstrap tools don't really participate in staging (unless you're cross-compiling, but we don't even support cross-compiling these tools, as they are host tools, essentially). So in theory we could allow them to be built with whatever --stage you pass. However, to achieve better consistency in bootstrap staging, they should in fact be treated stage 1 tools, as they are built with a stage 0 compiler. If we force their stage to be 1, I don't think that the error on --stage 0 should anger anyone, the most natural thing (x build <host-tool>) should just work. No need to pass --stage 0.

bors · 2025-06-18T22:19:28Z

☔ The latest upstream changes (presumably #142685) made this pull request unmergeable. Please resolve the merge conflicts.

rustbot assigned jieyouxu Jun 16, 2025

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jun 16, 2025

This comment has been minimized.

Sign in to view

Kobzol marked this pull request as ready for review June 17, 2025 07:12

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 17, 2025

Kobzol added 8 commits June 17, 2025 09:15

Add Builder::std method for building a standard library

4a2ca11

Use Builder::std where possible

b9f1cbe

Add snapshot tests for compiler builds

2e423d2

Do not explicitly build Rustc when bootstrap is invoked with the `c…

b48bcb2

…ompiler` path It was confusing that the `Rustc` step was invoked twice, once with all crates and once with no crates (even though both of these mean the same thing).

Do not allow building anything on stage 0

7c9cb2b

Add build staging tests for library, miri and a bootstrap tool

8f426d7

Remove build --stage 0 from CI scripts

84dcb78

Fix Clippy

6bf0ab7

Kobzol force-pushed the bootstrap-std-method branch from 49ed3f9 to 6bf0ab7 Compare June 17, 2025 07:15

This comment has been minimized.

Sign in to view

Fix bootstrap tracing mode

f8a105c

Kobzol marked this pull request as draft June 17, 2025 14:26

jieyouxu reviewed Jun 18, 2025

View reviewed changes

rustbot assigned Mark-Simulacrum and unassigned jieyouxu Jun 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enforce in bootstrap that build must have stage at least 1 #142581

Enforce in bootstrap that build must have stage at least 1 #142581

Uh oh!

Kobzol commented Jun 16, 2025 •

edited

Loading

Uh oh!

rustbot commented Jun 16, 2025

Uh oh!

Kobzol commented Jun 16, 2025

Uh oh!

This comment has been minimized.

rustbot commented Jun 17, 2025

Uh oh!

This comment has been minimized.

rust-log-analyzer commented Jun 17, 2025

Uh oh!

jieyouxu Jun 18, 2025

Uh oh!

Kobzol Jun 18, 2025

Uh oh!

Kobzol Jun 18, 2025

Uh oh!

jieyouxu Jun 18, 2025

Uh oh!

Kobzol Jun 18, 2025

Uh oh!

jieyouxu Jun 18, 2025

Uh oh!

jieyouxu commented Jun 18, 2025 •

edited

Loading

Uh oh!

Kobzol commented Jun 18, 2025

Uh oh!

bors commented Jun 18, 2025

Uh oh!

Uh oh!

Enforce in bootstrap that build must have stage at least 1 #142581

Are you sure you want to change the base?

Enforce in bootstrap that build must have stage at least 1 #142581

Uh oh!

Conversation

Kobzol commented Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jun 16, 2025

Uh oh!

Kobzol commented Jun 16, 2025

Uh oh!

This comment has been minimized.

rustbot commented Jun 17, 2025

Uh oh!

This comment has been minimized.

rust-log-analyzer commented Jun 17, 2025

Uh oh!

jieyouxu Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

Kobzol Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kobzol commented Jun 18, 2025

Uh oh!

bors commented Jun 18, 2025

Uh oh!

Uh oh!

Kobzol commented Jun 16, 2025 •

edited

Loading

jieyouxu commented Jun 18, 2025 •

edited

Loading