./x.py build -i does not fill stage2 directory

[RalfJung]

Just like ./x.py build ends by filling the stage 2 directory...

Copying stage1 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Copying stage2 compiler (x86_64-unknown-linux-gnu)
Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)

... I would expect ./x.py build -i to do the same. But it does not, the output ends with

Copying stage1 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Build completed successfully in 0:00:03

So to get a usable stage 2 while using incremental compilation, I need to ./x.py build -i && ./x.py build. That seems wrong.

[Mark-Simulacrum]

./x.py build -i defaults the stage to 1 (

rust/src/bootstrap/flags.rs

Lines 309 to 311 in 5cc1baa

	if matches.opt_present("incremental") && stage.is_none() {
	stage = Some(1);
	}

) so we don't build a stage2 directory. However, you can pass --stage 2 to override that. Closing as not a bug.

[RalfJung]

That is very confusing. Why should the -i flag affect the stage at all? There is no reason for the user to expect this, nor is it documented in --help.

I still think there is a bug here.

[Mark-Simulacrum]

Hm, perhaps so. I'll reopen.

The thought here I believe is that if you want an incremental build, you probably want to only build to stage 1, since only stage 0 is incrementally built. However, since I believe @nikomatsakis added the logic in 83453bc, I'll let them try to explain why. cc @alexcrichton as well.

[RalfJung]

The thought here I believe is that if you want an incremental build, you probably want to only build to stage 1, since only stage 0 is incrementally built.

Notice that x.py build --stage 1 does build everything. The only thing it does not do is copy the output of the stage 1 compilation into the stage 2 folder.

[Mark-Simulacrum]

Note that the fact that ./x.py build --stage 1 basically builds everything is only a side-effect of having full-bootstrap turned off, and hence us promoting stage1's target libdir into stage2's sysroot. It's somewhat true that as it is today, ./x.py build --stage 1 is mostly not what you want.

[RalfJung]

Ah, so probably behavior was different when the -i flag was made to behave the way it does.

[Mark-Simulacrum]

So the one thing here is that as it is today, most of the time you don't need to build the stage1 compiler artifacts, so -i's behavior of setting the stage to 1 is nice, since ./x.py test -i src/test/run-pass just does what you (probably) want: builds libstd and libtest in stage 1, and runs run pass tests using that.

[RalfJung]

most of the time you don't need to build the stage1 compiler artifacts

Wouldn't that be an argument to generally make --stage 1 the default? Wiring this to -i seems strange to me.

[Mark-Simulacrum]

Perhaps, yes. I don't really know why stage 2 is the default.. maybe @alexcrichton can comment there? It seems to be the wrong default...

[nikomatsakis]

As @Mark-Simulacrum noted, the reason that --incremental implies --stage 1 was that it is what you almost certainly want. That is, we are not able to build stage 2 (from stage 1) incrementally, since the stage 1 compiler will have changed since the last build.

[RalfJung]

@nikomatsakis As I already said, however, if you run ./x.py build -i, the entire stage 2 does get built in the stage 1 output directories. The only thing that does not happen is copying it to the stage 2 directories. The default stage should be 0 if you want to make sure that everything that gets built, gets built incrementally.

[ghost]

As I already said, however, if you run ./x.py build -i, the entire stage 2 does get built in the stage 1 output directories. The only thing that does not happen is copying it to the stage 2 directories.

the entire stage 2 does get built in the stage 1 output directories

^ That's interesting... I hope that does not happen.
I'm just gonna do this to be sure ./x.py build --incremental --stage 1 --keep-stage 0 -j 4

EDIT: well, by the time it takes to build stage2 from the above(3mins), Ralfjung is probably right:

./x.py build -j 5
Updating submodules
    Finished dev [unoptimized] target(s) in 0.0 secs
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage0 std from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage0 test from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage0 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.1 secs
Copying stage0 rustc from stage0 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Assembling stage1 compiler (x86_64-unknown-linux-gnu)
Building stage1 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage1 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage1 test from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 compiler artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Finished release [optimized] target(s) in 0.1 secs
Copying stage1 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Assembling stage2 compiler (x86_64-unknown-linux-gnu)
Uplifting stage1 std (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage2 std from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Uplifting stage1 test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage2 test from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
Building stage1 tool rustdoc (x86_64-unknown-linux-gnu)
   Compiling pulldown-cmark v0.0.14
   Compiling log v0.3.8
   Compiling gcc v0.3.51
   Compiling bitflags v0.8.2
   Compiling libc v0.2.29
   Compiling env_logger v0.4.3
   Compiling filetime v0.1.10
   Compiling build_helper v0.1.0 (file:///home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/src/build_helper)
   Compiling rustdoc v0.0.0 (file:///home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/src/librustdoc)
   Compiling rustdoc-tool v0.0.0 (file:///home/xftroxgpx/build/2nonpkgs/rust.stuff/rust/rust/src/tools/rustdoc)
    Finished release [optimized] target(s) in 161.27 secs
Build completed successfully in 0:03:07

real	3m7.558s
user	4m50.740s
sys	0m9.694s

[RalfJung]

That's interesting... I hope that does not happen.

What's the problem with this?

[Mark-Simulacrum]

The block that sets stage 1 when incremental is passed can be removed to fix this. Since stage 2 isn't going to take much time in practice due to copying libs forward, this shouldn't be a problem.