1.10.0 fails to build with bootstrap compiler #34722

the-kenny · 2016-07-08T11:19:42Z

While trying to package rust-1.10.0 for NixOS, I ran into the following issue:

We're using a bootstrap compiler from https://static.rust-lang.org/dist/rust-1.10.0-$platforn.tar.gz to build rustc with the following configure flags (among others):

--enable-local-rust
--local-rust-root=...

While the build used to work just fine for 1.9.0, we get errors building libcore for 1.10.0:

src/libcore/lib.rs:63:1: 63:37 error: #[feature] may not be used on the stable release channel
src/libcore/lib.rs:63 #![feature(allow_internal_unstable)]
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/libcore/lib.rs:64:1: 64:17 error: #[feature] may not be used on the stable release channel
src/libcore/lib.rs:64 #![feature(asm)]
                      ^~~~~~~~~~~~~~~~
src/libcore/lib.rs:65:1: 65:38 error: #[feature] may not be used on the stable release channel
src/libcore/lib.rs:65 #![feature(associated_type_defaults)]
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/libcore/lib.rs:66:1: 66:32 error: #[feature] may not be used on the stable release channel
src/libcore/lib.rs:66 #![feature(cfg_target_feature)]
                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/libcore/lib.rs:67:1: 67:27 error: #[feature] may not be used on the stable release channel
src/libcore/lib.rs:67 #![feature(concat_idents)]
...

Full output (don't worry about the strange-looking absolute paths, they're normal on NixOS):

https://gist.github.com/the-kenny/1424cd6a03467326dc1b6ad2e6480272

The text was updated successfully, but these errors were encountered:

nagisa · 2016-07-08T13:41:34Z

You must use corresponding beta (i.e. 1.11 to build 1.10) when bootstraping rustc. You’re trying to bootstrap 1.10 with 1.10.

nagisa · 2016-07-08T13:42:31Z

cc @alexcrichton 2nd cycle we get exactly the same error reported. Maybe some sort of diagnostic is in order?

the-kenny · 2016-07-08T14:00:15Z

This is quite inconvenient. As far as I see, there isn't any beta of 1.11 available on static.rust-lang.net yet. Without starting a big discussion - why isn't it possible to bootstrap rustc with the previous stable version?

alexcrichton · 2016-07-08T15:58:14Z

Hm I'm a bit confused here, as @nagisa mentioned you can only bootstrap the 1.10 compiler with the 1.9 compiler, no other compiler in the world. @the-kenny are you trying to bootstrap 1.10 with 1.10? How is 1.11 coming into play?

the-kenny · 2016-07-08T16:32:45Z

@alexcrichton Yes, apparently I was trying to bootstrap 1.10 with 1.10. Did I misunderstand @nagisa? My understanding of his comment was that I have to use 1.11 as a bootstrap compiler for 1.10.

Using 1.9 from static.rust-lang.org as a bootstrap compiler for 1.10 doesn't work for me either. It's still complaining about usage of unstable library features (this time it's extended_compare_and_swap).

alexcrichton · 2016-07-08T17:26:45Z

@the-kenny ah yes you can neither bootstrap 1.10 with 1.10 nor 1.11, the only way it can work is with the 1.9 compiler.

Can you gist a full build log where you're using the 1.9 compiler to bootstrap? Also, where did the 1.9 compiler come from? If you're also building the 1.9 compiler manually then the bootstrap will only work if the 1.9 compiler is configured with --release-channel=stable

the-kenny · 2016-07-08T21:12:05Z

@alexcrichton Thanks! Full build-log is at https://gist.github.com/the-kenny/4b6dc5991771a82873ab29bbd8179692

I'm using rustc-1.9.0 from https://static.rust-lang.org/dist/rust-1.9.0-x86_64-unknown-linux-gnu.tar.gz (sha256: 288ff13efa2577e81c77fc2cb6e2b49b1ed0ceab51b4fa12f7efb87039ac49b7). Nix does some minimal after-work on the binary, all shown in https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/rust/bootstrap.nix#L46-L53 (don't worry about the wrapProgram, it's actually a no-op there)

MagaTailor · 2016-07-08T22:46:38Z

#33980

alexcrichton · 2016-07-09T01:24:27Z

@the-kenny hm that's certainly an unusual build log! You're clearly able to use unstable features because libcore succesfully compiled, but then apparently liballoc got linked to something like a stale copy? It's not clear to me what exactly could trigger that error.

There's some suspicious errors at the top though which may be relevant? Could you try it again but with VERBOSE=1 so all the commands are printed out?

the-kenny · 2016-07-09T07:33:05Z

Verbose build log: https://gist.github.com/the-kenny/6707ad42d1c4fe70a805198795edc905

alexcrichton · 2016-07-10T06:34:42Z

Hm unfortunately that didn't help me at much, I'm not actually sure what's going on here. It'd probably be best to play around with various RUST_LOG annotations to see exactly which crates are being loaded, and this may also be a situation where local_stage0.sh is totally broken (I don't think it's exercised much).

Is there a way I could easily reproduce this locally?

the-kenny · 2016-07-10T10:22:13Z

It should be quite easy to reproduce (with Nix). Unfortunately I'm on a mobile internet connection right now and can't download a virgin NixOS VM to check if it's reproducible and document the steps.

The other way would be to try to reproduce it on a normal Ubuntu (or whatever) installation without Nix. I can try that, but unfortunately need to download an ISO for that too.

Stay tuned, I'll update this issue as soon as I'm back on a fast and unlimited connection later today.

the-kenny · 2016-07-10T15:59:16Z

Dumb question: How is a channel=stable compiler of 1.9 able to build code using unstable features only stabilized in 1.10? Is this possibly a fundamental issue?

sfackler · 2016-07-10T16:31:16Z

There is a secret environment variable that enables unstable features. It's used for the bootstrap (i.e. building stable stage 2 with stable stage 1).

Ericson2314 · 2016-07-10T16:57:16Z

My guess is that the secret key is not being routed correctly in our case.

the-kenny · 2016-07-10T17:30:18Z

I inserted a call to env in main.mk:280. This seems to print the environment right before the problematic call to rustc.

The value of CFG_BOOTSTRAP_KEY really seems to change between builds of libcore and liballoc. The latter has CFG_BOOTSTRAP_KEY=e8edd0fd while the former has the correct key found in stage0.txt of rustc-1.10.0.

Full build log with env call at https://gist.github.com/the-kenny/f248aeae4307335b5d7a5db00f9205d8#file-gistfile1-txt-L1327

the-kenny · 2016-07-10T20:49:11Z

If someone here uses NixOS or has Nix installed on his system, he can give https://github.com/the-kenny/nixpkgs/tree/rust-1.10.0 a try. nix-build path/to/git-clone -A rustc should be enough to reproduce the issue.

I can also push the branch which patches env into the makefile if there's interest.

alexcrichton · 2016-07-11T16:34:18Z

I can reproduce the error locally, but unfortunately I can't make heads or tails of how to debug nix or what the build file is doing :(

the-kenny · 2016-07-12T08:59:29Z

@alexcrichton For a start, you can pass --keep-failed to nix-build. This will keep the build directory in /tmp when a build fails. You can then cd into /tmp/nix-build-rustc*, source ./env-vars, cd into the build directory (rustc-*) and run make etc. there.

dvc94ch · 2016-07-12T13:59:38Z

Maybe I can help, since I wrote those build files. I know time is scare and nix is weird if you're not used to the nix way, but it would help if you were a little more specific on the heads and tails part.

Currently in the file bootstrap.nix we download the official release tarball for rust 1.9.0 and split it into two packages rustc and cargo to be consistent with the source builds.

The snapshot.nix file should become irrelevant. And the beta.nix and head.nix are for building beta and master.

Then we have makeRustPlatform here: https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/all-packages.nix#L5428 that also isn't particularly important at the moment (since we're still using make to build rustc). Then in default.nix we try building rustc 1.10.0 with the bootstrapped 1.9.0 release binary by passing some parameters to the generic rustc.nix that is used for all versions (stable, beta, master).

I hope this clarifies things.

Thanks,
David

alexcrichton · 2016-07-12T20:11:20Z

Aha, thanks for the flag and the explanation @the-kenny and @dvc94ch! Sorry I'm just a total newb with nix and I was having trouble seeing how things connected in the nix configuration file or just how to get the build directory to stick around so I could poke at it. The --keep-failed flag worked like a charm!

It looks like the compiler is loading the wrong libcore:

INFO:rustc_metadata::loader: lib candidate: /nix/store/pg6rhnijbnm6znb16kfsq6ddnxrclqmi-rustc-bootstrap-1.9.0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-d16b8f0e.rlib
INFO:rustc_metadata::loader: rlib reading metadata from: /nix/store/pg6rhnijbnm6znb16kfsq6ddnxrclqmi-rustc-bootstrap-1.9.0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-d16b8f0e.rlib
INFO:rustc_metadata::loader: reading "libcore-d16b8f0e.rlib" => Duration { secs: 0, nanos: 120785 }
INFO:rustc_metadata::creader: resolved crates:
INFO:rustc_metadata::creader:   name: core
INFO:rustc_metadata::creader:   cnum: 1
INFO:rustc_metadata::creader:   hash: d7f2dc4f887c0a35
INFO:rustc_metadata::creader:   reqd: true
INFO:rustc_metadata::creader:    rlib: /nix/store/pg6rhnijbnm6znb16kfsq6ddnxrclqmi-rustc-bootstrap-1.9.0/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-d16b8f0e.rlib

That is, when it's compiling liballoc it's linking against the 1.9.0 libcore rather than the one that was just created. This appears to be because the snapshot compiler thinks the sysroot is in /nix/... instead of the build directory in /tmp/nix/....

I believe this is because there is apparently a .rustc-wrapper binary where rustc is just a shell script (perhaps inserted in a nix script somewhere?). The compiler currently discovers a sysroot relative to the current binary, so that causes it to go awry.

Does that make sense? Would it be possible to not use a wrapper script? Another possible solution would be to ensure that the binary is copied down as part of src/etc/local_stage0.sh and the script stays in place for other users.

the-kenny · 2016-07-12T20:17:49Z

Yes, this absolutely make sense. I'll investigate it right now and keep you updated.

the-kenny · 2016-07-12T20:20:26Z

Good news: Build seems to continue farther. I'll comment as soon as it's done.

It's funny - I actually mentioned earlier in this issue that the wrapProgram in bootstrap.nix is a no-op in this case. Sometimes no-ops cause the worst kind of bugs!

dvc94ch · 2016-07-12T20:23:03Z

That was all!? Hah I still have to wrap my head around why this happens...
On Jul 12, 2016 10:21 PM, "Moritz Ulrich" [email protected] wrote:

Good news: The build process continues.

It's funny - I actually mentioned earlier in this issue that the
wrapProgram in bootstrap.nix is a no-op in this case. Sometimes no-ops
cause the worst kind of bugs!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#34722 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAtRrxKS7wIcxDiI25G6llEJV7nLltHuks5qU_cpgaJpZM4JH8Uk
.

the-kenny · 2016-07-12T20:34:28Z

@dvc94ch As far as I see, the stage0 copies rustc to the current build directory. As it's wrapped, it copies only the wrapper script, which still calls .rustc-wrapped at its original location.

dvc94ch · 2016-07-12T20:36:48Z

Oh, now I understood the problem. Thanks!!
On Jul 12, 2016 10:35 PM, "Moritz Ulrich" [email protected] wrote:

@dvc94ch https://github.com/dvc94ch As far as I see, the stage0 copies
rustc to the current build directory. As it's wrapped, it copies only the
wrapper script, which still calls .rustc-wrapped at its original
location.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#34722 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAtRr1p3C0YRYB5ewxfVcXLb4eeadUfPks5qU_pzgaJpZM4JH8Uk
.

the-kenny · 2016-07-12T21:38:26Z

Okay, we got much further this time :) It fails somewhere at the end with cargotest complaining about dirty lockfiles.

@alexcrichton Do you think there should be any safeguards in the compiler for this bug? If so, I'd keep the issue open for reference. If not I'm fine with closing this as we found the culprit. Thank you so much for your work!

alexcrichton · 2016-07-12T23:05:41Z

Ok, awesome! I'll close this for now, but that remaining bug sounds bad! It may be a case where we think git exists but it doesn't actually and we're reporting something erroneously. Want to open a new bug with those logs?

Wrapping $out/bin/rustc causes issues in bootstrap builds which are *very* hard to track down. For details, see: rust-lang/rust#34722 (comment)

dvc94ch · 2016-07-13T10:18:09Z

Ups thought it was the nixos thread, I would have replied differently otherwise.

@alexcrichton Thank you for taking the time and doing such an awesome job at tracking down my bugs.

@the-kenny I worked around the lockfile issue by disabling the test, there is a patch in the patches directory for that.

the-kenny · 2016-07-13T10:25:21Z

@dvc94ch Right, I just came up with exactly the same patch by myself without looking at the patches directory :D

@alexcrichton I'll open a separate issue - but I think it's quite minor. Basically the tidy tool fails when the rustc build directory doesn't contain a .git anymore. This is a pet-peeve we have in Nix (for predictability of builds) so I'm not even sure it's worth fixing in rust.

alexcrichton · 2016-07-13T17:37:57Z

@the-kenny oh that's definitely worth fixing! Our source tarballs don't have .git, so I'm surprised it hasn't come up on our releases yet. Feel free to file a bug though!

steveklabnik added the A-build label Jul 8, 2016

the-kenny mentioned this issue Jul 10, 2016

[WIP] Rust: 1.9.0 -> 1.10.0 NixOS/nixpkgs#16807

Closed

7 tasks

alexcrichton closed this as completed Jul 12, 2016

Ericson2314 pushed a commit to Ericson2314/nixpkgs that referenced this issue Jul 13, 2016

rustc-bootstrap-1.9.0: Don't run makeWrapper.

dd169c6

Wrapping $out/bin/rustc causes issues in bootstrap builds which are *very* hard to track down. For details, see: rust-lang/rust#34722 (comment)

the-kenny mentioned this issue Nov 27, 2016

bootstrap rustNightly from rustBeta never works with the 'correct' channels #38026

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

1.10.0 fails to build with bootstrap compiler #34722

1.10.0 fails to build with bootstrap compiler #34722

the-kenny commented Jul 8, 2016 •

edited

Loading

nagisa commented Jul 8, 2016

nagisa commented Jul 8, 2016

the-kenny commented Jul 8, 2016

alexcrichton commented Jul 8, 2016

the-kenny commented Jul 8, 2016

alexcrichton commented Jul 8, 2016

the-kenny commented Jul 8, 2016

MagaTailor commented Jul 8, 2016

alexcrichton commented Jul 9, 2016

the-kenny commented Jul 9, 2016

alexcrichton commented Jul 10, 2016

the-kenny commented Jul 10, 2016

the-kenny commented Jul 10, 2016 •

edited

Loading

sfackler commented Jul 10, 2016

Ericson2314 commented Jul 10, 2016

the-kenny commented Jul 10, 2016 •

edited

Loading

the-kenny commented Jul 10, 2016 •

edited

Loading

alexcrichton commented Jul 11, 2016

the-kenny commented Jul 12, 2016

dvc94ch commented Jul 12, 2016

alexcrichton commented Jul 12, 2016

the-kenny commented Jul 12, 2016

the-kenny commented Jul 12, 2016 •

edited

Loading

dvc94ch commented Jul 12, 2016

the-kenny commented Jul 12, 2016

dvc94ch commented Jul 12, 2016

the-kenny commented Jul 12, 2016

alexcrichton commented Jul 12, 2016

dvc94ch commented Jul 13, 2016

the-kenny commented Jul 13, 2016

alexcrichton commented Jul 13, 2016

1.10.0 fails to build with bootstrap compiler #34722

1.10.0 fails to build with bootstrap compiler #34722

Comments

the-kenny commented Jul 8, 2016 • edited Loading

nagisa commented Jul 8, 2016

nagisa commented Jul 8, 2016

the-kenny commented Jul 8, 2016

alexcrichton commented Jul 8, 2016

the-kenny commented Jul 8, 2016

alexcrichton commented Jul 8, 2016

the-kenny commented Jul 8, 2016

MagaTailor commented Jul 8, 2016

alexcrichton commented Jul 9, 2016

the-kenny commented Jul 9, 2016

alexcrichton commented Jul 10, 2016

the-kenny commented Jul 10, 2016

the-kenny commented Jul 10, 2016 • edited Loading

sfackler commented Jul 10, 2016

Ericson2314 commented Jul 10, 2016

the-kenny commented Jul 10, 2016 • edited Loading

the-kenny commented Jul 10, 2016 • edited Loading

alexcrichton commented Jul 11, 2016

the-kenny commented Jul 12, 2016

dvc94ch commented Jul 12, 2016

alexcrichton commented Jul 12, 2016

the-kenny commented Jul 12, 2016

the-kenny commented Jul 12, 2016 • edited Loading

dvc94ch commented Jul 12, 2016

the-kenny commented Jul 12, 2016

dvc94ch commented Jul 12, 2016

the-kenny commented Jul 12, 2016

alexcrichton commented Jul 12, 2016

dvc94ch commented Jul 13, 2016

the-kenny commented Jul 13, 2016

alexcrichton commented Jul 13, 2016

the-kenny commented Jul 8, 2016 •

edited

Loading

the-kenny commented Jul 10, 2016 •

edited

Loading

the-kenny commented Jul 10, 2016 •

edited

Loading

the-kenny commented Jul 10, 2016 •

edited

Loading

the-kenny commented Jul 12, 2016 •

edited

Loading