no-op remap_path_prefix change invalidates incremental cache

[ashi009]

I'm building rust code in a CI environment, setting -Cincremental to a shared location. CI allocates a new working directory for each build. To prevent leaking that info to the debug info, we set --remap_path_prefix=${PWD}= to eliminate the unwanted prefix from the debug info.

rustc src/lib.rs \
  --crate-name=xxx \
  --crate-type=rlib \
  --error-format=human \
  --out-dir=${OUTDIR} \
  --remap-path-prefix=${PWD}= \
  --emit=dep-info,link \
  -Ccodegen-units=256 \
  -Zincremental_info \
  -Cincremental=${CACHEDIR}

I expected this will reuse the incremental build cache, but it's not:

[incremental] session directory: 259 files hard-linked
[incremental] session directory: 0 files copied
[incremental] completely ignoring cache because of differing commandline arguments

Further inspection of how the hash being generated, I found:

rust/compiler/rustc_session/src/options.rs

Lines 195 to 222 in 017ae1b

	/// Remap source path prefixes in all output (messages, object files, debug, etc.).
	remap_path_prefix: Vec<(PathBuf, PathBuf)> [TRACKED_NO_CRATE_HASH],
	/// Base directory containing the `src/` for the Rust standard library, and
	/// potentially `rustc` as well, if we can find it. Right now it's always
	/// `$sysroot/lib/rustlib/src/rust` (i.e. the `rustup` `rust-src` component).
	///
	/// This directory is what the virtual `/rustc/$hash` is translated back to,
	/// if Rust was built with path remapping to `/rustc/$hash` enabled
	/// (the `rust.remap-debuginfo` option in `config.toml`).
	real_rust_source_base_dir: Option<PathBuf> [TRACKED_NO_CRATE_HASH],
	edition: Edition [TRACKED],
	/// `true` if we're emitting JSON blobs about each artifact produced
	/// by the compiler.
	json_artifact_notifications: bool [TRACKED],
	/// `true` if we're emitting a JSON blob containing the unused externs
	json_unused_externs: JsonUnusedExterns [UNTRACKED],
	/// `true` if we're emitting a JSON job containing a future-incompat report for lints
	json_future_incompat: bool [TRACKED],
	pretty: Option<PpMode> [UNTRACKED],
	/// The (potentially remapped) working directory
	working_dir: RealFileName [TRACKED],
	color: ColorConfig [UNTRACKED],

for our case, working_dir always uses remapped_path "", which is stable over build. but the remap_path_prefix is different everytime. and the logic for populating working_dir is at:

rust/compiler/rustc_session/src/config.rs

Lines 2690 to 2695 in 017ae1b

	let working_dir = std::env::current_dir().unwrap_or_else(|e| {
	early_dcx.early_fatal(format!("Current directory is invalid: {e}"));
	});
	let file_mapping = file_path_mapping(remap_path_prefix.clone(), &unstable_opts);
	let working_dir = file_mapping.to_real_filename(&working_dir);

So to get correct remapped working_dir, I must set --remap-path-prefix, but setting it will change remap_path_prefix and causing hash mismatch:

rust/compiler/rustc_incremental/src/persist/load.rs

Lines 159 to 164 in 017ae1b

	if prev_commandline_args_hash != expected_hash {
	if sess.opts.unstable_opts.incremental_info {
	eprintln!(
	"[incremental] completely ignoring cache because of \
	differing commandline arguments"
	);

Meta

rustc --version --verbose:

rustc 1.82.0-nightly (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: aarch64-apple-darwin
release: 1.82.0-nightly
LLVM version: 19.1.1

[Noratrieb]

I suspect this wouldn't be sound, as this is not a "no-op" mapping, it actually does stuff, and removing the flag changes the output.

[Noratrieb]

so I'm going to close this as not implementable.
you need the same directory every time to keep incremental working, which could be done with containers or something like that

[ashi009]

this is not always an option for people to control where the code sits. For example, bazel sandboxes.

And the newish unstable feature -Zremapping_cwd_path supposed to be a more intentional flag for this use case, but still triggering the side effect of causing the cache invalidation.

The incremental cache is based on typing info and remapped file paths. Why the config for calculating the remapped file path is even considered a factor affecting the cache state. Most importantly, if it's critical it should be marked as [TRACKED] instead, but it won't make into the final rlib/rmeta but only its effects.

Please reopen for further discussion. Thanks!

[adamcavendish]

so I'm going to close this as not implementable. you need the same directory every time to keep incremental working, which could be done with containers or something like that

Please do not make assumptions ahead of time that people are allowed to control the sandbox implementation.

However, I would suggest it super weird that moving around incremental cache directory along with the source code to another place shall ever cause a complete rebuild. Please do re-consider the feature again.

[Noratrieb]

I can reopen, but this is not easily fixable with the current scheme.

Incremental compilation is based on the entire "query" graph of the compiler. All inputs must be tracked, especially if they affect outputs, which this flag does.

It would be unsound to reuse a cache where the flag was used in a session where it was not used.
Now in your special case it wouldn't be wrong to re-use it, but we just don't do such fine-grained tracking of options.