Windows: Already canonicalized path cannot be re-canonicalized with relative paths

[Ciantic]

I tried this code:

use std::path::PathBuf;
let cwindows: PathBuf = "C:\\Windows\\".into();
let cwindows_can = cwindows.canonicalize().unwrap();

// works correctly:
cwindows.join(r".\notepad.exe").canonicalize().unwrap();

// unexpectedly, following throws error "Not Found" error:
cwindows_can.join(r".\notepad.exe").canonicalize().unwrap();

I expected to see this happen:

Last line should work. Error on the last line means that already canonicalized path (with UNC start) cannot be re-canonicalized with relative paths in them.

This is especially problematic because canonicalize is only way to normalize paths. Issue could be solved if there were a way to normalize paths (e.g. #59117) without calling Windows API, which probably is the culprit here.

Instead, this happened:

Last line throws an error.

Meta

rustc --version --verbose:

rustc --version --verbose
rustc 1.49.0 (e1884a8e3 2020-12-29)
binary: rustc
commit-hash: e1884a8e3c3e813aada8254edfa120e85bf5ffca
commit-date: 2020-12-29
host: x86_64-pc-windows-msvc
release: 1.49.0

[ehuss]

I think this is fundamentally a duplicate of #42869 and #59117. On Windows, \\?\ paths fundamentally cannot be normalized. Some of the various considerations:

• canonicalize should stop returning \\?\ device paths, since they are so cumbersome, and instead use something like GetFullPathNameW. One major drawback here is the difficulty with long path names.
• A new function should be added that does essentially what GetFullPathNameW does to replace canonicalize. This also struggles with long path names.
• You can swap \\?\ with \\.\ which does canonicalization, but you lose long-path support.
• Path::join and other methods could be smart and do normalization on the fly. This is a controversial option for many reasons.
• Some external crate should develop a new Path API that is smarter and prove itself as useful and reliable, allowing experimentation outside of std.

Unfortunately, none of these are easy or without substantial risk.

[Ciantic]

On Windows, \\?\ paths fundamentally cannot be normalized.

This was news for me, and reason I opened this issue. It's not obvious.

In short term, implementing a normalize function along the lines of #59117 should be doable already. It would be highly beneficial. At least I don't find it risky, it just requires someone to implement it.

On your last point, any external crate experiment for larger re-think should probably try to architect it along these lines: #66621 because it's not uncommon need to handle posix paths in Windows; or vice versa.

[jonas-schievink]

I think this is fundamentally a duplicate of #42869 and #59117.

Closing in favor of those.

[dylni]

Some external crate should develop a new Path API that is smarter and prove itself as useful and reliable, allowing experimentation outside of std.

If you're looking to safely use GetFullPathNameW, I created normpath for this reason. It's still pretty new, but I've been attempting to integrate it into Cargo to improve Windows support: rust-lang/cargo#8881, rust-lang/cargo#8964