Two source files with same name, different extension generate same object file name

[Noxime]

Hi,

When building a project that has 2 sibling files with same name but different extension, they both result in the same filename for the object file. In my case:

...
.file("rv003usb.c")
.file("rv003usb.S")

produces two rv003usb.o files, with the latter overwriting the former. Thus, the resulting build either fails or is missing some symbols. I traced this to this line:

cc-rs/src/command_helpers.rs

Lines 330 to 332 in 29d6ca1

	let obj = dst
	.join(format!("{:016x}-{}", hasher.finish(), basename))
	.with_extension("o");

[madsmtm]

Yeah that's problematic. We could solve this two ways:

1. Add the extension to the hash.
2. Rewrite the whole thing to instead keep a HashMap<OsString, NonZero<u32>> for the compilation session, with each output file bumping a number in there.
   • This would make absolutely sure we never leak information about absolute dirs
   • Though might make the build less reproducible if the order of files change?
   • Related: Fix for different hashes between different build paths #1277

[madsmtm]

@NobodyXu do you know why we're even invoking the compiler for each source file? As far as I know, C compilers support building multiple files and linking them together in one step?

(Of course if we did #545 we'd have to do something else)

[NobodyXu]

I think adding extension to the hash is the right solution.

do you know why we're even invoking the compiler for each source file

Primary reason is for parallel compilation.

Also some people depends on it (extract the file from archive or uses compile_intermediate)

[madsmtm]

Primary reason is for parallel compilation.

Doesn't Clang/GCC handle this internally themselves?

I just mean, having one process that loads and shares all the header files has bound to be faster than 10 processes doing that?

[NobodyXu]

Doesn't Clang/GCC handle this internally themselves?

No, AFAIK clang/gcc is single-thread.

I just mean, having one process that loads and shares all the header files has bound to be faster than 10 processes doing that?

Well that depends, if a header is large enough and reused multiple times, pre-compiling it to gcc/clang internal format would speed that up (and is supported).

In cc-rs, we compile the source files in parallel (.c/.cxx/.S), in the same way how Makefile would usually compile them in parallel.

[madsmtm]

Doesn't Clang/GCC handle this internally themselves?

No, AFAIK clang/gcc is single-thread.

I just mean, having one process that loads and shares all the header files has bound to be faster than 10 processes doing that?

Well that depends, if a header is large enough and reused multiple times, pre-compiling it to gcc/clang internal format would speed that up (and is supported).

In cc-rs, we compile the source files in parallel (.c/.cxx/.S), in the same way how Makefile would usually compile them in parallel.

Ah, okay, thanks for the explanation!

[NobodyXu]

No worries.

BTW rustc is also designed in the same way originally, but recently rustc adds in multithreading to speedup compilation since the compile unit in rust is crate, whilst in c/c++ it's file.

The c/c++ model actually scales better despite it creates multiple processes (would be great if they can do multithreading), but with C++ module things are getting more complex.

https://stackoverflow.com/questions/33307657/how-do-i-use-c-modules-in-clang

and c++ module might be more expensive to compile as well

https://www.reddit.com/r/cpp/s/wLcdeTUndc