Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML #113040

Kobzol · 2023-06-25T22:06:46Z

This PR adds an option for rustc to emit LLVM optimization remarks to a set of YAML files, which can then be digested by existing tools, like https://github.com/OfekShilon/optview2. When -Cremark-dir is passed, and remarks are enabled (-Cremark=all), the remarks will be now written to the specified directory, instead of being printed to standard error output. The files are named based on the CGU from which they are being generated.

Currently, the remarks are written using the LLVM streaming machinery, directly in the diagnostics handler. It seemed easier than going back to Rust and then form there back to C++ to use the streamer from the diagnostics handler. But there are many ways to implement this, of course, so I'm open to suggestions :)

I included some comments with questions into the code. Also, I'm not sure how to test this.

r? @tmiasko

rustbot · 2023-06-25T22:06:54Z

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

compiler/rustc_session/src/options.rs

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp

compiler/rustc_codegen_llvm/src/back/write.rs

bors · 2023-06-28T01:13:06Z

☔ The latest upstream changes (presumably #113105) made this pull request unmergeable. Please resolve the merge conflicts.

Kobzol · 2023-06-28T14:51:09Z

Rebased and turned the flag into an unstable one. The files are now also split based on the "section" of codegen that is being performed.

compiler/rustc_codegen_llvm/src/back/write.rs

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp

compiler/rustc_codegen_llvm/src/back/write.rs

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp

tmiasko · 2023-06-28T21:10:38Z

As for testing maybe a run-make test similar to the existing ui test optimization-remark.rs? I would be nice to test filtering as well.

There seems to be a an additional pre-existing issue: the diagnostic handler is not configured at all during post-link ThinLTO optimization phase.

Kobzol · 2023-06-29T09:41:48Z

There seems to be a an additional pre-existing issue: the diagnostic handler is not configured at all during post-link ThinLTO optimization phase.

Oh, so that is not on purpose? I was wondering why is there only fat LTO 😅

Kobzol · 2023-06-29T10:04:04Z

I added a simple test, let me know if it's any good or if I should extend it :)

compiler/rustc_codegen_llvm/src/back/write.rs

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp

Kobzol · 2023-07-01T08:38:58Z

Thanks for the review, I tried to improve error reporting in general.

tmiasko · 2023-07-01T13:05:14Z

Thanks!

@bors r+

bors · 2023-07-01T13:05:16Z

📌 Commit fc61ad3 has been approved by tmiasko

It is now in the queue for this repository.

…asko Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML This PR adds an option for `rustc` to emit LLVM optimization remarks to a set of YAML files, which can then be digested by existing tools, like https://github.com/OfekShilon/optview2. When `-Cremark-dir` is passed, and remarks are enabled (`-Cremark=all`), the remarks will be now written to the specified directory, **instead** of being printed to standard error output. The files are named based on the CGU from which they are being generated. Currently, the remarks are written using the LLVM streaming machinery, directly in the diagnostics handler. It seemed easier than going back to Rust and then form there back to C++ to use the streamer from the diagnostics handler. But there are many ways to implement this, of course, so I'm open to suggestions :) I included some comments with questions into the code. Also, I'm not sure how to test this. r? `@tmiasko`

matthiaskrgr · 2023-07-02T08:26:47Z

@bors r-
#113255 (comment)

tests/run-make/optimization-remarks-dir/Makefile

tmiasko · 2023-07-02T09:51:30Z

@bors delegate=Kobzol

compiler/rustc_session/src/config.rs

Kobzol · 2023-07-02T11:40:48Z

@bors r-

Kobzol · 2023-07-02T11:41:57Z

@bors r=@tmiasko

bors · 2023-07-02T11:41:59Z

📌 Commit 62728c7 has been approved by tmiasko

It is now in the queue for this repository.

workingjubilee · 2023-07-02T11:44:22Z

@bors rollup=iffy

bors · 2023-07-02T12:48:49Z

⌛ Testing commit 62728c7 with merge 131a036...

bors · 2023-07-02T15:14:51Z

☀️ Test successful - checks-actions
Approved by: tmiasko
Pushing 131a036 to master...

rust-timer · 2023-07-02T16:31:48Z

Finished benchmarking commit (131a036): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	1.0%	[0.7%, 1.3%]	2
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.4%, -0.4%]	2
Improvements ✅ (secondary)	-3.2%	[-3.2%, -3.2%]	1
All ❌✅ (primary)	0.3%	[-0.4%, 1.3%]	4

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.7%	[-0.7%, -0.7%]	1
Improvements ✅ (secondary)	-4.3%	[-5.3%, -3.2%]	6
All ❌✅ (primary)	-0.7%	[-0.7%, -0.7%]	1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 662.06s -> 661.354s (-0.11%)

Add hotness data to LLVM remarks Slight improvement of rust-lang#113040. This makes sure that if PGO is used, remarks generated using `-Zremark-dir` will include the `Hotness` attribute. r? `@tmiasko`

Add hotness data to LLVM remarks Slight improvement of rust-lang/rust#113040. This makes sure that if PGO is used, remarks generated using `-Zremark-dir` will include the `Hotness` attribute. r? `@tmiasko`

OfekShilon · 2025-03-27T09:01:18Z

@Kobzol Thank you for this! I hope to incorporate this into compiler-explorer: compiler-explorer/compiler-explorer#7525

Did this make it into official releases? Is there a non--Z switch for it, and if so - from which version?

Kobzol · 2025-03-27T09:05:26Z

It wasn't stabilized yet, and I'm not sure if we should stabilize it, tbh, it's quite niche. I created a Cargo tool for automating the usage of the flag and visualizing the results (https://github.com/Kobzol/cargo-remark).

For using it in compiler explorer, you'd need to use a nightly Rust compiler.

OfekShilon · 2025-04-15T08:01:08Z

@Kobzol this is now in compiler-explorer for rustc nightly: https://www.compiler-explorer.com/z/b8adnsz5q
Note the example link checks all filters, by default only missed remarks are displayed.

Thank you for this work!

Kobzol · 2025-04-15T08:02:53Z

That's very cool, thanks!

rustbot assigned tmiasko Jun 25, 2023

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Jun 25, 2023

bjorn3 reviewed Jun 25, 2023

View reviewed changes

compiler/rustc_session/src/options.rs Outdated Show resolved Hide resolved

Kobzol force-pushed the llvm-remark-streamer branch from 6d1171d to 293bd10 Compare June 25, 2023 22:18

Kobzol commented Jun 25, 2023

View reviewed changes

compiler/rustc_llvm/llvm-wrapper/RustWrapper.cpp Outdated Show resolved Hide resolved

tmiasko reviewed Jun 26, 2023

View reviewed changes

compiler/rustc_codegen_llvm/src/back/write.rs Outdated Show resolved Hide resolved

Kobzol changed the title ~~Add -Cremark-dir to write LLVM optimization remarks to YAML~~ Add -Zremark-dir unstable flag to write LLVM optimization remarks to YAML Jun 28, 2023

Kobzol force-pushed the llvm-remark-streamer branch from 293bd10 to adfcae2 Compare June 28, 2023 14:50

tmiasko reviewed Jun 28, 2023

View reviewed changes

tmiasko reviewed Jun 30, 2023

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 1, 2023

workingjubilee mentioned this pull request Jul 2, 2023

Rollup of 7 pull requests #113255

Closed

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 2, 2023

tmiasko reviewed Jul 2, 2023

View reviewed changes

tests/run-make/optimization-remarks-dir/Makefile Outdated Show resolved Hide resolved

klensy reviewed Jul 2, 2023

View reviewed changes

compiler/rustc_session/src/config.rs Outdated Show resolved Hide resolved

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 2, 2023

Add rustc option to output LLVM optimization remarks to YAML files

62728c7

Kobzol force-pushed the llvm-remark-streamer branch from 79a1866 to 62728c7 Compare July 2, 2023 11:41

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 2, 2023

bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 2, 2023

bors merged commit 131a036 into rust-lang:master Jul 2, 2023

rustbot added this to the 1.72.0 milestone Jul 2, 2023

Kobzol deleted the llvm-remark-streamer branch July 2, 2023 15:15

Kobzol mentioned this pull request Jul 6, 2023

Emit optimization remarks to the final binary, not the console #96705

Open

Kobzol mentioned this pull request Aug 3, 2023

Add hotness data to LLVM remarks #114439

Merged

OfekShilon mentioned this pull request Mar 27, 2025

[REQUEST]: Opt-remarks for rustc nightly compiler-explorer/compiler-explorer#7525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML #113040

Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML #113040

Kobzol commented Jun 25, 2023

rustbot commented Jun 25, 2023

bors commented Jun 28, 2023

Kobzol commented Jun 28, 2023

tmiasko commented Jun 28, 2023

Kobzol commented Jun 29, 2023

Kobzol commented Jun 29, 2023

Kobzol commented Jul 1, 2023

tmiasko commented Jul 1, 2023

bors commented Jul 1, 2023

matthiaskrgr commented Jul 2, 2023

tmiasko commented Jul 2, 2023

Kobzol commented Jul 2, 2023

Kobzol commented Jul 2, 2023

bors commented Jul 2, 2023

workingjubilee commented Jul 2, 2023

bors commented Jul 2, 2023

bors commented Jul 2, 2023

rust-timer commented Jul 2, 2023

OfekShilon commented Mar 27, 2025

Kobzol commented Mar 27, 2025

OfekShilon commented Apr 15, 2025

Kobzol commented Apr 15, 2025

Add -Zremark-dir unstable flag to write LLVM optimization remarks to YAML #113040

Add -Zremark-dir unstable flag to write LLVM optimization remarks to YAML #113040

Conversation

Kobzol commented Jun 25, 2023

rustbot commented Jun 25, 2023

bors commented Jun 28, 2023

Kobzol commented Jun 28, 2023

tmiasko commented Jun 28, 2023

Kobzol commented Jun 29, 2023

Kobzol commented Jun 29, 2023

Kobzol commented Jul 1, 2023

tmiasko commented Jul 1, 2023

bors commented Jul 1, 2023

matthiaskrgr commented Jul 2, 2023

tmiasko commented Jul 2, 2023

Kobzol commented Jul 2, 2023

Kobzol commented Jul 2, 2023

bors commented Jul 2, 2023

workingjubilee commented Jul 2, 2023

bors commented Jul 2, 2023

bors commented Jul 2, 2023

rust-timer commented Jul 2, 2023

Overall result: no relevant changes - no action needed

Instruction count

Max RSS (memory usage)

Cycles

Binary size

OfekShilon commented Mar 27, 2025

Kobzol commented Mar 27, 2025

OfekShilon commented Apr 15, 2025

Kobzol commented Apr 15, 2025

Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML #113040

Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML #113040