Add basic help page to document rust-timer commands #846
Conversation
|
The page looks like this on my local machine:
EDIT: Updated screenshot based on latest changes. |
| <li><code>exclude=<EXCLUDE></code> is similar to <code>include=</code>, but instead skips any benchmarks that have one of the items in the list of substrings as a substring of its name.</li> | ||
| <li><code>runs=<RUNS></code> configures how many times the benchmark is run. <code><RUNS></code> is an integer.</li> | ||
| </ul> | ||
| <p><code>@rust-timer</code> has more commands than just <code>@rust-timer queue</code>, but the <code>queue</code> command is the most used.</p> |
There was a problem hiding this comment.
What other commands does it support?
There was a problem hiding this comment.
There's also build, make-pr-for, and update-branch-for, but I can't document them because I don't anything about them (except for a little bit about make-pr-for).
There was a problem hiding this comment.
I think documenting build is worthwhile, the other two are buggy and we shouldn't document them for now.
There was a problem hiding this comment.
Could we save documenting build for a later PR? I know nothing about it so I can't document it.
There was a problem hiding this comment.
Ok! It's pretty much equivalent to queue, just takes an additional SHA argument (useful if e.g. bors flakes and forgets to post or so)
site/static/help.html
Outdated
| <ul> | ||
| <li><code>include=<INCLUDE></code> only runs benchmarks with <code><INCLUDE></code> in their title. For example, if you just want to run rustdoc benchmarks, do <code>@bors try @rust-timer queue include=-doc</code>; the <code>-doc</code> matches the <code>-doc</code> appended to rustdoc benchmarks. <code><INCLUDE></code> is a comma-separated list of substrings; only one of the items in the list of substrings must match.</li> | ||
| <li><code>exclude=<EXCLUDE></code> is similar to <code>include=</code>, but instead skips any benchmarks that have one of the items in the list of substrings as a substring of its name.</li> | ||
| <li><code>runs=<RUNS></code> configures how many times the benchmark is run. <code><RUNS></code> is an integer.</li> |
There was a problem hiding this comment.
This is like cap-lints -- some benchmarks run once, some run 3 times by default (we generally take the minimum of the metrics), and by default there's not an override, so we use the configuration from the benchmark.
There was a problem hiding this comment.
So you're saying that some benchmarks run multiple times, e.g. with different cap-lints settings? And this option will let you override that?
There was a problem hiding this comment.
No, not with different cap-lints settings. I'm saying that this option forces a particular number of runs; without it set, the number of runs varies from benchmark to benchmark.
There was a problem hiding this comment.
I think we're saying the same thing, but I must not have been clear enough. As I understand it, you're saying that by default different benchmarks have different numbers of runs (some might be 1, some 2, some 3, etc.), and if you use runs= you can make all the benchmarks have the same number of runs (e.g., 2).
Are there any benchmarks that have their default run count set to 0?
There was a problem hiding this comment.
No, all benchmarks run at least once by default. I'm not sure if runs=0 would work, but it would also be pretty useless I'd expect.
There was a problem hiding this comment.
I added more info in 677a309, does that look good to you?
Both I and `@jyn514` had no idea that `@rust-timer queue exclude` existed because there's no documentation, or at least I wasn't able to find any publicized documentation.
|
CI failures look spurious. |
Mark-Simulacrum
left a comment
Mark-Simulacrum
left a comment
There was a problem hiding this comment.
This looks great to me, left a few comments which I think could be integrated, and then we can merge.
3 participants
Both I and
@jyn514had no idea that@rust-timer queue excludeexisted because there's no documentation, or at least I wasn't able to
find any publicized documentation.
r? @Mark-Simulacrum
cc @jyn514