From 9d515e5163cd41972b6f31333b21d0170dbc0558 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Thu, 1 Aug 2019 10:31:02 +0200 Subject: [PATCH 1/5] describe toolstate system --- index.md | 9 +++++--- toolstate.md | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 3 deletions(-) create mode 100644 toolstate.md diff --git a/index.md b/index.md index 93c2414b2..63903e0b0 100644 --- a/index.md +++ b/index.md @@ -16,8 +16,10 @@ This site contains supplementary documentation useful to the members of

No tool breakages week

- To ensure the beta release includes all the tools, no tool breakages - are allowed in the week before the beta cutoff. + To ensure the beta release includes all the tools, no + tool breakages + are allowed in the week before the beta cutoff + (except for nightly-only tools).

@@ -38,7 +40,8 @@ This site contains supplementary documentation useful to the members of - [The current PR testing queue](https://buildbot2.rust-lang.org/homu/queue/rust) - [The PR queue over time](https://rust-lang-nursery.github.io/rustc-pr-tracking/) -- [Toolstate (rls, rustfmt, clippy)](https://rust-lang-nursery.github.io/rust-toolstate/) +- [Current toolstate (rls, rustfmt, clippy)](https://rust-lang-nursery.github.io/rust-toolstate/) + and [how the toolstate system works](toolstate.html). - [Components availability history](https://rust-lang.github.io/rustup-components-history/index.html) - [Release history](releases.html). Links to previous release artifacts. - [Platform support](platform-support.html). diff --git a/toolstate.md b/toolstate.md new file mode 100644 index 000000000..9ca6b1741 --- /dev/null +++ b/toolstate.md @@ -0,0 +1,63 @@ +--- +layout: default +title: Handling of tools embedded in the rustc repo ("toolstate") +--- + +# Handling of tools embedded in the rustc repo ("toolstate") + +The Rust repository contains several external tools and documents as git +submodules (e.g. clippy, rls, the [Book], the [Reference]). Many of those are +very tightly coupled to the compiler and depend on internal APIs that change all +the time, but they are not actually essential to get the compiler itself to +work. To make API changes less painful, these tools are allowed to "break" +temporarily. PRs can still land and nightlies still get released even when some +tools are broken. Their current status is managed by the +[toolstate system][toolstate]. (Cargo is needed to build rustc, so it is not +subject to the toolstate system and instead just has to always work.) + +The three possible states of a "tool" (this includes the documentation managed +by the toolstate system, where we run doctests) are: `test-pass`, `test-fail`, +`build-fail`. + +This page gives a rough overview how the toolstate system works, and what the +rules are for when which tools are (not) allowed to break. + +## Toolstate Rules + +* For all tools, if a PR changes that tool (if it changes the commit used by the + submodule), the tool has to be in `test-pass` after this PR or else CI will + fail. + +* For all tools except for "nightly only" tools, the following extra rules are applied: + * If a PR lands on the `beta` or `stable` branch, the tool has to be `test-pass`. + * If a PR lands on `master` in the week before the beta is cut, and that PR + regresses the tool (if it makes the state "worse"), CI fails. This is to + help make sure all these tools become `test-pass` so that a beta can be + cut. (See the [Forge index][forge] for when the next beta cutoff is + happening.) + + At the time of writing, the following tools are "nightly only": rustc-guide, + miri, embedded-book. + +## Updating the toolstate repository + +Updating the [toolstate repository][toolstate] happens in two steps: when CI +runs on the `auto` branch (where bors moves a PR to test if it is good for +integration), the "tool" runners for the individual platforms (at the time of +writing, Linux and Windows) each submit a JSON file to the repository recording +the state of each tool for the commit they are testing. Later, if that commit +actually entirely passed CI and bors moves it to the `master` branch, the +"current tool status" in the toolstate repository is updated appropriately. + +These scripts also automatically ping some people and create issues when tools +break. + +For further details, see the comments in the involved files: +[`checktools.sh`](https://github.com/rust-lang/rust/blob/master/src/ci/docker/x86_64-gnu-tools/checktools.sh), +[`publish_toolstate.py`](https://github.com/rust-lang/rust/blob/master/src/tools/publish_toolstate.py), +as well as the other files mentioned there. + +[Book]: https://doc.rust-lang.org/book/ +[Reference]: https://doc.rust-lang.org/reference/ +[toolstate]: https://rust-lang-nursery.github.io/rust-toolstate/ +[forge]: index.html From 30ed2439263b95182a771099a694e77b4bd9ff75 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Thu, 1 Aug 2019 18:16:21 +0200 Subject: [PATCH 2/5] factual correction --- toolstate.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/toolstate.md b/toolstate.md index 9ca6b1741..71ca40ff9 100644 --- a/toolstate.md +++ b/toolstate.md @@ -12,8 +12,8 @@ the time, but they are not actually essential to get the compiler itself to work. To make API changes less painful, these tools are allowed to "break" temporarily. PRs can still land and nightlies still get released even when some tools are broken. Their current status is managed by the -[toolstate system][toolstate]. (Cargo is needed to build rustc, so it is not -subject to the toolstate system and instead just has to always work.) +[toolstate system][toolstate]. (Cargo is not subject to the toolstate system and +instead just has to always work.) The three possible states of a "tool" (this includes the documentation managed by the toolstate system, where we run doctests) are: `test-pass`, `test-fail`, From d914390cfa15ffbd601e03aeecceef95e9d7b314 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Thu, 1 Aug 2019 18:19:45 +0200 Subject: [PATCH 3/5] explain how to add a tool (text by ehuss) --- toolstate.md | 33 ++++++++++++++++++++++++++++----- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/toolstate.md b/toolstate.md index 71ca40ff9..b6da703fd 100644 --- a/toolstate.md +++ b/toolstate.md @@ -41,7 +41,7 @@ rules are for when which tools are (not) allowed to break. ## Updating the toolstate repository -Updating the [toolstate repository][toolstate] happens in two steps: when CI +Updating the [toolstate repository] happens in two steps: when CI runs on the `auto` branch (where bors moves a PR to test if it is good for integration), the "tool" runners for the individual platforms (at the time of writing, Linux and Windows) each submit a JSON file to the repository recording @@ -52,12 +52,35 @@ actually entirely passed CI and bors moves it to the `master` branch, the These scripts also automatically ping some people and create issues when tools break. -For further details, see the comments in the involved files: -[`checktools.sh`](https://github.com/rust-lang/rust/blob/master/src/ci/docker/x86_64-gnu-tools/checktools.sh), -[`publish_toolstate.py`](https://github.com/rust-lang/rust/blob/master/src/tools/publish_toolstate.py), -as well as the other files mentioned there. +For further details, see the comments in the involved files: [`checktools.sh`], +[`publish_toolstate.py`] as well as the other files mentioned there. +## Adding a tool + +To add a new tool to be tracked, the following steps must be taken: + +1. Create a PR to rust-lang/rust that adds the submodule along with any + necessary build system / bootstrap updates. Be careful that the tests + properly support `--no-fail-fast`. +2. Include changes to [`checktools.sh`]: + - Build the tool at the top. This is the step that actually generates the + JSON status for the tool. When `save-toolstates` is set in + `config.toml`, the rust build system will write a JSON file with the + status of each test. + - Add the tool to `status_check` with whether it should be a beta blocker + or not. +3. Update [`publish_toolstate.py`] to add the tool. This includes a list of + people to ping if the tool is broken, and its source repo. (Note: At the + time of this writing, these users must have permissions to be assignable on + rust-lang/rust GitHub.) +4. Submit a PR to the [toolstate repository] to manually add the tool to the + [`latest.json`] file. + +[`checktools.sh`]: https://github.com/rust-lang/rust/blob/master/src/ci/docker/x86_64-gnu-tools/checktools.sh +[`publish_toolstate.py`]: https://github.com/rust-lang/rust/blob/master/src/tools/publish_toolstate.py +[`latest.json`]: https://github.com/rust-lang-nursery/rust-toolstate/blob/master/_data/latest.json [Book]: https://doc.rust-lang.org/book/ [Reference]: https://doc.rust-lang.org/reference/ [toolstate]: https://rust-lang-nursery.github.io/rust-toolstate/ +[toolstate repository]: https://github.com/rust-lang-nursery/rust-toolstate/ [forge]: index.html From 841352bf8cc767c185c28e206e388287af7a6fb1 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Thu, 1 Aug 2019 19:13:43 +0200 Subject: [PATCH 4/5] add link to possible --no-fail-fast issue --- toolstate.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/toolstate.md b/toolstate.md index b6da703fd..a7011fe37 100644 --- a/toolstate.md +++ b/toolstate.md @@ -61,7 +61,8 @@ To add a new tool to be tracked, the following steps must be taken: 1. Create a PR to rust-lang/rust that adds the submodule along with any necessary build system / bootstrap updates. Be careful that the tests - properly support `--no-fail-fast`. + properly support `./x.py --no-fail-fast` to avoid + [issues like this](https://github.com/rust-lang/rust/pull/63089). 2. Include changes to [`checktools.sh`]: - Build the tool at the top. This is the step that actually generates the JSON status for the tool. When `save-toolstates` is set in From bbc41041f5965edbdb85047f54ca235b53b19701 Mon Sep 17 00:00:00 2001 From: Ralf Jung Date: Tue, 6 Aug 2019 09:04:25 +0200 Subject: [PATCH 5/5] add link from infrastructure page --- infrastructure.md | 1 + 1 file changed, 1 insertion(+) diff --git a/infrastructure.md b/infrastructure.md index c28018a27..3b47cc706 100644 --- a/infrastructure.md +++ b/infrastructure.md @@ -113,6 +113,7 @@ The state of tools included with Rust are tracked on the [toolstate page](https://rust-lang-nursery.github.io/rust-toolstate/). When each PR is merged via CI, the status of each tool is recorded in a JSON file and stored in the [toolstate repo](https://github.com/rust-lang-nursery/rust-toolstate). +For further information, see the [toolstate system documentation](toolstate.html). ## Rustup components history