From 04c8191719761dd8617fb84a86fe114ed7ac6eab Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Wed, 1 Apr 2020 13:19:11 +0200 Subject: [PATCH 1/7] Explain how to work with subrepos --- CONTRIBUTING.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 2843944b2e181..69ce9dae5aa9b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -188,7 +188,53 @@ with one another are rolled up. Speaking of tests, Rust has a comprehensive test suite. More information about it can be found [here][rctd]. -### External Dependencies +### External Dependencies (subrepo) + +As a developer to this repository, you don't have to treat the following external projects +differently from other crates that are directly in this repo: + +* none so far, see https://github.com/rust-lang/rust/issues/70651 for more info + +They are just regular files and directories. This is in contrast to `submodule` dependencies +(see below for those). + +If you want to synchronize or otherwise work with subrepos, install the `git subrepo` command via +instructions found at https://github.com/ingydotnet/git-subrepo + +#### Synchronizing a subrepo + +There are two synchronization directions: `subrepo push` and `subrepo pull`. Both operations create +a synchronization commit in the rustc repo. +This commit is very important in order to make future synchronizations work. +Do not rebase this commit under any circumstances. +Prefer to merge in case of conflicts or redo the operation if you really need to rebase. + +A `git subrepo push src/tools/clippy` +takes all the changes that +happened to the copy in this repo and creates commits on the remote repo that match the local +changes (so every local commit that touched the subrepo causes a commit on the remote repo). + +A `git subrepo pull src/tools/clippy` takes all changes since the last `subrepo pull` from the clippy +repo and creates a single commit in the rustc repo with all the changes. + +#### Creating a new subrepo dependency + +If you want to create a new subrepo dependency from an existing repository, call (from this +repository's root directory!!) + +``` +git subrepo clone https://github.com/rust-lang/rust-clippy.git src/tools/clippy +``` + +This will create a new commit, which you may not rebase under any circumstances! Delete the commit +and redo the operation if you need to rebase. + +Now you're done, the `src/tools/clippy` directory behaves as if clippy were part of the rustc +monorepo, so no one but you (or others that synchronize subrepos) needs to have `git subrepo` +installed. + + +### External Dependencies (submodules) Currently building Rust will also build the following external projects: @@ -221,7 +267,6 @@ before the PR is merged. Rust's build system builds a number of tools that make use of the internals of the compiler. This includes -[Clippy](https://github.com/rust-lang/rust-clippy), [RLS](https://github.com/rust-lang/rls) and [rustfmt](https://github.com/rust-lang/rustfmt). If these tools break because of your changes, you may run into a sort of "chicken and egg" From 56d0c81c750fa3cbe63f20bb30570cf084b9b98c Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Wed, 1 Apr 2020 14:53:10 +0200 Subject: [PATCH 2/7] s/subrepo/subtree/ --- CONTRIBUTING.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 69ce9dae5aa9b..fb1d43c87112c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -188,7 +188,7 @@ with one another are rolled up. Speaking of tests, Rust has a comprehensive test suite. More information about it can be found [here][rctd]. -### External Dependencies (subrepo) +### External Dependencies (subtree) As a developer to this repository, you don't have to treat the following external projects differently from other crates that are directly in this repo: @@ -198,39 +198,39 @@ differently from other crates that are directly in this repo: They are just regular files and directories. This is in contrast to `submodule` dependencies (see below for those). -If you want to synchronize or otherwise work with subrepos, install the `git subrepo` command via -instructions found at https://github.com/ingydotnet/git-subrepo +If you want to synchronize or otherwise work with subtrees, install the `git subtree` command via +instructions found at https://github.com/ingydotnet/git-subtree -#### Synchronizing a subrepo +#### Synchronizing a subtree -There are two synchronization directions: `subrepo push` and `subrepo pull`. Both operations create -a synchronization commit in the rustc repo. -This commit is very important in order to make future synchronizations work. -Do not rebase this commit under any circumstances. -Prefer to merge in case of conflicts or redo the operation if you really need to rebase. +There are two synchronization directions: `subtree push` and `subtree pull`. -A `git subrepo push src/tools/clippy` +A `git subtree push -P src/tools/clippy` takes all the changes that happened to the copy in this repo and creates commits on the remote repo that match the local -changes (so every local commit that touched the subrepo causes a commit on the remote repo). +changes (so every local commit that touched the subtree causes a commit on the remote repo). -A `git subrepo pull src/tools/clippy` takes all changes since the last `subrepo pull` from the clippy +A `git subtree pull -P src/tools/clippy` takes all changes since the last `subtree pull` from the clippy repo and creates a single commit in the rustc repo with all the changes. -#### Creating a new subrepo dependency +You always need to specifiy the `-P` prefix to the subtree directory. If you specify the wrong directory +you'll get very fun merges that try to push the wrong directory to the remote repository. Luckily you +can just abort this without any consequences. -If you want to create a new subrepo dependency from an existing repository, call (from this +#### Creating a new subtree dependency + +If you want to create a new subtree dependency from an existing repository, call (from this repository's root directory!!) ``` -git subrepo clone https://github.com/rust-lang/rust-clippy.git src/tools/clippy +git subtree add -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git master ``` This will create a new commit, which you may not rebase under any circumstances! Delete the commit and redo the operation if you need to rebase. Now you're done, the `src/tools/clippy` directory behaves as if clippy were part of the rustc -monorepo, so no one but you (or others that synchronize subrepos) needs to have `git subrepo` +monorepo, so no one but you (or others that synchronize subtrees) needs to have `git subtree` installed. From 521eb0df67bd2b822d7dbe5f4613a1e9beb98e1c Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Wed, 1 Apr 2020 16:18:25 +0200 Subject: [PATCH 3/7] Git subtree is upstream, no need to install --- CONTRIBUTING.md | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fb1d43c87112c..be31cc26da531 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -198,9 +198,6 @@ differently from other crates that are directly in this repo: They are just regular files and directories. This is in contrast to `submodule` dependencies (see below for those). -If you want to synchronize or otherwise work with subtrees, install the `git subtree` command via -instructions found at https://github.com/ingydotnet/git-subtree - #### Synchronizing a subtree There are two synchronization directions: `subtree push` and `subtree pull`. @@ -230,8 +227,7 @@ This will create a new commit, which you may not rebase under any circumstances! and redo the operation if you need to rebase. Now you're done, the `src/tools/clippy` directory behaves as if clippy were part of the rustc -monorepo, so no one but you (or others that synchronize subtrees) needs to have `git subtree` -installed. +monorepo, so no one but you (or others that synchronize subtrees) needs actually use `git subtree`. ### External Dependencies (submodules) From af553528ade187a0f85bb3fa9ffb22420c630238 Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Wed, 1 Apr 2020 17:33:20 +0200 Subject: [PATCH 4/7] Explain that you have to specify path and repository during subrepo synchronizations --- CONTRIBUTING.md | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index be31cc26da531..156ebdb0795c6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -196,23 +196,31 @@ differently from other crates that are directly in this repo: * none so far, see https://github.com/rust-lang/rust/issues/70651 for more info They are just regular files and directories. This is in contrast to `submodule` dependencies -(see below for those). +(see below for those). Only tool authors will actually use any operations here. #### Synchronizing a subtree There are two synchronization directions: `subtree push` and `subtree pull`. -A `git subtree push -P src/tools/clippy` +`git subtree push -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git` + takes all the changes that happened to the copy in this repo and creates commits on the remote repo that match the local changes (so every local commit that touched the subtree causes a commit on the remote repo). -A `git subtree pull -P src/tools/clippy` takes all changes since the last `subtree pull` from the clippy -repo and creates a single commit in the rustc repo with all the changes. - -You always need to specifiy the `-P` prefix to the subtree directory. If you specify the wrong directory -you'll get very fun merges that try to push the wrong directory to the remote repository. Luckily you -can just abort this without any consequences. +`git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git` +takes all changes since the last `subtree pull` from the clippy +repo and adds these commits to the rustc repo + a merge commit with the existing changes. +It is recommended that you always do a push before a pull, so that the merge works without conflicts. +While definitely possible to resolve conflicts during a pull, you may have to redo the conflict +resolution if your PR doesn't get merged fast enough and there are new conflicts. Do not try to +rebase the result of a `git subtree pull`, rebasing merge commits is a bad idea in general. + +You always need to specify the `-P` prefix to the subtree directory and the corresponding remote +repository. If you specify the wrong directory or repository +you'll get very fun merges that try to push the wrong directory to the wrong remote repository. +Luckily you can just abort this without any consequences and try again. It is usually fairly obvious +that this is happening because you suddenly get thousands of commits that want to be synchronized. #### Creating a new subtree dependency From 59cfb8035c387bcf7297c08b78dd099dec2dd7f4 Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Wed, 1 Apr 2020 17:45:47 +0200 Subject: [PATCH 5/7] Triple backticks --- CONTRIBUTING.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 156ebdb0795c6..486129ac4fada 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -202,13 +202,19 @@ They are just regular files and directories. This is in contrast to `submodule` There are two synchronization directions: `subtree push` and `subtree pull`. -`git subtree push -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git` +``` +git subtree push -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git +``` takes all the changes that happened to the copy in this repo and creates commits on the remote repo that match the local changes (so every local commit that touched the subtree causes a commit on the remote repo). -`git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git` +``` +git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git +``` + + takes all changes since the last `subtree pull` from the clippy repo and adds these commits to the rustc repo + a merge commit with the existing changes. It is recommended that you always do a push before a pull, so that the merge works without conflicts. From df91b40b43e6fb631f09a36d0d305585822eeb74 Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Thu, 2 Apr 2020 17:20:56 +0200 Subject: [PATCH 6/7] Address review comments --- CONTRIBUTING.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 486129ac4fada..fee16a1764da5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -203,21 +203,27 @@ They are just regular files and directories. This is in contrast to `submodule` There are two synchronization directions: `subtree push` and `subtree pull`. ``` -git subtree push -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git +git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy rustup ``` takes all the changes that happened to the copy in this repo and creates commits on the remote repo that match the local -changes (so every local commit that touched the subtree causes a commit on the remote repo). +changes. Every local commit that touched the subtree causes a commit on the remote repo, but is +modified to move the files from the specified directory to the tool repo root. + +Make sure to not pick the `master` branch, so you can open a normal PR to the tool to merge that +subrepo push. ``` -git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git +git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master ``` +takes all changes since the last `subtree pull` from the tool repo +repo and adds these commits to the rustc repo + a merge commit that moves the tool changes into +the specified directory in the rust repository. -takes all changes since the last `subtree pull` from the clippy -repo and adds these commits to the rustc repo + a merge commit with the existing changes. -It is recommended that you always do a push before a pull, so that the merge works without conflicts. +It is recommended that you always do a push first and get that merged to the tool master branch. +Then, when you do a pull, the merge works without conflicts. While definitely possible to resolve conflicts during a pull, you may have to redo the conflict resolution if your PR doesn't get merged fast enough and there are new conflicts. Do not try to rebase the result of a `git subtree pull`, rebasing merge commits is a bad idea in general. From 7928c45c062b330f4654faaeedac3722bb71ee3a Mon Sep 17 00:00:00 2001 From: Oliver Scherer Date: Thu, 2 Apr 2020 18:33:48 +0200 Subject: [PATCH 7/7] Address reviews --- CONTRIBUTING.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fee16a1764da5..051f5af7bc105 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -203,7 +203,7 @@ They are just regular files and directories. This is in contrast to `submodule` There are two synchronization directions: `subtree push` and `subtree pull`. ``` -git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy rustup +git subtree push -P src/tools/clippy git@github.com:your-github-name/rust-clippy sync-from-rust ``` takes all the changes that @@ -211,8 +211,8 @@ happened to the copy in this repo and creates commits on the remote repo that ma changes. Every local commit that touched the subtree causes a commit on the remote repo, but is modified to move the files from the specified directory to the tool repo root. -Make sure to not pick the `master` branch, so you can open a normal PR to the tool to merge that -subrepo push. +Make sure to not pick the `master` branch on the tool repo, so you can open a normal PR to the tool +to merge that subrepo push. ``` git subtree pull -P src/tools/clippy https://github.com/rust-lang/rust-clippy master @@ -224,20 +224,21 @@ the specified directory in the rust repository. It is recommended that you always do a push first and get that merged to the tool master branch. Then, when you do a pull, the merge works without conflicts. -While definitely possible to resolve conflicts during a pull, you may have to redo the conflict +While it's definitely possible to resolve conflicts during a pull, you may have to redo the conflict resolution if your PR doesn't get merged fast enough and there are new conflicts. Do not try to rebase the result of a `git subtree pull`, rebasing merge commits is a bad idea in general. You always need to specify the `-P` prefix to the subtree directory and the corresponding remote repository. If you specify the wrong directory or repository you'll get very fun merges that try to push the wrong directory to the wrong remote repository. -Luckily you can just abort this without any consequences and try again. It is usually fairly obvious +Luckily you can just abort this without any consequences by throwing away either the pulled commits +in rustc or the pushed branch on the remote and try again. It is usually fairly obvious that this is happening because you suddenly get thousands of commits that want to be synchronized. #### Creating a new subtree dependency If you want to create a new subtree dependency from an existing repository, call (from this -repository's root directory!!) +repository's root directory!) ``` git subtree add -P src/tools/clippy https://github.com/rust-lang/rust-clippy.git master @@ -247,7 +248,7 @@ This will create a new commit, which you may not rebase under any circumstances! and redo the operation if you need to rebase. Now you're done, the `src/tools/clippy` directory behaves as if clippy were part of the rustc -monorepo, so no one but you (or others that synchronize subtrees) needs actually use `git subtree`. +monorepo, so no one but you (or others that synchronize subtrees) actually needs to use `git subtree`. ### External Dependencies (submodules)