Consider splitting MSCs and spec into separate github repositories

[richvdh]

Combining MSCs and finalised spec docs into a single repo brings several problems:

• Pull requests have a very different lifecycle. It is hard to keep on top of non-MSC PRs, which generally get merged much quicker than MSC PRs. (It is possible to use labels to do some filtering, but I still think it means that spec PRs tend to get forgotten)
• Having 291 open PRs is, rightly or wrongly, seen as a terrible smell for a project of this size. At least we could compartmentalise it into a single "proposals" repository.
• I generally think it's a bit confusing for new contributors.
• As a spec author, it's often necessary to have two checkouts for matrix-doc: one for MSCs and one for spec writing: the tooling conflicts with MSC branches.

I think my strategy to fix it would be:

• close as many non-MSC matrix-doc PRs as possible (https://github.com/matrix-org/matrix-doc/pulls?q=is%3Aopen+is%3Apr+-label%3Aproposal)
• create a new copy of the repo at matrix-org/matrix-spec
• rename matrix-doc to matrix-spec-proposals (thus retaining HTTP links to existing MSCs)
• remove the irrelevant bits from each repo
• recreate any remaining non-MSC matrix-doc PRs against matrix-spec
• transfer all of the issues to matrix-spec. Process for this involves https://github.com/github/hub:

  # get a list of issues
  hub issue -f '%I%n' > issues.txt
  # transfer them all
  while read issue; do
      hub issue transfer $issue `matrix-org/matrix-spec`
  done < issues.txt
  

[richvdh]

close as many non-MSC matrix-doc PRs as possible (https://github.com/matrix-org/matrix-doc/pulls?q=is%3Aopen+is%3Apr+-label%3Aproposal)

there's only a handful of these, so it might be quite a good time to do this.

[turt2live]

@matrix-org/spec-core-team if you have thoughts or reason to believe we shouldn't do this, please raise them imminently.

[richvdh]

I'm going to begin this now.

[richvdh]

This is now largely done, and https://github.com/matrix-org/matrix-spec-proposals and https://github.com/matrix-org/matrix-spec exist.

I've got a script which is going through and transferring the issues to https://github.com/matrix-org/matrix-spec, however the github API seems to be somewhat rate-limited, so it may be a matter of days rather than hours before it completes.

Other follow-up tasks:

• update links to the old repository; most of them will now want to point to https://github.com/matrix-org/matrix-spec rather than be 302ed to https://github.com/matrix-org/matrix-spec-proposals.
• Update documentation for creating proposals - make sure it points to https://github.com/matrix-org/matrix-spec-proposals
• Give https://github.com/matrix-org/matrix-spec-proposals a README (README and CONTRIBUTING matrix-spec-proposals#3776)
• Check that the autodeploy webhook for the spec works correctly
• Figure out if there are any other webhooks which need transferring to the new repo
• Transfer the netlify secret to matrix-spec, so that the preview builds work
• Fix @mscbot so that it correctly manages FCPs (cf MSC3316: Add timestamp massaging to the spec matrix-spec-proposals#3316)
• Update descriptions for issue labels on https://github.com/matrix-org/matrix-spec

[turt2live]

to add:

• Convert old issue-based MSCs to PRs
• Copy comments from old split MSCs to the PR counterpart (there are a handful of MSCs which used a process where the MSC number was the issue number and the PR would be referenced by that issue - the MSC was the issue, not the PR. We should renumber these to make the MSC the PR instead)
• Turn off issue creation for the proposals repo (use PRs for everything)
  • but reopened, because turning issues off made all historical issues into 404s.
• Verify the MSC bots work and are updated

[anoadragon453]

• Update the text on https://spec.matrix.org/unstable/proposals/ to point to the new repositories.
• Modify the script that updates https://spec.matrix.org/unstable/proposals/ to point to matrix-spec-proposals (line here).