rustc_src is optional

[illicitonion]

This is a tar containing a huge number of files (hundreds of thousands)
which can take double-digit seconds to extract.

It's only used for rust analyzer support, so hide this behind a flag
people can opt in to if they want.

[UebelAndre]

Perhaps rust_analyzer should have it's own toolchain which goes and fetches the rustc sources? That might make configuring it a bit clunky but might be warranted and provide a good way to separate the rust_analyzer functionality from core builds. Maybe there's a path forward where rust analyzer is enabled by a bazelrc flag and we'd always define the toolchains in repositories.bzl knowing they'd otherwise never be used without the aspect enabled.

I do feel the amount of time spent fetching dependencies is quite painful though so this is a good direction.

[illicitonion]

I finally got around to addressing these comments - PTAL!

[UebelAndre]

What is the user story for having this work with rust-analyzer? I think if we expect users to set some flag to True in their workspace that we should also update documentation for that here at the very least.

[illicitonion]

I added a note to the rust_analyzer docs to tell users they need to set this.

[UebelAndre]

I was actually thinking something should be added to docs/rust_analyzer.vm. If I'm going to setup rust_analyzer in my existing rules_rust project, I may not see the updated note about rust_repositories. If you add some documentation there I'm happy with this PR 😄

[illicitonion]

Sigh, I edited exactly what you wanted, but I edited the .md file rather than the .vm file so it got reverted by the re-generation. I am looking forward to not putting generated docs in the repo.

Updated the right file this time :)

[UebelAndre]

Looks good to me! Thanks!

[UebelAndre]

Having thought about this a bit more. Could this also be gated by an action_env or something a user could add to their .bazelrc files? Basically, I'm never going to set include_rustc_src = False if I use rust_analyzer in development or I'm gonna awkwardly have a modified WORKSPACE file or something. Maybe --repo_env would work here?

Or perhaps do something for the inverse where a .bazelrc is set for CI to disable fetching these?

[illicitonion]

Having thought about this a bit more. Could this also be gated by an action_env or something a user could add to their .bazelrc files? Basically, I'm never going to set include_rustc_src = False if I use rust_analyzer in development or I'm gonna awkwardly have a modified WORKSPACE file or something. Maybe --repo_env would work here?

Or perhaps do something for the inverse where a .bazelrc is set for CI to disable fetching these?

I put together an alternate patch where a repo env determines whether to do the fetching, and we set the repo env, but it has the down-side that the repository rule is re-run whenever you switch between rust analyzer and non-rust-analyzer bazel usage, which is a pretty bad experience.

I guess a reasonable alternative would be a WORKSPACE file opt-in (as this PR does), and then an env var to specify "No really, I'm never going to want to use rust analyzer" that you could set on CI... But that's starting to feel like a somewhat confusing state-space to need to consider, as a user...

I think @hlopko was talking about looking into why we unconditionally re-fetch toolchain stuff quite so often, which would maybe make my alternate patch more viable.

In the mean time, I'm going to merge this (as discussed offline), we can re-visit when we have a better plan :)