Grepping all the Rust code

[behnam]

(Bringing over the discussion here, as suggested: rust-lang/crates.io#841)

GitHub already allows searching in the code, but it's not always easy to grep in all the Rust code: If the search term is generic and matches projects in other languages better, "Rust" won't be listed in the sidebar and you'll have to edit the URL manually to get what you need.

Example: https://github.com/search?utf8=%E2%9C%93&q=hello&type=Code

I think somewhere there should be an easy-to-use grep UI for the Rust projects. Does docs.rs have anything planned? If not, maybe we can have a UI to make it easier to search in GitHub for now? What do you think?

[Boscop]

You can grep through all the Rust crates on github on sourcegraph, e.g.:
https://sourcegraph.com/search?q=repogroup:crates+%5Cbhead%28er%29%3F

But I find it not so useful for looking up symbol definitions.. (Too noisy results.)

@onur
I wish docs.rs would support global symbol searching (in all crates):
It only searches for matches in public symbols (the same that occur in generated docs) in all crates (and doesn't search in the bodies of functions or private symbols).
THAT would be very useful! Also for people who are new to Rust and are looking up symbols in code they are reading, when they aren't sure where the symbol is imported from.
But also for experienced Rust users (when they know the crate) for convenience of being able to configure a browser short key to search, e.g. I have r to search in Rust's std docs (so typing r Foo in my address bar will search for "Foo" in std's docs), and d for docs.rs, but then I always have to enter the name of the crate first, and then click on the first match, and only THEN can I enter my actual search term, it's not very efficient :/

The PureScript symbol search works in this global way:
E.g. https://pursuit.purescript.org/search?q=sum
or https://pursuit.purescript.org/search?q=style

I find it very useful to have this kind of search in PureScript and would really like to have it for Rust, too!

[jyn514]

I wish docs.rs would support global symbol searching (in all crates):

I don't know how feasible this is. We currently have 251882 releases of 43725 different crates. The search index alone would be several gigabytes large and take several hours for most browsers to load (since it's JavaScript).

As an alternative, there is a suggestion to search in all dependencies which seems more doable (and more helpful to the end user): #494

GitHub already allows searching in the code, but it's not always easy to grep in all the Rust code: If the search term is generic and matches projects in other languages better, "Rust" won't be listed in the sidebar and you'll have to edit the URL manually to get what you need.

I think this would be better as an extension to https://github.com/rust-dev-tools/cargo-src. Right now docs.rs doesn't have an easy way to access all pages, it has to download each HTML individually from S3. Doing that for all files (even once!) would be extremely expensive.

[jyn514]

I have r to search in Rust's std docs (so typing r Foo in my address bar will search for "Foo" in std's docs), and d for docs.rs, but then I always have to enter the name of the crate first, and then click on the first match, and only THEN can I enter my actual search term, it's not very efficient :/

You could set up your hotkeys to search d crate search_term and have it take you to https://docs.rs/crate?search=search_term.

[jyn514]

I'm going to close this, there are just too many crates for it to be feasible.

If you like, you could run offline searches following the instructions in https://www.pietroalbini.org/blog/downloading-crates-io/, that would be significantly faster since it doesn't have to go through JavaScript or download the crates more than once.