Make commands in dev-dependencies available to run

[casey]

I have a rust project which uses a few shell scripts that need commands from other crates. I added them as dev-dependencies, but I couldn't find the binaries anywhere. It would be nice if the binaries were available to run from the command line.

As an example, npm makes binaries available in node_modules/.bin, so you can put them in devDependencies and use them in scripts.

[alexcrichton]

This sounds like it may be more of a use case for cargo install? We typically don't have binaries from any dependencies available to the main application

[casey]

The issues that I can see with cargo install for this use case are:

1. No record of dependencies. Let's say your project needs various binary crates for development, for example because they're called by non-rust glue and utility scripts. This creates the need to record somewhere that you'll need to cargo install xyz, which complicates getting started on a project, or starting work on it on a new machine.
2. No versioning. Although you can use cargo install --vers, dependency on a specific version of a crate isn't recorded.
3. Global install by default. Since cargo install installs globally by default, it isn't possible out of the box to keep separate installations of tools for different projects, in case they depend on different versions of the same binary crate. It's possible to get around this by doing cargo install --root or using the install.root configuration key and specifying a local location, but I suspect that this is a common enough use-case to warrant special handling.

My current motivation for raising this issue is that I'm developing bindings for a C-library using github.com/crabtw/rust-bindgen. rust-bindgen has a library crate which provides a rust api, as well as a binary crate which provides a command line API. I need to use the binary crate because the library API won't work for my use case. So, rust-bindgen needs to be installed and available as a runnable binary for doing routine development, and it would be nice to formally record that somewhere, and have a simple 'cargo update' or 'cargo install' that would fetch crates with the commands that I need.

[alexcrichton]

For that use case, wouldn't a build script suffice? In theory rust-bindgen would also have a library API and then your build script would take care of what while building.

[casey]

Rust-bindgen does indeed have a library API, but it is not exactly the same as the CLI. Not to get into too much detail, but the library API is a macro that generates bindings at compile time, and the CLI writes the bindings to a .rs file. I'm wrapping a library with safe rust bindings, so writing the bindings explictly to a .rs file where I'll see in git if they've changed and can review the changes before committing them is preferable.

I suspect that many crates are like this. The library api and the CLI are slightly different use cases, and so will be designed slightly differently and be good for different things. Or, the library will be a command-line oriented tool with no library API, but still used in the development of a particular rust project.

[casey]

Unless I'm mistaken, this was accidentally closed.

[alexcrichton]

Oh dear I did indeed fat finger that, sorry!

[jansegre]

I have a use case that also requires commands available from dev-dependencies.

My project has to run arbitrary user script/commands (sort of like codingame), those scripts are run as a child process by the library, which pipes and interact with stdin, stdout and stderr of the child. I want to test that, and the most sensible way I thought would be having in-project dev-dependencies with executables to act as user scripts. But then I found out that cargo does not compile dev-depenencies commands.

Although somewhat different than the original issue this also involves having binaries from dev-dependencies compiled. I'm open to alternatives, my strongest motivation for having this done in cargo is the continuous integration testing of the child process piping functionality on multiple platforms.

Is there a strong reasoning to oppose binaries in dependencies in general and in dev-dependencies in particular?

Edit: fiddle a bit and found #1581, which could also be used to compile my test user scripts.

[alexcrichton]

There's not really any particularly strong reason to not do this, it basically just needs a principled design. For example we can't really "just compile" all dev-dependency binaries and put them somewhere. Some concerns I have with that are:

• What if I don't actually need all those binaries? Will this just clutter up build times?
• When are those binaries actually compiled? Does cargo build do that? cargo test?
• How do we resolve binary name conflicts between dependencies?

In general build scripts tend to solve most of these problems naturally, so I personally like to push on those as hard as possible, but I can definitely see that for CLI tools this may fall down from time to time (but cargo install can often fill in)

[jansegre]

In my case I miss the ability to compile binaries for use inside tests, not running commands from dev-dependencies outside tests.

The more I think about it dev-build scripts seem more reasonable as in the general case it could be used to generate anything for tests and benches.

For now I'll resort to a Makefile + PS script (windows) for compiling a binary and making it available to tests via some environment variable. But I'm willing to write my first RFC for #1581.