Make it easy for pub packages to ship binaries that can be added to my path easily

[sethladd]

When I install a gem from ruby today, and if it has binaries/scripts, they get added to my path. This is awesome. I'm sure I did a one-time setup but every other gem can supply util scripts that I can run easily.

I wanted to ship a simple util written in Dart, and it would be awesome if I could use pub to ship it. I would need pub packages to dump (or link or whatever) everything in bin/ to some well-known location that ends up on my PATH.

I searched and didn't find this issue, so here we are. Sorry if it's a dupe, I know we've talked about this before.

[munificent]

This is definitely on our radar but thanks for making it official.

---

Set owner to @munificent.
Added this to the Later milestone.

[munificent]

Issue #8281 has been merged into this issue.

---

cc @nex3.

[munificent]

Issue #11785 has been merged into this issue.

[munificent]

Issue #8815 has been merged into this issue.

[DartBot]

This comment was originally written by dangling.fe...@gmail.com

---

This example project demonstrates how this can be implemented in package manager.

http://pub.dartlang.org/packages/package_installer

The automatic execution of the process of installation of packages that required for their proper operation before they begin to be used. These processes of installation may include the following tasks: compilation of the binary files, installing the third party software and so on.

And this example demonstrates how to avoid requirements to include the binary files for all operating systems in package.

https://github.com/mezoni/demo_native_extension_usage

As the result of work of this example package that requirey compilation of binary files will be compiled and shared (it never be re-compiled when re-installing).

Bonus.
Not requires to writing batch files for each operating systems. The command batch files for compilation tasks can be replaced by single file written in Dart language (very convenient configuration).

http://pub.dartlang.org/packages/ccompilers

A set of classes for easy access to the system С/С++ compilers from the Dart language scripts.

This tools is a library written in Dart programming language. It is intended to compile C/C++ source files from Dart scripts.

[nex3]

Issue #5473 has been merged into this issue.

[DartBot]

This comment was originally written by nerdre...@gmail.com

---

Is anyone working on this? Is there a plan for how this should be implemented? Is it on the roadmap?

[DartBot]

This comment was originally written by yulian...@yuvaks.com

---

This is one of the biggest features preventing me from writing half the things i want in Dart. I write them all in Node.js because node has simple package.json entries to create binaries from you main file, or provide paths to custom ones. or just wrappers scripts. The key is there is a global bin that lets you install it as an app.

This is a very important thing that many have asked for, and still being ignored. It can't be that hard to add.

[DartBot]

This comment was originally written by nerdr...@gmail.com

---

@seth: Funny enough, my name is also Andrew, and I thought your comment was
directed at me. I was thinking of working on a PR (or at least exploring
how difficult it is), but I wanted to make sure I wasn't wasting my time
before starting. Are there thoughts on how this should be implemented / the
pubspec.yaml changes?

Here's how npm does it: (quoting from
https://www.npmjs.org/doc/files/package.json.html)

---

To use this, supply a bin field in your package.json which is a map of
command name to local file name.
On install, npm will symlink that file into prefix/bin for global installs,
or ./node_modules/.bin/ for local installs.

For example, npm has this:

{ "bin" : { "npm" : "./cli.js" } }
So, when you install npm, it'll create a symlink from the cli.js script to
/usr/local/bin/npm.

---

And Rubygems: (quoting from
http://guides.rubygems.org/specification-reference/#executables)

---

Executables included in the gem.

For example, the rake gem has rake as an executable. You don’t specify the
full path (as in bin/rake);
all application-style files are expected to be found in bindir. These files
must be executable Ruby files.
Files that use bash or other interpreters will not work.

Usage:

spec.executables << 'rake'

[sethladd]

Nerdrew, sorry for the confusion. I don't know if there's a design document for this feature yet. Sounds like you've just started one in your comment in #­14. :) Maybe start a Google Doc and share it with the community? We could ask the pub engineers to take a look at your doc to see if we're on the right track, as they probably have some context and thoughts.

[munificent]

Are there thoughts on how this should be implemented / the
pubspec.yaml changes?

Yes, we've given it a bunch of thought but haven't had any time to work on it.

The pubspec part of it is actually easy: there's probably nothing to do there. Since we already have guidelines saying that command-line apps go in bin/, we can just assume all of the binaries there are the ones that should be made available.

The challenge is that there are a bunch of other issues to deal with. Say you have a package "foo" that has a single file bin/foo.dart that you want to make available. Here's a non-exhaustive list of the stuff that comes to mind.

* If you run a command from within a package that depends on "foo", it should use the version of "foo" that that package depends on.

• It needs to handle that package depending on foo but not having foo in its lockfile.
• It needs to handle the package depending on foo, having a lockfile, but not having installed foo already.
• If you run the command outside of a package (or within a package that doesn't depend on foo), we need to determine which version of "foo" to use.
• That probably means we need some idea of a "global" or "active" foo version, and we'll need a user experience to let the user configure that.
• We need to make sure it works with things like path dependencies where "foo" may not be in the user's system cache.
• Since "foo" just has a .dart file, we'd like to wrap that in something so that the user doesn't need to type ".dart". That means we need to locate the Dart executable. What's the best way to do that?
• How do we handle "package:" imports in foo? We currently rely on symlinks for that, but we're trying to move away from that.
• How do we ensure foo's transformers, and those of its dependencies get run when foo starts up?
• If foo wants to load data files from within its own package, how does it locate them? We don't have anything like PATH or FILE in Dart.
• We have to make sure it works in Linux, Mac, and Windows.
• We have thorough integration tests for pub, so all of this will also need detailed tests.

I'm probably forgetting a bunch of stuff.

We'd love to have more external contributions to pub (and I can probably point you to a few bugs if you'd like), but there's a reason we haven't started working on this. It's quite complex, and it depends on a bunch of other stuff we are working on right now like transformers.

I can tell you what our rough plan is:

1. We'll add a "pub run" command that invokes a named binary from a package. It can handle selecting the right version, running the transformers, etc. Similar to bundle exec.
2. To let users just type command names without having to type "pub run" everytime, we can consider creating little shell wrappers that forward to that, similar to binstubs.

I don't mean to spook you here. I think it's awesome that you'd like to help out, and I can definitely help you get up to speed! But this is one of the most difficult corners of pub for someone (internal or external) to take on.

[nex3]

Some additional thoughts:

That probably means we need some idea of a "global" or "active" foo version, and we'll need a user experience to let the user configure that.

I think this can come later; defaulting to the most recent version is probably enough for a first take on this.

Since "foo" just has a .dart file, we'd like to wrap that in something so that the user doesn't need to type ".dart". That means we need to locate the Dart executable. What's the best way to do that?

Pub's sdk.dart can tell us this.

We also don't want to require root privileges to install executables. This means we'll need to have some way of explaining to users on each platform how to add a directory within their .pub-cache to their PATH.

[DartBot]

This comment was originally written by yulian...@yuvaks.com

---

We also don't want to require root privileges to install executables. This means we'll need to have some way of explaining to users on each platform how to add a directory within their .pub-cache to their PATH.

Not necessarily. On OSX, /usr/local/bin seems to be open for writes without sudo. I know Homebrew uses it. Unless i missed a detail.

Also if I remember right, Linux has the same thing, since we use it here at work. I think its even the same location.

It is usually already part of the default path.

For Windows, setx does not need admin rights to set the path. so you can do "setx path "%path%:C:\Some\Path\To\Pub\bin"

[DartBot]

This comment was originally written by yulian...@yuvaks.com

---

With my last comment, I am trying to say it can be done by pub or assumed in default configurations.

[nex3]

On my Linux box, /usr/local/bin is not user-writable. I don't think it's safe to assume otherwise, nor is it safe to overwrite other executables with conflicting names.