Can't depend on a proto in the same Bazel package but with a different proto package name

[jf647]

I'm having trouble trying to depend on a proto in a subdirectory of my bazel package that has a different proto package name.

I've put a minimal broken example here:

https://github.com/jf647/stackb_example

I can build everything in subdirectory foo without issue, as it depends on no other protos:

elocie ➜  stackb_example git:(master) bazel build foo/...
INFO: Invocation ID: 8baafcac-4132-421b-b493-9b1967bcfa71
INFO: Analysed 3 targets (0 packages loaded, 0 targets configured).
INFO: Found 3 targets...
INFO: Elapsed time: 0.324s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
elocie ➜  stackb_example git:(master) ls -la bazel-stackb_example//bazel-out/darwin-fastbuild/genfiles/foo/foo_go_proto_pb/foo/foo.pb.go
-r-xr-xr-x  1 james  wheel  2377 Jan 28 08:57 bazel-stackb_example//bazel-out/darwin-fastbuild/genfiles/foo/foo_go_proto_pb/foo/foo.pb.go

But if I try to build everything in bar (which depends on foo), I get errors from protoc about inconsistent import paths:

elocie ➜  stackb_example git:(master) bazel build bar/...
INFO: Invocation ID: 42247b9b-02cb-4ce2-b7d5-dbb5a87963ce
INFO: Analysed 3 targets (1 packages loaded, 4 targets configured).
INFO: Found 3 targets...
ERROR: /Users/james/projects/stackb_example/bar/BUILD.bazel:23:1: error executing shell command: '/bin/bash -c bazel-out/host/bin/external/com_google_protobuf/protoc --descriptor_set_out=bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb/descriptor.source.bin --proto_path=bazel-out/darwin-...' failed (Exit 1) bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
2019/01/28 08:58:51 protoc-gen-gogo: error:inconsistent package import paths: "bar", "foo"
--gogo_out: protoc-gen-gogo: Plugin failed with status code 1.
INFO: Elapsed time: 0.475s, Critical Path: 0.15s
INFO: 3 processes: 3 darwin-sandbox.
FAILED: Build did NOT complete successfully

Using --sandbox_debug, it appears that protoc is being invoked on all of the protos simultaneously:

elocie ➜  stackb_example git:(master) bazel build bar/... --sandbox_debug
INFO: Invocation ID: f5921fcd-eaf3-4d33-9ae8-85fe73836d7a
INFO: Analysed 3 targets (0 packages loaded, 0 targets configured).
INFO: Found 3 targets...
ERROR: /Users/james/projects/stackb_example/bar/BUILD.bazel:23:1: error executing shell command: '/bin/bash -c bazel-out/host/bin/external/com_google_protobuf/protoc --descriptor_set_out=bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb/descriptor.source.bin --proto_path=bazel-out/darwin-...' failed (Exit 1) sandbox-exec failed: error executing command
  (cd /private/var/tmp/_bazel_james/adc497d6faec319414998cd6b215e16f/execroot/__main__ && \
  exec env - \
    TMPDIR=/var/folders/gm/kk0_399j7_7cdd6qh1j_xsbr0000gn/T/ \
  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_james/adc497d6faec319414998cd6b215e16f/sandbox/darwin-sandbox/1/sandbox.sb /var/tmp/_bazel_james/install/b9adaa5c006b96c0300b56da1ec2ef08/_embedded_binaries/process-wrapper '--timeout=0' '--kill_delay=15' /bin/bash -c 'bazel-out/host/bin/external/com_google_protobuf/protoc --descriptor_set_out=bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb/descriptor.source.bin --proto_path=bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb --include_imports --include_source_info --gogo_out=Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb --plugin=protoc-gen-gogo=bazel-out/host/bin/external/com_github_gogo_protobuf/protoc-gen-gogo/darwin_amd64_stripped/protoc-gen-gogo bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb/bar/bar.proto bazel-out/darwin-fastbuild/genfiles/bar/bar_go_proto_pb/foo/foo.proto')
2019/01/28 08:59:11 protoc-gen-gogo: error:inconsistent package import paths: "bar", "foo"
--gogo_out: protoc-gen-gogo: Plugin failed with status code 1.
INFO: Elapsed time: 0.339s, Critical Path: 0.08s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

(a little hard to read but you can see we have one protoc-gen-go invocation with two args - bar.proto and foo.proto)

This appears not to be supported (see golang/protobuf#39 for a rather drawn-out discussion). It looks like a rewrite to protoc-gen-go might be in the works, but for now I'm wondering if it might be easier to just process each proto_library on its own.

[schoren]

I'm having the same issue. Is there a fix or workaround?

[pcj]

A few issues here:

1. The default behavior of the rules is to compile all protos at once. This has the benefit of being able to generate artifacts for the entire transitive proto dependency tree. It has disadvantages too, including the one(s) detailed here. This can be controlled by the transitive = False rule attribute. It's entirely possible that transitive = True is a misguided default value and that the default should be False. Note that the gogo rules were not propagating this attribute which is now fixed by Propagate transitive and verbose flags in gogo_* rules #50.

2. One can do importmapping via the importmap dict attribute (not exactly well-documented). Here's the diff on your example repo that I used to get this to compile:

diff --git a/WORKSPACE b/WORKSPACE
index 456e1bf..6616ebf 100644
--- a/WORKSPACE
+++ b/WORKSPACE
@@ -16,9 +16,9 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 
 http_archive(
     name = "build_stack_rules_proto",
-    sha256 = "36f11f56f6eb48a81eb6850f4fb6c3b4680e3fc2d3ceb9240430e28d32c47009",
-    strip_prefix = "rules_proto-d86ca6bc56b1589677ec59abfa0bed784d6b7767",
-    urls = ["https://github.com/stackb/rules_proto/archive/d86ca6bc56b1589677ec59abfa0bed784d6b7767.tar.gz"],
+    urls = ["https://github.com/stackb/rules_proto/archive/91cbae9bd71a9c51406014b8b3c931652fb6e660.tar.gz"],
+    sha256 = "5474d1b83e24ec1a6db371033a27ff7aff412f2b23abba86fedd902330b61ee6",
+    strip_prefix = "rules_proto-91cbae9bd71a9c51406014b8b3c931652fb6e660",
 )
 
 load("@build_stack_rules_proto//github.com/gogo/protobuf:deps.bzl", "gogo_proto_compile", "gogo_proto_library")
diff --git a/bar/BUILD.bazel b/bar/BUILD.bazel
index 6d59081..40b833b 100644
--- a/bar/BUILD.bazel
+++ b/bar/BUILD.bazel
@@ -24,9 +24,14 @@ gogo_proto_library(
     name = "bar_go_proto",
     go_deps = [
         "@com_github_gogo_protobuf//gogoproto:go_default_library",
+        "//foo:foo_go_proto",
     ],
     importpath = "github.com/jf647/stackb_example/bar",
+    importmap = {
+        "foo/foo.proto": "github.com/jf647/stackb_example/foo",
+    },
     deps = [
         ":bar_proto",
     ],
+    transitive = False,
 )
diff --git a/foo/BUILD.bazel b/foo/BUILD.bazel
index fae9e0d..317b971 100644
--- a/foo/BUILD.bazel
+++ b/foo/BUILD.bazel
@@ -29,4 +29,6 @@ gogo_proto_library(
     deps = [
         ":foo_proto",
     ],
+    visibility = ["//bar:__pkg__"],
+    verbose = 3,
 )