Skip to content

scalapb_proto_library does not support well known protobufs #312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rynbrd opened this issue Oct 18, 2017 · 4 comments
Closed

scalapb_proto_library does not support well known protobufs #312

rynbrd opened this issue Oct 18, 2017 · 4 comments

Comments

@rynbrd
Copy link
Contributor

rynbrd commented Oct 18, 2017

So I've got this build file:

load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scalapb_proto_library")

proto_library(
    name = "scalapb_proto",
    srcs = ["scalapb.proto"],
    visibility = ["//visibility:public"],
    deps = ["@com_google_protobuf//:descriptor_proto"],
)

java_proto_library(
    name = "scalapb_java_library",
    deps = [":scalapb_proto"],
)

scalapb_proto_library(
    name = "scalapb_scala_library",
    deps = [":scalapb_proto"],
    with_flat_package = True,
)

Building it results in the following output:

INFO: Found 1 target...
INFO: From scala @io_bazel_rules_scala//src/scala/scripts:scalapb_generator_lib [for host]:
warning: there were two deprecation warnings; re-run with -deprecation for details
one warning found
INFO: From creating scalapb files //protocols/scalapb:scalapb_scala_library_srcjar:
protoc-jar: protoc version: 320, detected platform: linux/amd64
protoc-jar: executing: [/tmp/protocjar7612841753549369523/bin/protoc.exe, --plugin=protoc-gen-scala=/tmp/protocbridge137921835608203876, --scala_out=flat_package:/tmp/bazelscalapb6289116713956964561, -Iprotocols/scalapb/scalapb.proto=protocols/scalapb/scalapb.proto, -Ifiles/external/com_google_protobuf/google/protobuf/descriptor.proto=external/com_google_protobuf/files/external/com_google_protobuf/google/protobuf/descriptor.proto, protocols/scalapb/scalapb.proto, external/com_google_protobuf/files/external/com_google_protobuf/google/protobuf/descriptor.proto]
external/com_google_protobuf/files/external/com_google_protobuf/google/protobuf/descriptor.proto: warning: directory does not exist.
external/com_google_protobuf/files/external/com_google_protobuf/google/protobuf/descriptor.proto: No such file or directory
ERROR: /home/sadpengu/Work/src/github.com/zenreach/platypus/protocols/scalapb/BUILD:32:1: scala //protocols/scalapb:scalapb_scala_library failed (Exit 1).
java.lang.RuntimeException: Must have input files from either source jars or local files.
	at io.bazel.rulesscala.scalac.ScalacProcessor.processRequest(ScalacProcessor.java:60)
	at io.bazel.rulesscala.worker.GenericWorker.run(GenericWorker.java:125)
	at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:41)
Target //protocols/scalapb:scalapb_scala_library failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 8.582s, Critical Path: 6.45s

Using with_java and setting the dependency to the scalapb_java_library results in a more cryptic error:

INFO: Found 1 target...
ERROR: /home/sadpengu/Work/src/github.com/zenreach/platypus/protocols/scalapb/BUILD:32:1: creating scalapb files //protocols/scalapb:scalapb_scala_library_srcjar failed (Exit 1).
java.lang.ArrayIndexOutOfBoundsException: 1
	at scripts.ScalaPBGenerator$$anonfun$1.apply(ScalaPBGenerator.scala:46)
	at scripts.ScalaPBGenerator$$anonfun$1.apply(ScalaPBGenerator.scala:42)
	at scala.collection.immutable.List.map(List.scala:284)
	at scripts.ScalaPBGenerator.processRequest(ScalaPBGenerator.scala:42)
	at io.bazel.rulesscala.worker.GenericWorker.run(GenericWorker.java:125)
	at scripts.ScalaPBWorker$.main(ScalaPBGenerator.scala:24)
	at scripts.ScalaPBWorker.main(ScalaPBGenerator.scala)
Target //protocols/scalapb:scalapb_scala_library failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 186.994s, Critical Path: 10.78s

Based on the description of #281 I assume this is due to these files (technically) being remote.

Keeping a local copy of all the well-known protos (and writing a stack of BUILD files) seems like an unnecessary step since Bazel has to download this to get protobuf support at all. Is there a more elegant work-around? Or am I just doing something wrong?
@rynbrd
Copy link
Contributor Author

rynbrd commented Oct 18, 2017

I've managed to solve the problem. Ultimately it came down to using HEAD of https://github.com/google/protobuf instead of v3.4.1. The well known proto changes did not make it into that release.

@rynbrd rynbrd closed this as completed Oct 19, 2017
@rynbrd
Copy link
Contributor Author

rynbrd commented Oct 19, 2017

I take it back. I'm still struggling with this. Given this WORKSPACE file:

workspace(
    name = "com_github_zenreach_platypus_protocols",
)

protobuf_version = "09e0dbcf8f8a078070921e18b0592dbe7f9628df"
protobuf_sha = "3e037866c379c902d3d3c26ef930a3d4dc230773f830de853a35ddf3d9012518"

rules_scala_version = "fda0f4b1d68ad371481b7f5ad1592d4323497d8e"

http_archive(
    name = "com_google_protobuf",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "com_google_protobuf_cc",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "com_google_protobuf_java",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "io_bazel_rules_go",
    url = "https://github.com/bazelbuild/rules_go/releases/download/0.6.0/rules_go-0.6.0.tar.gz",
    sha256 = "ba6feabc94a5d205013e70792accb6cce989169476668fbaf98ea9b342e13b59",
)

http_archive(
	name = "io_bazel_rules_scala",
	url = "https://github.com/bazelbuild/rules_scala/archive/%s.tar.gz" % rules_scala_version,
	strip_prefix = "rules_scala-%s" % rules_scala_version
)

load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains")
load("@io_bazel_rules_go//proto:def.bzl", "proto_register_toolchains")
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scala_proto_repositories")

proto_register_toolchains()
go_rules_dependencies()
go_register_toolchains()
scala_repositories()
scala_proto_repositories()

And this scalapb/BUILD file:

load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scalapb_proto_library")

proto_library(
    name = "scalapb_proto",
    srcs = ["scalapb.proto"],
    deps = [
        "@com_google_protobuf//:descriptor_proto",
    ],
    visibility = ["//visibility:public"],
)

java_proto_library(
    name = "scalapb_java_proto",
    deps = [":scalapb_proto"],
    visibility = ["//visibility:public"],
)

scalapb_proto_library(
    name = "scalapb_scala_proto",
    deps = [":scalapb_proto"],
    #deps = [":scalapb_java_proto"],
    #with_java = True,
    with_flat_package = True,
    visibility = ["//visibility:public"],
)

Building scalapb_proto and scalapb_java_proto succeed. However, building scalapb_scala_proto fails with the following output:

$ bazel build scalapb:scalapb_scala_proto
INFO: Found 1 target...
ERROR: /home/sadpengu/Work/src/github.com/zenreach/platypus/protocols/scalapb/BUILD:18:1: scala //scalapb:scalapb_scala_proto failed (Exit 1).
java.lang.RuntimeException: Must have input files from either source jars or local files.
	at io.bazel.rulesscala.scalac.ScalacProcessor.processRequest(ScalacProcessor.java:60)
	at io.bazel.rulesscala.worker.GenericWorker.run(GenericWorker.java:125)
	at io.bazel.rulesscala.scalac.ScalaCInvoker.main(ScalaCInvoker.java:41)
Target //scalapb:scalapb_scala_proto failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.322s, Critical Path: 0.19s

I'm using the latest https://github.com/google/protobuf and https://github.com/bazelbuild/rules_scala but that doesn't seem to make a difference. I don't know what I did to break things since yesterday but here we are.

@ittaiz
Copy link
Contributor

ittaiz commented Oct 19, 2017 via email

@rynbrd rynbrd reopened this Oct 19, 2017
@rynbrd
Copy link
Contributor Author

rynbrd commented Oct 19, 2017 •
edited
Loading

No I meant to re-open it actually.

I've worked around the problem by copying the well-known protos from https://github.com/google/protobuf into our repo and adding proto_library rules for them. That works in the mean time but I don't feel as if it should be necessary.

@rynbrd rynbrd mentioned this issue Oct 23, 2017
Merged
@beala-stripe beala-stripe mentioned this issue Feb 8, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rynbrd @ittaiz