using Classpath shrinker to warn on indirect targets #1

nadavwe · 2017-06-28T15:39:25Z

this is written in top of bazel-contrib#227.
@natansil - we should probably rebase on master.
this still fails - we have something unexpected regarding the output targets, need to investigate.

@ittaiz @or-shachar

The key change is that the scala provider has been completely restructured to match the structure of the JavaProvider. It no longer tracks exports separately but instead has the following fields: * outputs = contains all the outputs generated by the current rule (nothing or ijar and class_jar); however, no rules here actually use it. * compile_jars = contains all the jars dependent rules should compile with. Thus, ijar and exports.compile_jars * transitive_runtime_jars = contains all the jars needed to handle this target at runtime. Thus, class_jar plus (deps + exports + runtime_deps).transitive_runtime_jars The created java provider (used by dependent native java rules) uses just compile_jars and transitive_runtime_jars. In general, this change was seamless. The major exception is if you were specifically relying on only exports being re-exported by specifically gen_scrooge_srcjar. Beforehand, any jars produced by direct dependencies were not being exported to dependents but now they are. Note that, if you were using scrooge_scala_library instead, nothing should change. Other changes: * Use depset instead of set. (set docs already say set should be considered deprecated and just points to depset anyway) * Tests amended to exploit that JavaProvider is properly constructed. Note that things are still a little strange until Bazel releases incorporate some fixes to JavaProvider and native provider rules. Generally, only deps for java_library and java_binary work at the moment.

Remove scala_exports_to_java Prefer using JavaProvider over scala provider (can probably avoid using scala provider completely?)

nadavwe · 2017-06-28T15:43:24Z

relates to bazel-contrib#235

natansil · 2017-06-29T09:17:08Z

@nadavwe , there were two bugs related to string encoding/escaping.
first, we encoded the jars and targets with ":" instead of "," to the temp file read by the worker.
second, specifically for targets, they contain ":", themselves, so they needed to be escaped...

both are fixed now

ittaiz · 2017-06-29T09:21:06Z

1) would you like me to review? 2) are you copying the code in? If not should I review it in the other repo?

…

On Thu, 29 Jun 2017 at 12:17 Natan Silnitsky ***@***.***> wrote: @nadavwe <https://github.com/nadavwe> , there were two bugs related to string encoding/escaping. first, we encoded the jars and targets with ":" instead of "," to the temp file read by the worker. second, specifically for targets, they contain ":", themselves, so they needed to be escaped... both are fixed now — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIFy1WeNYamqJHiqDZ7e35zg--o-3Kks5sI2uVgaJpZM4OIJnJ> .

nadavwe · 2017-06-29T09:35:02Z

@ittaiz

please do - i think this is a bit too POC code yet - it should have major beautification refactoring.
please review in the other repository anyway and we can decide on copying later.

ittaiz · 2017-06-29T09:43:38Z

Will do

…

On Thu, 29 Jun 2017 at 12:35 Nadav Wexler ***@***.***> wrote: @ittaiz <https://github.com/ittaiz> 1. please do - i think this is a bit too POC code yet - it should have major beautification refactoring. 2. please review in the other repository anyway and we can decide on copying later. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABUIF6-3Hd7Hj5h1iDyKnsKcsIiNvcbuks5sI2_HgaJpZM4OIJnJ> .

ittaiz

falling asleep. didn't finish. didn't want you to be blocked in the meanwhile

ittaiz · 2017-07-02T19:44:26Z

plugin/src/main/resources/toolbox.classpath

@@ -0,0 +1 @@
+/Users/natans/Work/rules_scala/bazel-genfiles/external/scala/_ijar/scala-compiler/external/scala/lib/scala-compiler-ijar.jar:/Users/natans/Work/rules_scala/bazel-genfiles/external/scala/_ijar/scala-library/external/scala/lib/scala-library-ijar.jar


why do you need this? only for tests? take a look at $location expansion on bazel.build

ittaiz · 2017-07-02T19:48:46Z

plugin/src/test/scala/io/github/retronym/dependencyanalyzer/DependencyAnalyzerSpec.scala

+        |  org.apache.commons.lang3.ArrayUtils.EMPTY_BOOLEAN_ARRAY.length
+        |}
+      """.stripMargin
+    val commonsPath = Coursier.getArtifact(commons)


this looks like a really problematic pattern for a bazel build. why not have "commons" in the deps?

ittaiz · 2017-07-02T19:49:26Z

plugin/src/test/BUILD

+        deps = ["//main:dependency_analyzer",
+                "@io_get_coursier_coursier_cache//jar",
+                "@io_get_coursier_coursier//jar",
+                "@org_scalaz_scalaz_concurrent_2_11//jar",


why do you need this? also, why do you need the scalaz stuff?

…trib#232)

instead of overloading jvm_flags for both scalac and executable (breaking change)

…ention deprecation of jvm_flags

* Restructure scala provider, add JavaProvider The key change is that the scala provider has been completely restructured to match the structure of the JavaProvider. It no longer tracks exports separately but instead has the following fields: * outputs = contains all the outputs generated by the current rule (nothing or ijar and class_jar); however, no rules here actually use it. * compile_jars = contains all the jars dependent rules should compile with. Thus, ijar and exports.compile_jars * transitive_runtime_jars = contains all the jars needed to handle this target at runtime. Thus, class_jar plus (deps + exports + runtime_deps).transitive_runtime_jars The created java provider (used by dependent native java rules) uses just compile_jars and transitive_runtime_jars. In general, this change was seamless. The major exception is if you were specifically relying on only exports being re-exported by specifically gen_scrooge_srcjar. Beforehand, any jars produced by direct dependencies were not being exported to dependents but now they are. Note that, if you were using scrooge_scala_library instead, nothing should change. Other changes: * Use depset instead of set. (set docs already say set should be considered deprecated and just points to depset anyway) * Tests amended to exploit that JavaProvider is properly constructed. Note that things are still a little strange until Bazel releases incorporate some fixes to JavaProvider and native provider rules. Generally, only deps for java_library and java_binary work at the moment. * Using JavaProvider, requires Bazel 0.5.2 Remove scala_exports_to_java Prefer using JavaProvider over scala provider (can probably avoid using scala provider completely?) * update minimum bazel version to 0.5.2

note we need 0.5.2 now.

* don't put signatures in fat jar * Add tests

Add a provider for binary targets

The key change is that the scala provider has been completely restructured to match the structure of the JavaProvider. It no longer tracks exports separately but instead has the following fields: * outputs = contains all the outputs generated by the current rule (nothing or ijar and class_jar); however, no rules here actually use it. * compile_jars = contains all the jars dependent rules should compile with. Thus, ijar and exports.compile_jars * transitive_runtime_jars = contains all the jars needed to handle this target at runtime. Thus, class_jar plus (deps + exports + runtime_deps).transitive_runtime_jars The created java provider (used by dependent native java rules) uses just compile_jars and transitive_runtime_jars. In general, this change was seamless. The major exception is if you were specifically relying on only exports being re-exported by specifically gen_scrooge_srcjar. Beforehand, any jars produced by direct dependencies were not being exported to dependents but now they are. Note that, if you were using scrooge_scala_library instead, nothing should change. Other changes: * Use depset instead of set. (set docs already say set should be considered deprecated and just points to depset anyway) * Tests amended to exploit that JavaProvider is properly constructed. Note that things are still a little strange until Bazel releases incorporate some fixes to JavaProvider and native provider rules. Generally, only deps for java_library and java_binary work at the moment.

nadavwe · 2017-08-13T15:56:34Z

changes were merged to upstream in bazel-contrib#243.

Stephen Twigg and others added 3 commits June 19, 2017 17:16

Using JavaProvider, requires Bazel 0.5.2

49fae27

Remove scala_exports_to_java Prefer using JavaProvider over scala provider (can probably avoid using scala provider completely?)

using classpath shrinker plugin to warn about indirect dependency usage

031ddee

fix escaping bugs

a827601

attempt to put dependency-analyzer inside rules_scala

f0a96ff

ittaiz requested changes Jul 2, 2017

View reviewed changes

natansil and others added 19 commits July 3, 2017 12:42

merge workspaces

22cb9d2

renamed MoreScala* targets to ResourcesStripScala* targets (bazel-con…

3ec634d

…trib#232)

introduced scalac_jvm_flags

9aa0499

instead of overloading jvm_flags for both scalac and executable (breaking change)

introduced javac_jvm_flags

3e31081

removed duplication of failing tests

78d6ed6

changed docs to include scalac_jvm_flags and javac_jvm_flags and to m…

0f8bb37

…ention deprecation of jvm_flags

update rules_scala version to latest in readme (bazel-contrib#237)

7330451

update README.md

3e2f34d

note we need 0.5.2 now.

don't put signatures in deploy jar (bazel-contrib#229)

6459879

* don't put signatures in fat jar * Add tests

Add a provider for binary targets (bazel-contrib#238)

0dc305a

Add a provider for binary targets

misc: add line for test and build formatting

b700d96

Use workspace name in tut rule

d51e465

using classpath shrinker plugin to warn about indirect dependency usage

d725ad6

fix escaping bugs

303af80

merge workspaces

6d1de17

fix bugs

a46aefe

remove hardcoded jar paths

93db8c3

nadavwe closed this Aug 13, 2017

ittaiz deleted the classpath-shrinker branch August 22, 2017 20:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

using Classpath shrinker to warn on indirect targets #1

using Classpath shrinker to warn on indirect targets #1

Uh oh!

nadavwe commented Jun 28, 2017 •

edited

Loading

Uh oh!

nadavwe commented Jun 28, 2017

Uh oh!

natansil commented Jun 29, 2017

Uh oh!

ittaiz commented Jun 29, 2017 via email

Uh oh!

nadavwe commented Jun 29, 2017

Uh oh!

ittaiz commented Jun 29, 2017 via email

Uh oh!

ittaiz left a comment

Uh oh!

ittaiz Jul 2, 2017

Uh oh!

ittaiz Jul 2, 2017

Uh oh!

ittaiz Jul 2, 2017

Uh oh!

nadavwe commented Aug 13, 2017

Uh oh!

Uh oh!

		@@ -0,0 +1 @@
		/Users/natans/Work/rules_scala/bazel-genfiles/external/scala/_ijar/scala-compiler/external/scala/lib/scala-compiler-ijar.jar:/Users/natans/Work/rules_scala/bazel-genfiles/external/scala/_ijar/scala-library/external/scala/lib/scala-library-ijar.jar

using Classpath shrinker to warn on indirect targets #1

using Classpath shrinker to warn on indirect targets #1

Uh oh!

Conversation

nadavwe commented Jun 28, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nadavwe commented Jun 28, 2017

Uh oh!

natansil commented Jun 29, 2017

Uh oh!

ittaiz commented Jun 29, 2017 via email

Uh oh!

nadavwe commented Jun 29, 2017

Uh oh!

ittaiz commented Jun 29, 2017 via email

Uh oh!

ittaiz left a comment

Choose a reason for hiding this comment

Uh oh!

ittaiz Jul 2, 2017

Choose a reason for hiding this comment

Uh oh!

ittaiz Jul 2, 2017

Choose a reason for hiding this comment

Uh oh!

ittaiz Jul 2, 2017

Choose a reason for hiding this comment

Uh oh!

nadavwe commented Aug 13, 2017

Uh oh!

Uh oh!

nadavwe commented Jun 28, 2017 •

edited

Loading