push scala-java8-compat stdlib integration through 2.13 community build

[SethTisue]

looking at last night's 2.13 CB run, the obvious candidates to try are:

• akka
• better-files
• metrics-scala
• play-file-watch

the first three have published for M5, the fourth hasn't so might need other changes for 2.13

[SethTisue]

now that I take a closer look, I see that it's just Akka, the other projects are getting scala-java8-compat through the akka dependency

[info] Project better-files
[info]   depends on: akka, cloc-plugin, genjavadoc, scala, scala-collection-compat, scala-java8-compat, scala-xml, scalacheck, scalatest, shapeless
[info] Project akka
[info]   depends on: cloc-plugin, genjavadoc, scala, scala-collection-compat, scala-java8-compat, scala-xml, scalacheck, scalatest
[info] Project metrics-scala
[info]   depends on: akka, cloc-plugin, genjavadoc, scala, scala-collection-compat, scala-java8-compat, scala-xml, scalacheck, scalatest
[info] Project play-file-watch
[info]   depends on: akka, better-files, cloc-plugin, genjavadoc, kind-projector, scala, scala-collection-compat, scala-java8-compat, scala-js-stubs, scala-parser-combinators, scala-xml, scalacheck, scalatest, shapeless, specs2

[lrytz]

I managed to get the community build to run, but was busy with scala/scala#7929 for the rest of the week :-) I will take a look at akka today.

[lrytz]

Akka builds, there are some test failures. Minimal community build for akka: https://github.com/scala/community-builds/compare/2.13.x...lrytz:akka-2.13?expand=1

My changes: https://github.com/lrytz/akka/commits/no-java8-compat

[lrytz]

[info] ---==  Execution Report ==---
[info] Report from the dbuild run for dbuild:
[info] Project akka------------------------: FAILED (RuntimeException: Tests unsuccessful)
[info] Project cloc-plugin-----------------: SUCCESS (unchanged, not rebuilt)
[info] Project scala-----------------------: SUCCESS (unchanged, not rebuilt)
[info] Project scala-collection-compat-----: SUCCESS (unchanged, not rebuilt)
[info] Project scala-xml-------------------: SUCCESS (unchanged, not rebuilt)
[info] Project scalacheck------------------: SUCCESS (unchanged, not rebuilt)
[info] Project scalatest-------------------: SUCCESS (unchanged, not rebuilt)

[lrytz]

They seem to be various kinds of failures, maybe best for someone from Akka to look into?

[akka] [error] Error: Total 1074, Failed 65, Errors 1, Passed 1008, Ignored 1, Pending 1
[akka] [error] Failed tests:
[akka] [error] 	akka.routing.ConfiguredLocalRoutingSpec
[akka] [error] 	akka.routing.ResizerSpec
[akka] [error] 	akka.actor.ActorSystemSpec
[akka] [error] 	akka.routing.RoutingSpec
[akka] [error] 	akka.util.ByteStringSpec
[akka] [error] 	akka.routing.BalancingSpec
[akka] [error] 	akka.actor.ActorMailboxSpec
[akka] [error] 	akka.actor.DeployerSpec
[akka] [error] 	akka.io.TcpConnectionSpec
[akka] [error] 	akka.util.LineNumberSpec
[akka] [error] 	akka.actor.dispatch.DispatchersSpec
[akka] [error] 	akka.serialization.SerializationCompatibilitySpec
[akka] [error] Error during tests:
[akka] [error] 	akka.routing.ConsistentHashingRouterSpec

[SethTisue]

you can push your changes to 2.13.x, let’s book the progress

[SethTisue]

you can push your changes to 2.13.x, let’s book the progress

Lukas did this, and Jenkins has finished chewing on it

@patriknw https://scala-ci.typesafe.com/job/scala-2.13.x-integrate-community-build/1913/consoleFull has the detailed error log

[SethTisue]

in case it's not clear: ultimately we want to address the failures of course, but for the RC1 release we're aiming to complete this week, the bar we're trying to clear is to be reasonably confident that the failures aren't caused by Scala regressions.

/cc @viktorklang in case it turns out that scala.concurrent changes are somehow involved

[patriknw]

Seems to be many test failures. Do you have a published build we can try?
We run all tests with 2.13.0-M5 without problems.

[SethTisue]

@patriknw the Scala nightly we're using currently is 2.13.0-pre-b4926ef, which is resolvable from https://scala-ci.typesafe.com/artifactory/scala-integration/

but just having the Scala nightly isn't enough, I think, since you have dependencies, such as ScalaTest. so you would have to publish the dependencies locally for that nightly, or use dbuild to do your troubleshooting, which is doable but a bit involved. see https://github.com/scala/community-builds/wiki/Troubleshooting-a-failure . Lukas and I are both available to help with this

[SethTisue]

@patriknw I see Johan and Helena are discussing the Akka specifics on Lightbend's Slack, so maybe better to continue w/ specifics there and then summarize here once some conclusion is reached

[SethTisue]

I've made a branch where the community build config has only the akka entry (consisting of these subprojects: "akka-actor", "akka-actor-typed", "akka-testkit", "akka-actor-tests") and pushed it to https://github.com/SethTisue/community-builds/tree/2.13-akka-only

so your steps to reproduce the problem are:

hub clone SethTisue/community-builds
cd community-builds
git checkout 2.13-akka-only
./run.sh

which will then run for, I don't know, 20 minutes, something like that (or considerably longer, should you happen to be on shitty internet at an Airbnb in London)

at the end you will see:

[akka] [error] Error: Total 1074, Failed 64, Errors 1, Passed 1009, Ignored 1, Pending 1
[akka] [error] Failed tests:
...
The repeatable UUID of this build was: e30be73033236284f66b4b246b2b9ab802ed13b2

then to troubleshoot, follow with:

./dbuild-0.9.16/bin/dbuild checkout e30be73033236284f66b4b246b2b9ab802ed13b2 akka akka-wip
./akka-wip/.dbuild/start

and from the sbt prompt:

akka-actor-tests/test

or whatever other sbt commands you want to issue, at this point you're in normal sbt. dbuild-build will run all the exact same stuff the community build would. normal commands like akka-actor-tests/test are fine to issue as well

you can also make source code changes in the akka-wip directory and test them like normal.

[patriknw]

One important finding is that a line is missing from the Akka sources that are used in the community build: https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/actor/Deployer.scala#L158

deploy is not called. That is causing many tests to fail (those related to routers)

I think the community build must be updated to latest Akka master.

After fixing that line, the following tests are failing in akka-actor-tests:

• SerializeSpec, NoMessage did not equal NoMessage (SerializeSpec.scala:365)
• LineNumberSpec, e.g. LineNumberSpecCodeForScala.scala:26-27 did not equal LineNumberSpecCodeForScala.scala:25-27 (LineNumberSpec.scala:44)
• ByteStringSpec, e.g. when adding various contents using ++= and +=
• ActorSystemSpec, tracked in InternalCallbackExecutor vs ExecutionContext.parasitic akka/akka#26655, will be fixed in Akka

ByteStringSpec might be most important since ByteString is using collections.

[patriknw]

I tried with latest ByteString and then ByteStringSpec is also passing apart from one serialization test that probably just need to be updated in Akka: https://github.com/akka/akka/blob/abbbfb5b5465596b27811f93c48dd7fe11992d32/akka-actor-tests/src/test/scala/akka/util/ByteStringSpec.scala#L900

[SethTisue]

@patriknw so there's nothing here should that hold up RC1?

[patriknw]

Seems good. Remaining unknown is possible incompatibility in java serialization, but I guess that would be ok (at least for RC). The test is here: https://github.com/akka/akka/blob/1a2cf3d4c66d85f4435f42663c835305ed71e054/akka-actor-tests/src/test/scala/akka/serialization/SerializeSpec.scala#L445

It's a case object, trying to deserialize format from older Scala version (probably 2.11). It can deserialize but it is not the same (eq) instance

[SethTisue]

okay, I agree that sounds like something we can resolve after RC1 is out, feel free to open a new ticket on that and/or just talk to us about it