community build: Shapeless fails to build in latest 2.12.x

[SethTisue]

@retronym perhaps this has to do with your recent exhaustivity checking changes. can you take a look and see if you think this is something @milessabin should address upstream, or what?

[shapeless] [error] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/shapeless-00c8fb2f48375687d7726897cc240406c471df46/core/src/test/scala/shapeless/hlist.scala:1762: match may not be exhaustive.
[shapeless] [error] It would fail on the following inputs: ::((x: Int forSome x not in 23), _), ::(23, _)
[shapeless] [error]     val tl = l2 match {
[shapeless] [error]              ^
[shapeless] [error] one error found
[shapeless] [error] (core/test:compileIncremental) Compilation failed

(from https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/306/consoleFull)

[retronym]

I believe the new warning is legit, and the code ought to be:

    val tl = (l2 : @unchecked) match {
      case 23 #: 3.0 #: s #: xs => (s, xs)
    }

Or just :

   val 23 #: 3.0 #: s #: xs = l2

Which expands to an match on an @unchecked scrutinee.

[retronym]

Here are all the exhaustivity warnings in that build log. These might not all be new, and there might be plenty of new ones that we are yet to see in projects that depend on shapeless and have not been built.

[scala-refactoring] [warn] /**
[scala-refactoring] [warn] ^
[scala-refactoring] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scala-refactoring-ad6967d6ac6ee7d3da4be0814377b7eb51efa7b2/org.scala-refactoring.library/src/main/scala/scala/tools/refactoring/sourcegen/LayoutHelper.scala:298: match may not be exhaustive.
[scala-refactoring] [warn] It would fail on the following inputs: ('\n', '\r'), ('\n', _), ('\r', '\n'), ('\r', _), (??, _), (_, '\n'), (_, '\r'), (_, ??), (_, _)
[scala-refactoring] [warn]     def mergeLayoutWithComment(l: Seq[Char], c: Seq[Char]) = l.zip(c).map {
[scala-refactoring] [warn]                                                                           ^
[scala-refactoring] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scala-refactoring-ad6967d6ac6ee7d3da4be0814377b7eb51efa7b2/org.scala-refactoring.library/src/main/scala/scala/tools/refactoring/common/PimpedTrees.scala:1261: Could not find any member to link for "scala.reflect.internal.Trees.ValOrDefDef".
--
--
[shapeless] [info] Compiling 1 Scala source to /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/shapeless-00c8fb2f48375687d7726897cc240406c471df46/core/target/scala-2.12.0-dbuildx46f1683c194439ea9f50cc2a814fc505b1117859/classes...
[shapeless] [info] Compiling 36 Scala sources to /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/shapeless-00c8fb2f48375687d7726897cc240406c471df46/core/target/scala-2.12.0-dbuildx46f1683c194439ea9f50cc2a814fc505b1117859/test-classes...
[shapeless] [error] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/shapeless-00c8fb2f48375687d7726897cc240406c471df46/core/src/test/scala/shapeless/hlist.scala:1762: match may not be exhaustive.
[shapeless] [error] It would fail on the following inputs: ::((x: Int forSome x not in 23), _), ::(23, _)
[shapeless] [error]     val tl = l2 match {
[shapeless] [error]              ^
[shapeless] [error] one error found
--
--
[scalatest] [warn]                typeList: List[TypeTree]
[scalatest] [warn]                          ^
[scalatest] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalatest-56f21355f681be4de6810ac90dbb3b97fcb1cc8a/src/main/scala/org/scalatest/Doc.scala:193: match may not be exhaustive.
[scalatest] [warn] It would fail on the following inputs: (None, Some(_)), (Some(_), None)
[scalatest] [warn]     (firstNonWhiteLine, lastNonWhiteLine) match {
[scalatest] [warn]     ^
[scalatest] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalatest-56f21355f681be4de6810ac90dbb3b97fcb1cc8a/src/main/scala/org/scalatest/DocSpec.scala:48: match may not be exhaustive.
--
--
[scalatest] [warn]     (firstNonWhiteLine, lastNonWhiteLine) match {
[scalatest] [warn]     ^
[scalatest] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalatest-56f21355f681be4de6810ac90dbb3b97fcb1cc8a/src/main/scala/org/scalatest/DocSpec.scala:48: match may not be exhaustive.
[scalatest] [warn] It would fail on the following inputs: (None, Some(_)), (Some(_), None)
[scalatest] [warn]     (firstNonWhiteLine, lastNonWhiteLine) match {
[scalatest] [warn]     ^
[scalatest] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/scalatest-56f21355f681be4de6810ac90dbb3b97fcb1cc8a/src/main/scala/org/scalatest/PrivateMethodTester.scala:254: unreachable code
--
--
[akka] [info] Done updating.
[akka] [info] Compiling 18 Scala sources and 1 Java source to /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/akka-622a540c67e9bc2514e3685a0f16c7ad317992b0/akka-cluster/target/classes...
[akka] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/akka-622a540c67e9bc2514e3685a0f16c7ad317992b0/akka-cluster/src/main/scala/akka/cluster/Member.scala:142: match may not be exhaustive.
[akka] [warn] It would fail on the following inputs: (Up, _), (_, Up), (_, _)
[akka] [warn]   def highestPriorityOf(m1: Member, m2: Member): Member = (m1.status, m2.status) match {
[akka] [warn]                                                           ^
[akka] [warn] there were 19 deprecation warnings; re-run with -deprecation for details
--
--
[twitter-util] [warn]                        ^
[twitter-util] [warn] there was one feature warning; re-run with -feature for details
[twitter-util] [warn] /home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/twitter-util-9af793cecbb0c98806e6e9dd6b3751cda540aa01/util-zk/src/main/scala/com/twitter/zk/Event.scala:49: match may not be exhaustive.
[twitter-util] [warn] It would fail on the following inputs: ConnectedReadOnly, SaslAuthenticated, Unknown
[twitter-util] [warn]     w.getState() match {
[twitter-util] [warn]               ^
[twitter-util] model contains 68 documentable templates

[dwijnand]

Sorry for the highjack but is it legitimate because the type of l2 is Int :: Double :: (etc)?

[retronym]

Yep, rather than 23.type :: ....

[dwijnand]

Gotcha. Thank you.

[retronym]

Proposed a fix for the akka warning: akka/akka#19680

The Scalatest one looks legit, a code comment justifies the missing cases, this should be : @unchecked.

The twitter-util warning also looks legit, and is probably an old one.

[milessabin]

I agree that the new warning in the shapeless test is correct. @SethTisue is that the only example you've seen?

[milessabin]

Better still is there compiler snapshot build I could test with?

[milessabin]

I've pushed a fix for the instance that @SethTisue spotted.

[SethTisue]

I've pushed a fix for the instance that @SethTisue spotted.

awesome, thanks

@SethTisue is that the only example you've seen?

well, let's just see what a new build does: https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/307/consoleFull

Better still is there compiler snapshot build I could test with?

we don't currently have -SNAPSHOT publishing, though I plan to bring it back before tooooo much longer, certainly before 2.12 goes into RCs (it's scala/scala-jenkins-infra#133)

for now, what we have instead are:

• nightly distributions (just distributions, not Maven/Ivy artifacts) downloadable from http://www.scala-lang.org/files/archive/nightly/2.12.x/
• PR validation snapshots, as described in the README.md in the scala/scala repo

the easiest thing at the moment is probably to untar a recent nightly distribution and then use scalaHome to point sbt at it.

[dwijnand]

• PR validation snapshots, as described in the README.md in the scala/scala repo

TIL, cool

[SethTisue]

sigh, I forgot that in the 2.12 community build, Shapeless is actually frozen at d6a4f56e0783655c48244ec23c4f4c2d98401a72, even thought there's no reason for it to be anymore since scala/community-build#202 was resolved. I'll go fix that right now.

[SethTisue]

ah, I messed up when I merged scala/community-build#206 onto 2.12.x. let's try this again: https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/308/consoleFull

UPDATE: again, with further fix to bad merge: https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/309/consoleFull

[SethTisue]

I have opened a ticket on the new failure at scalamacros/paradise#78. I'm going to close this ticket for now on the optimistic assumption that once paradise is updated, Shapeless will start working again too.

[dwijnand]

Wow that output.. before you lose your sanity we should really try and fix this toString:

ScopedKey(Scope(Select(ProjectRef(file:/home/jenkins/workspace/scala-2.12.x-integrate-community-build/target-0.9.5/project-builds/macro-paradise-21ffe7a7ba52c5fa52f67b89b7e263d1bd2af480/,paradise)),Select(ConfigKey(compile)),Global,Global),copyResources)

[milessabin]

Where are we with this now? What's the ETA for 2.12.0-M4?

[SethTisue]

We might build M4 as soon as Feb 11, but it depends on a couple of large PRs crossing the finish line, so we can't say for sure.

As for this ticket, it's waiting on scalamacros/paradise#78; I will comment there.

[SethTisue]

The latest community build run shows that both scalamacros/paradise#78 and this ticket are fixed.

[milessabin]

Many thanks :-)