case class companion object generates two apply methods

[DieBauer]

I've encountered an issue when upgrading the compiler to 2.12.7.

I have a piece of java code that uses the apply method of a scala case class.

final MyClass myClass = MyClass.apply(1, 2, 3L, 4)

When using MyClass compiled with Scala 2.12.7, I get: incompatible types: java.lang.Object cannot be converted to MyClass.

Where, before it compiled fine.

I'm now looking into the generated bytecode and can see this.

Test.scala:

case class MyClass(a: Int, b: Int, c: Long, d: Int)

$ scalac Test.scala
$ scala -classpath .
Welcome to Scala 2.12.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_162).
Type in expressions for evaluation. Or try :help.

scala> :javap -p MyClass
Compiled from "Test.scala"
public class MyClass implements scala.Product,scala.Serializable {
  private final int a;
  private final int b;
  private final long c;
  private final int d;
  public static java.lang.Object apply(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object);
  public static scala.Option<scala.Tuple4<java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object>> unapply(MyClass);
  public static MyClass apply(int, int, long, int);
  public static scala.Function1<scala.Tuple4<java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object>, MyClass> tupled();
  public static scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, MyClass>>>> curried();
  public int a();
  public int b();
  public long c();
  public int d();
  public MyClass copy(int, int, long, int);
  public int copy$default$1();
  public int copy$default$2();
  public long copy$default$3();
  public int copy$default$4();
  public java.lang.String productPrefix();
  public int productArity();
  public java.lang.Object productElement(int);
  public scala.collection.Iterator<java.lang.Object> productIterator();
  public boolean canEqual(java.lang.Object);
  public int hashCode();
  public java.lang.String toString();
  public boolean equals(java.lang.Object);
  public MyClass(int, int, long, int);
}

When doing the same exercise on scala 2.12.6 I get:

$ scalac Test.scala
$ scala -classpath .
Welcome to Scala 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_162).
Type in expressions for evaluation. Or try :help.

scala> :javap -p MyClass
Compiled from "Test.scala"
public class MyClass implements scala.Product,scala.Serializable {
  private final int a;
  private final int b;
  private final long c;
  private final int d;
  public static scala.Option<scala.Tuple4<java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object>> unapply(MyClass);
  public static MyClass apply(int, int, long, int);
  public static scala.Function1<scala.Tuple4<java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object>, MyClass> tupled();
  public static scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, scala.Function1<java.lang.Object, MyClass>>>> curried();
  public int a();
  public int b();
  public long c();
  public int d();
  public MyClass copy(int, int, long, int);
  public int copy$default$1();
  public int copy$default$2();
  public long copy$default$3();
  public int copy$default$4();
  public java.lang.String productPrefix();
  public int productArity();
  public java.lang.Object productElement(int);
  public scala.collection.Iterator<java.lang.Object> productIterator();
  public boolean canEqual(java.lang.Object);
  public int hashCode();
  public java.lang.String toString();
  public boolean equals(java.lang.Object);
  public MyClass(int, int, long, int);
}

A duplicate apply method is generated in scala 2.12.7 with the following signature:
public static java.lang.Object apply(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object);
and it seems this one is being picked in my Java project, causing the exception.
It seems that this is an unintended breaking change between these two versions.

Of course a workaround is using the new keyword:

final MyClass myClass = new MyClass(1, 2, 3L, 4)

[lrytz]

Hmm, this is quite likely an unintended consequence of scala/scala#7035.

[martin-g]

I've faced the same problem while upgrading to 2.12.7

[lrytz]

Commented here for the reason we get more forwarders: https://github.com/scala/scala/pull/7035/files#r226274350

[lrytz]

However, @DieBauer and @martin-g, I cannot reproduce the problem that you are having with javac. For me, javac compiles the apply call successfully:

$> cat Test.scala
case class MyClass(a: Int, b: Int, c: Long, d: Int)
$> scalac -version
Scala compiler version 2.12.7 -- Copyright 2002-2018, LAMP/EPFL and Lightbend, Inc.
$> scalac Test.scala
$> cat Test.java
public class Test {
  final MyClass myClass = MyClass.apply(1, 2, 3L, 4);
}
$> javac -version
javac 1.8.0_181
$> javac Test.java -cp ../build/quick/classes/library:.
$> scala -e "println((new Test).myClass)"
MyClass(1,2,3,4)
$>

Which version of Java are you using? Where do you get the error: command line / build tool / IDE?

[martin-g]

I had the problem both in Sbt and IDEA.
In my case the Scala case class is in module1. The Java class is in module2 that depends on module1.

Let me try to reproduce it!

[martin-g]

I also was not able to reproduce it in a mini app.
javap shows the extra apply() method with all parameters being java.lang.Object but it doesn't break neither Sbt nor IDEA.

[hrhino]

Another workaround is to explicitly add a companion object (this prevents it from tacitly extending a FunctionN class, which is where the bridge is coming from).

[DieBauer]

Alright, it was a bit more complex than I initial thought. I think the gist is that the java types are not primitives, but boxed (hence the selection of the Object constructor).

I've created a reproducer: https://github.com/DieBauer/reproducer-scala on 2.12.6 in build sbt it compiles, setting it to 2.12.7 it doesn't.

reproducer/src/main/java/Test.java:3:1: incompatible types: java.lang.Object cannot be converted to MyClass
[error]     MyClass.apply(Integer.valueOf(1), Integer.valueOf(2), Long.valueOf(3), Integer.valueOf(4));
[error] (Compile / compileIncremental) javac returned non-zero exit code

I'm using com.oracle.jdk8u162

edit: simplified reproducer in repository

[lrytz]

@DieBauer thanks! To summarize it here:

Scala:

case class MyClass(a: Int)

Java:

MyClass.apply(Integer.valueOf(1));

Test.java:5: error: incompatible types: Object cannot be converted to MyClass
  final MyClass myClass = MyClass.apply(Integer.valueOf(1));
                                       ^

So javac picks the (Object)Object overload, even though it's marked bridge in bytecode. It's a bit unfortunate because it kind of introduces a new version of #11061, the bug that was actually fixed in scala/scala#7035.

I think it's limited to the case of auto-unboxing. For non-primitives, the bridge method is less specific than the non-bridge, so javac should pick the non-bridge.

Also it looks like there's no issue with return types, the following compiles fine:

Scala:

abstract class A[T] { def f: T }
object B extends A[Int] { def f = 1 }

Java:

final int a = B.f();
final Integer b = B.f();
final Object c = B.f();

So it looks like the scope of the bug is quite limited. I'm a bit worried to fix it for 2.12.8, because it would remove methods that exist in 2.12.7, which is binary incompatible. Since it's already fixed in 2.13, I think we should just do nothing. Unfortunately...

[gehnaphore]

Our project uses a big hierarchy of case classes which is processed and transformed across many stages by both Scala and Java code, and this breaks a lot of our code.

[SethTisue]

hmm, maybe a 2.12-only flag that would let people opt in to the fix, if they are confident doing so won't cause headaches?

[lrytz]

That's an idea, we could backport the 2.13 fix under a flag. But it wouldn't help for libraries fetched from maven (that are compiled without that flag).

[DieBauer]

By the way, if you have a case class that has a mix of AnyRef and primitive fields, and you call it from Java (boxed primitive) with the generated apply method a second compiler errors gets emitted:

class X
case class MyClass(a: Int, aa: Int, b: String, c: X)

final MyClass myClass = MyClass.apply(1, Integer.valueOf(1), "bla", new X());

Test.java:3:1: reference to apply is ambiguous
[error]   both method apply(java.lang.Object,java.lang.Object,java.lang.Object,java.lang.Object) in MyClass and method apply(int,int,java.lang.String,X) in MyClass match
[error]             MyClass.apply(1, Integer.valueOf(1), "bla", new X());
[error] Test.java:3:1: incompatible types: java.lang.Object cannot be converted to MyClass
[error]             MyClass.apply(1, Integer.valueOf(1), "bla", new X());

So I think the potential impact is that any library that gets compiled with 2.12.7 (and if this is not rolled back, further future 2.12.x releases) which is used from a Java project and passes boxed primitives to the apply method is broken.

[SethTisue]

once we have a plan, we should probably publicize it on https://contributors.scala-lang.org/t/library-authors-rebuild-on-2-12-7-to-improve-callability-from-java-on-jdk-11/2465/7

[lrytz]

We discussed this regression internally today. @adriaanm and @retronym are less conservative than me and would prefer release 2.12.8 soon with a fix for this bug that restores the 2.12.6 behavior.

After all, the bug only affects Java code that uses Scala libraries (the static accessors cannot be used from Scala code). The majority of such Java code is application code, not libraries published to maven. So the risk for binary incompatibilities is small.

[shawjef3]

This also causes serialization surprises. https://users.scala-lang.org/t/serialversionuid-change-between-scala-2-12-6-and-2-12-7/3478

[dwijnand]

Fixed in scala/scala#7383.