Graalvm native image doesn't work with 2.13.7 release

[Christewart]

reproduction steps

It seems there is a regression introduced in Scala 2.13.7 breaking our graalvm native image build. Things worked just fine on 2.13.6

Here is the link to the CI build showing the failure:

https://github.com/bitcoin-s/bitcoin-s/runs/4091599423?check_suite_focus=true#step:5:134

Original issue:

scalameta/sbt-native-image#40 (comment)

[info] compiling 312 Scala sources to /Users/runner/work/bitcoin-s/bitcoin-s/core/.jvm/target/scala-2.13/classes ...
[info] done compiling
[info] compiling 32 Scala sources to /Users/runner/work/bitcoin-s/bitcoin-s/app-commons/target/scala-2.13/classes ...
[info] done compiling
[info] compiling 3 Scala sources to /Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/scala-2.13/classes ...
[info] done compiling
[info] /Users/runner/Library/Caches/Coursier/jvm/[email protected]/Contents/Home/bin/native-image -cp /Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image-internal/manifest.jar -H:+ReportExceptionStackTraces --initialize-at-build-time --no-fallback --enable-http --enable-https org.bitcoins.cli.Cli /Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]    classlist:   9,408.05 ms,  0.96 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]        (cap):   3,888.97 ms,  0.96 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]        setup:  12,235.85 ms,  0.96 GB
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]     (clinit):     530.04 ms,  2.68 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]   (typeflow):  36,780.80 ms,  2.68 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]    (objects):  25,726.13 ms,  2.68 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]   (features):   2,922.12 ms,  2.68 GB
[/Users/runner/work/bitcoin-s/bitcoin-s/app/cli/target/native-image/bitcoin-s-cli:5959]     analysis:  67,712.74 ms,  2.68 GB
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call or single field access. The method handle must be a compile time constant, e.g., be loaded from a `static final` field. Method that contains the method handle invocation: java.lang.invoke.LambdaForm$MH/945987348.invoke_MT(Object, Object)
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The error is then reported at run time when the invoke is executed.
Detailed message:
Trace: 
	at parsing scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
Call path from entry point to scala.runtime.NonLocalReturnControl.<init>(Object, Object): 
	at scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$.$anonfun$splitWithEscape$1(UriInterpolator.scala:294)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$$$Lambda$1477/0x00000007c1921040.apply$mcVI$sp(Unknown Source)
	at scala.runtime.java8.JFunction1$mcVI$sp.apply(JFunction1$mcVI$sp.scala:18)
	at scala.collection.immutable.Vector3.map(Vector.scala:1872)
	at scala.collection.immutable.Vector3.map(Vector.scala:527)
	at org.bitcoins.core.protocol.script.P2WSHWitnessV0.toString(ScriptWitness.scala:139)
	at java.lang.String.valueOf(String.java:2951)
	at java.io.PrintStream.print(PrintStream.java:745)
	at java.io.PrintStream.println(PrintStream.java:882)
	at com.oracle.svm.jni.functions.JNIFunctions.ExceptionDescribe(JNIFunctions.java:766)
	at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ExceptionDescribe_b5412f7570bccae90b000bc37855f00408b2ad73(generated:0)

com.oracle.svm.core.util.UserError$UserException: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call or single field access. The method handle must be a compile time constant, e.g., be loaded from a `static final` field. Method that contains the method handle invocation: java.lang.invoke.LambdaForm$MH/945987348.invoke_MT(Object, Object)
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The error is then reported at run time when the invoke is executed.
Detailed message:
Trace: 
	at parsing scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
Call path from entry point to scala.runtime.NonLocalReturnControl.<init>(Object, Object): 
	at scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$.$anonfun$splitWithEscape$1(UriInterpolator.scala:294)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$$$Lambda$1477/0x00000007c1921040.apply$mcVI$sp(Unknown Source)
	at scala.runtime.java8.JFunction1$mcVI$sp.apply(JFunction1$mcVI$sp.scala:18)
	at scala.collection.immutable.Vector3.map(Vector.scala:1872)
	at scala.collection.immutable.Vector3.map(Vector.scala:527)
	at org.bitcoins.core.protocol.script.P2WSHWitnessV0.toString(ScriptWitness.scala:139)
	at java.lang.String.valueOf(String.java:2951)
	at java.io.PrintStream.print(PrintStream.java:745)
	at java.io.PrintStream.println(PrintStream.java:882)
	at com.oracle.svm.jni.functions.JNIFunctions.ExceptionDescribe(JNIFunctions.java:766)
	at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ExceptionDescribe_b5412f7570bccae90b000bc37855f00408b2ad73(generated:0)

	at com.oracle.svm.core.util.UserError.abort(UserError.java:79)
	at com.oracle.svm.hosted.FallbackFeature.reportAsFallback(FallbackFeature.java:217)
	at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:765)
	at com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:555)
	at com.oracle.svm.hosted.NativeImageGenerator.lambda$run$0(NativeImageGenerator.java:468)
	at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call or single field access. The method handle must be a compile time constant, e.g., be loaded from a `static final` field. Method that contains the method handle invocation: java.lang.invoke.LambdaForm$MH/945987348.invoke_MT(Object, Object)
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The error is then reported at run time when the invoke is executed.
Detailed message:
Trace: 
	at parsing scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
Call path from entry point to scala.runtime.NonLocalReturnControl.<init>(Object, Object): 
	at scala.runtime.NonLocalReturnControl.<init>(NonLocalReturnControl.scala:19)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$.$anonfun$splitWithEscape$1(UriInterpolator.scala:294)
	at com.softwaremill.sttp.UriInterpolator$Tokenizer$$$Lambda$1477/0x00000007c1921040.apply$mcVI$sp(Unknown Source)
	at scala.runtime.java8.JFunction1$mcVI$sp.apply(JFunction1$mcVI$sp.scala:18)
	at scala.collection.immutable.Vector3.map(Vector.scala:1872)
	at scala.collection.immutable.Vector3.map(Vector.scala:527)
	at org.bitcoins.core.protocol.script.P2WSHWitnessV0.toString(ScriptWitness.scala:139)
	at java.lang.String.valueOf(String.java:2951)
	at java.io.PrintStream.print(PrintStream.java:745)
	at java.io.PrintStream.println(PrintStream.java:882)
	at com.oracle.svm.jni.functions.JNIFunctions.ExceptionDescribe(JNIFunctions.java:766)
	at com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_ExceptionDescribe_b5412f7570bccae90b000bc37855f00408b2ad73(generated:0)

	at com.oracle.graal.pointsto.constraints.UnsupportedFeatures.report(UnsupportedFeatures.java:126)
	at com.oracle.svm.hosted.NativeImageGenerator.runPointsToAnalysis(NativeImageGenerator.java:762)
	... 8 more
Caused by: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Invoke with MethodHandle argument could not be reduced to at most a single call or single field access. The method handle must be a compile time constant, e.g., be loaded from a `static final` field. Method that contains the method handle invocation: java.lang.invoke.LambdaForm$MH/945987348.invoke_MT(Object, Object)
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The error is then reported at run time when the invoke is executed.
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin.reportUnsupportedFeature(IntrinsifyMethodHandlesInvocationPlugin.java:761)
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin.access$1100(IntrinsifyMethodHandlesInvocationPlugin.java:161)
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin$Transplanter.bailout(IntrinsifyMethodHandlesInvocationPlugin.java:729)
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin$Transplanter.graph(IntrinsifyMethodHandlesInvocationPlugin.java:540)
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin.processInvokeWithMethodHandle(IntrinsifyMethodHandlesInvocationPlugin.java:479)
	at com.oracle.svm.hosted.phases.IntrinsifyMethodHandlesInvocationPlugin.handleInvoke(IntrinsifyMethodHandlesInvocationPlugin.java:232)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.tryNodePluginForInvocation(BytecodeParser.java:2206)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.appendInvoke(BytecodeParser.java:1871)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genDynamicInvokeHelper(BytecodeParser.java:1760)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeVirtual(BytecodeParser.java:1708)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeVirtual(BytecodeParser.java:1693)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5338)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3413)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3220)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1090)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:984)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:84)
	at com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:74)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:214)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:223)
	at com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:357)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.doParse(MethodTypeFlow.java:313)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureParsed(MethodTypeFlow.java:302)
	at com.oracle.graal.pointsto.flow.MethodTypeFlow.addContext(MethodTypeFlow.java:103)
	at com.oracle.graal.pointsto.DefaultAnalysisPolicy$DefaultSpecialInvokeTypeFlow.onObservedUpdate(DefaultAnalysisPolicy.java:368)
	at com.oracle.graal.pointsto.flow.TypeFlow.notifyObservers(TypeFlow.java:470)
	at com.oracle.graal.pointsto.flow.TypeFlow.update(TypeFlow.java:542)
	at com.oracle.graal.pointsto.flow.SourceTypeFlowBase.update(SourceTypeFlowBase.java:144)
	at com.oracle.graal.pointsto.BigBang$2.run(BigBang.java:552)
	at com.oracle.graal.pointsto.util.CompletionExecutor.lambda$execute$0(CompletionExecutor.java:173)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
	... 5 more
Error: Image build request failed with exit status 1

[lrytz]

This is quite certainly due to scala/scala#9704 (review)

The 2.13.7 version of NonLocalReturnControl's constructor has a releaseFence call.

On a first look, it seems that this call might have been added unnecessarily. The bytecode of the constructor doesn't use the if (!this.specInstance$) { ... } pattern. I will take a closer look.

But for native image, the same workaround for releaseFence that's used in other places should work here too.

[olafurpg]

But for native image, the same workaround for releaseFence that's used in other places should work here too.

I'm happy to add new substitutions to https://github.com/scalameta/svm-subs, which is what sbt-native-image uses. However, I'll admit that I don't fully understand what releaseFence does or how GraalVM substitutions work so any help is appreciated. We already have something for scala.runtime.Statics.releaseFence here https://github.com/scalameta/svm-subs/blob/master/svm-subs/src/main/java/scala/meta/internal/svm_subs/Target_scala_runtime_Statistics.java

[olafurpg]

It would be nice to get a minimized reproduction so that we can confirm that a fix works as expected.