MainGenericRunner: do not load jline classes until they are needed

[unkarjedy]

In Scala Plugin we use MainGenericRunner.main to start Scala REPL instance. Usage example:

"C:\Program Files\Java\jdk1.8.0_191\bin\java.exe" -Djline.terminal=NONE \
  -Dfile.encoding=UTF-8 \
  -classpath \
  "~\.ivy2\cache\org.scala-lang\scala-compiler\jars\scala-compiler-2.13.0.jar;~\.ivy2\cache\org.scala-lang\scala-library\jars\scala-library-2.13.0.jar;~\.ivy2\cache\org.scala-lang\scala-reflect\jars\scala-reflect-2.13.0.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_191\jre\lib\rt.jar;" \
  scala.tools.nsc.MainGenericRunner \
  -usejavacp

In Scala <=2.12 It works ok even if jline.x.jar is not present in the classpath.
Unfortunately, in 2.13 some changes were made that make it throw java.lang.NoClassDefFoundError exception.
The behaviour is even observed if -Xnojline parameter (Do not use JLine for editing) is passed explicitly (https://docs.scala-lang.org/overviews/compiler-options/index.html). It's strange that it tries to resolve Jline even if it doesn't need it.

AFAIU this is because some Jline stuff is still referenced in some classes and is being resolved during class loading:

• https://github.com/scala/scala/blob/948da0358c53933100fa643a0f9d841ac4dcfb34/src/repl-frontend/scala/tools/nsc/MainGenericRunner.scala
• https://github.com/scala/scala/blob/ba9701059216c629410f4f23a2175d20ad62484b/src/repl-frontend/scala/tools/nsc/interpreter/jline/JLineReader.scala

related issues:
https://youtrack.jetbrains.com/issue/SCL-15948
https://youtrack.jetbrains.com/issue/SCL-15818

[som-snytt]

Thanks, I noticed that too, when I was experimenting with class paths.

[SethTisue]

@unkarjedy scala/docs.scala-lang#1486 looks like a report from @tahsmith of this same issue. is there any workaround?

[SethTisue]

related issue: https://youtrack.jetbrains.com/issue/SCL-15818

that link doesn't work for me? do you mean https://youtrack.jetbrains.com/issue/SCL-15948 ?

[som-snytt]

Indeed,

[4.807s][info][class,load] scala.tools.nsc.interpreter.MemberHandlers$ImportHandler source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.807s][info][class,load] scala.tools.nsc.interpreter.IMain$Request$$Lambda$762/0x0000000100573040 source: scala.tools.nsc.interpreter.IMain$Request
[4.808s][info][class,load] scala.tools.nsc.interpreter.IMain$$Lambda$763/0x0000000100572840 source: scala.tools.nsc.interpreter.IMain
[4.809s][info][class,load] scala.tools.nsc.interpreter.IMain$$Lambda$764/0x0000000100572040 source: scala.tools.nsc.interpreter.IMain
[4.809s][info][class,load] scala.reflect.internal.Symbols$Symbol$$Lambda$765/0x0000000100571840 source: scala.reflect.internal.Symbols$Symbol
[4.810s][info][class,load] scala.tools.nsc.interpreter.Results$Success$ source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.811s][info][class,load] jline.console.completer.Completer source: /home/amarki/scala-2.13.0/lib/jline-2.14.6.jar
[4.811s][info][class,load] scala.tools.nsc.interpreter.jline.JLineCompletion source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.812s][info][class,load] scala.tools.nsc.interpreter.shell.ReplCompletion source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.812s][info][class,load] scala.tools.nsc.interpreter.shell.ILoop$$anon$4 source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.812s][info][class,load] scala.tools.nsc.interpreter.shell.ReplCompletion$ source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar
[4.812s][info][class,load] scala.tools.nsc.interpreter.shell.ReplCompletion$Request source: /home/amarki/scala-2.13.0/lib/scala-compiler.jar

But after spending a few minutes trying to remember the pieces on the garage floor, I remembered someone started a PR for JLine3, where this doesn't seem to be an issue. (Obviously, I don't remember what changed. Maybe everything.)

$ rm build/pack/lib/j*jar
$ ll build/pack/lib/
total 21800
drwxr-xr-x 2 amarki amarki     4096 Aug  8 20:33 ./
drwxr-xr-x 4 amarki amarki     4096 Aug  8 20:30 ../
-rw-r--r-- 1 amarki amarki  2155199 Aug  8 20:30 scala-compiler-doc.jar
-rw-r--r-- 1 amarki amarki 10222359 Aug  8 20:30 scala-compiler.jar
-rw-r--r-- 1 amarki amarki  5541428 Aug  8 20:28 scala-library.jar
-rw-r--r-- 1 amarki amarki   522730 Aug  8 20:29 scalap.jar
-rw-r--r-- 1 amarki amarki  3542401 Aug  8 20:28 scala-reflect.jar
-rw-r--r-- 1 amarki amarki   323513 Aug  8 20:30 scala-repl-frontend.jar
$ skala -Xjline:off 
Welcome to Scala 2.13.1 (OpenJDK 64-Bit Server VM 11.0.3)

scala> 42
res0: Int = 42

[unkarjedy]

related issue: https://youtrack.jetbrains.com/issue/SCL-15818

that link doesn't work for me? do you mean https://youtrack.jetbrains.com/issue/SCL-15948 ?

yes, the original link is to our internal ticket, I've added this link to the description

[unkarjedy]

@unkarjedy scala/docs.scala-lang#1486 looks like a report from @tahsmith of this same issue. is there any workaround?

The workaround is adding jline 2.14.6 to the classpath
(you can try another version, the main part is that requried classes are located in the same packages).

If you use IntelliJ non-sbt (JPS) project you should add JLine in your SDK.

We will automate this workaround soon in one of the bugfix releases of Scala Plugin

[SethTisue]

this is fixed for 2.13.2

java -classpath scala-compiler.jar:scala-library.jar \
  scala.tools.nsc.MainGenericRunner -usejavacp -Xjline:off

results in a working REPL.

[unkarjedy]

just for the record, -Xnojline now also works

[som-snytt]

scala/scala#8906 restores the flag under deprecation.

This is like when the company rehires you because they forgot to make you train your replacement.