fixup line number reporting and simplify implementation: part 1 (mpollmeier#157)

Part 1 (this PR): predef files. Prior to this we concatenated all predef
files
and handed that over to the REPL/script/embedded driver for
interpretation.
That worked, but the line numbers were wrong, and the implementation
more
complicated than necessary. 

A followup PR will address the issue for scripts as well. That's
somewhat
different because we need to embed the script content in some wrapper
code..

Author: mpollmeier

README.md

@@ -29,6 +29,7 @@ Prerequisite: jdk11+
   * [Operators: Redirect to file, pipe to external command](#operators-redirect-to-file-pipe-to-external-command)
   * [Add dependencies with maven coordinates](#add-dependencies-with-maven-coordinates)
   * [Importing additional script files interactively](#importing-additional-script-files-interactively)
+  * [Adding classpath entries](#adding-classpath-entries)
   * [Rendering of output](#rendering-of-output)
   * [Exiting the REPL](#exiting-the-repl)
   * [Looking up the current terminal width](#looking-up-the-current-terminal-width)
@@ -38,7 +39,7 @@ Prerequisite: jdk11+
   * [Importing files / scripts](#importing-files--scripts)
   * [Dependencies](#dependencies)
   * [@main entrypoints](#main-entrypoints)
-  * [multiple @main entrypoints: test-main-multiple.sc](#multiple-main-entrypoints-test-main-multiplesc)
+  * [multiple @main entrypoints](#multiple-main-entrypoints)
   * [named parameters](#named-parameters)
 - [Additional dependency resolvers and credentials](#additional-dependency-resolvers-and-credentials)
   * [Attach a debugger (remote jvm debug)](#attach-a-debugger-remote-jvm-debug)
@@ -60,7 +61,6 @@ Prerequisite: jdk11+
   * [Updating the shaded libraries](#updating-the-shaded-libraries)
 - [Fineprint](#fineprint)  
 
-  
 ## Benefits over / comparison with
 
 ### Regular Scala REPL
@@ -136,7 +136,7 @@ All operators use the same pretty-printing that's used within the REPL, i.e. you
 ```scala
 scala> case class PrettyPrintable(s: String, i: Int)
 scala> PrettyPrintable("two", 2) #> "out.txt"
-// out.txt now contains `PrettyPrintable(s = "two", i = 2)`
+// out.txt now contains `PrettyPrintable(s = "two", i = 2)` - in pretty colors
 ```
 
 The operators have a special handling for two common use cases that are applied at the root level of the object you hand them: list- or iterator-type objects are unwrapped and their elements are rendered in separate lines, and Strings are rendered without the surrounding `""`. Examples:
@@ -182,6 +182,7 @@ val res0: Int = 1
 For Scala dependencies use `::`:
 ```
 srp --dep com.michaelpollmeier::colordiff:0.36
+
 colordiff.ColorDiff(List("a", "b"), List("a", "bb"))
 // color coded diff
 ```
@@ -196,9 +197,8 @@ echo 'val bar = "foo"' > myScript.sc
 
 srp
 
-val foo = 1
 //> using file myScript.sc
-println(bar) //1
+println(bar) //foo
 ```
 
 You can specify the filename with relative or absolute paths:
@@ -280,26 +280,17 @@ val res0: util.Try[Int] = Success(value = 202)
 See [ScriptRunnerTest](core/src/test/scala/replpp/scripting/ScriptRunnerTest.scala) for a more complete and in-depth overview.
 
 ### Simple "Hello world" script
-test-simple.sc
-```scala
-println("Hello!")
-"i was here" #> "out.txt"
-```
-
 ```bash
+echo 'println("Hello!")' > test-simple.sc
+
 srp --script test-simple.sc
 cat out.txt # prints 'i was here'
 ```
 
 ### Predef file(s) used in script
-test-predef.sc
-```scala
-println(foo)
-```
-
-test-predef-file.sc
-```scala
-val foo = "Hello, predef file"
+```bash
+echo 'println(foo)' > test-predef.sc
+echo 'val foo = "Hello, predef file"' > test-predef-file.sc
 ```
 
 ```bash
@@ -308,76 +299,62 @@ srp --script test-predef.sc --predef test-predef-file.sc
 To import multiple scripts, you can specify this parameter multiple times.
 
 ### Importing files / scripts
-foo.sc:
-```scala
-val foo = 42
-```
+```bash
+echo 'val foo = 42' > foo.sc
 
-test.sc:
-```scala
-//> using file foo.sc
-println(foo)
-```
+echo '//> using file foo.sc
+println(foo)' > test.sc
 
-```bash
 srp --script test.sc
 ```
 
 ### Dependencies
 Dependencies can be added via `//> using dep` syntax (like in scala-cli).
 
-test-dependencies.sc:
-```scala
-//> using dep com.michaelpollmeier:versionsort:1.0.7
+```bash
+echo '//> using dep com.michaelpollmeier:versionsort:1.0.7
 
 val compareResult = versionsort.VersionHelper.compare("1.0", "0.9")
 assert(compareResult == 1,
        s"result of comparison should be `1`, but was `$compareResult`")
-```
+' > test-dependencies.sc
 
-```bash
 srp --script test-dependencies.sc
 ```
 
 Note: this also works with `using` directives in your predef code - for script and REPL mode.
 
 ### @main entrypoints
-test-main.sc
-```scala
-@main def main() = println("Hello, world!")
-```
-
 ```bash
+echo '@main def main() = println("Hello, world!")' > test-main.sc
+
 srp --script test-main.sc
 ```
 
-### multiple @main entrypoints: test-main-multiple.sc
-```scala
+### multiple @main entrypoints
+```bash
+echo '
 @main def foo() = println("foo!")
 @main def bar() = println("bar!")
-```
+' > test-main-multiple.sc
 
-```bash
 srp --script test-main-multiple.sc --command foo
 ```
 
 ### named parameters
-test-main-withargs.sc
-```scala
+```bash
+echo '
 @main def main(first: String, last: String) = {
   println(s"Hello, $first $last!")
-}
-```
+} ' > test-main-withargs.sc
 
-```bash
 srp --script test-main-withargs.sc --param first=Michael --param last=Pollmeier
 ```
 If your parameter value contains whitespace, just wrap it quotes so that your shell doesn't split it up, e.g. `--param "text=value with whitespace"`
 
-Note that on windows the parameters need to be triple-quoted in any case:
+On windows the parameters need to be triple-quoted in any case:
 `srp.bat --script test-main-withargs.sc --param """first=Michael""" --param """last=Pollmeier"""`
 
-
 ## Additional dependency resolvers and credentials
 Via `--repo` parameter on startup:
 ```bash
@@ -388,13 +365,13 @@ To add multiple dependency resolvers, you can specify this parameter multiple ti
 
 Or via `//> using resolver` directive as part of your script or predef code:
 
-script-with-resolver.sc
-```scala
+```bash
+echo '
 //> using resolver https://repo.gradle.org/gradle/libs-releases
 //> using dep org.gradle:gradle-tooling-api:7.6.1
 println(org.gradle.tooling.GradleConnector.newConnector())
-```
-```scala
+' > script-with-resolver.sc
+
 srp --script script-with-resolver.sc
 ```
 

build.sbt

@@ -1,4 +1,4 @@
-name := "root"
+name := "scala-repl-pp-root"
 publish/skip := true
 
 ThisBuild / organization := "com.michaelpollmeier"

core/src/main/scala/replpp/Config.scala

@@ -29,6 +29,9 @@ case class Config(
     if (nocolors) Colors.BlackWhite
     else Colors.Default
 
+  def withAdditionalClasspathEntry(entry: Path): Config =
+    copy(classpathConfig = classpathConfig.withAdditionalClasspathEntry(entry))
+
   /** inverse of `Config.parse` */
   lazy val asJavaArgs: Seq[String] = {
     val args = Seq.newBuilder[String]
@@ -272,7 +275,10 @@ object Config {
                           inheritClasspathIncludes: Seq[String] = ForClasspath.DefaultInheritClasspathIncludes,
                           inheritClasspathExcludes: Seq[String] = Seq.empty,
                           dependencies: Seq[String] = Seq.empty,
-                          resolvers: Seq[String] = Seq.empty)
+                          resolvers: Seq[String] = Seq.empty) {
+    def withAdditionalClasspathEntry(entry: Path): ForClasspath =
+      copy(additionalClasspathEntries = additionalClasspathEntries :+ util.pathAsString(entry))
+  }
 
 
   object ForClasspath {

core/src/main/scala/replpp/Dependencies.scala

@@ -24,7 +24,7 @@ object Dependencies {
     val repositoryArgs = additionalRepositories.flatMap { repo =>
       Seq("--repository", repo)
     }
-    val command = Seq("java", "-jar", coursierJarPath.toAbsolutePath.toString, "fetch") ++ repositoryArgs ++ coordinates
+    val command = Seq("java", "-jar", util.pathAsString(coursierJarPath), "fetch") ++ repositoryArgs ++ coordinates
     if (verbose) println(s"executing `${command.mkString(" ")}`")
 
     Try(os.proc(command).call()) match {

core/src/main/scala/replpp/DottyReplDriver.scala

@@ -44,8 +44,6 @@ import scala.language.implicitConversions
 import scala.util.control.NonFatal
 import scala.util.Using
 
-import DottyRandomStuff.newStoreReporter
-
 /** Based on https://github.com/lampepfl/dotty/blob/3.4.2/compiler/src/dotty/tools/repl/ReplDriver.scala
  * Main REPL instance, orchestrating input, compilation and presentation
  * */
@@ -60,14 +58,14 @@ class DottyReplDriver(settings: Array[String],
   override def sourcesRequired: Boolean = false
 
   /** Create a fresh and initialized context with IDE mode enabled */
-  private def initialCtx(settings: List[String]) = {
+  private def initialCtx(settings: List[String]): Context = {
     val rootCtx = initCtx.fresh.addMode(Mode.ReadPositions | Mode.Interactive)
     rootCtx.setSetting(rootCtx.settings.YcookComments, true)
     rootCtx.setSetting(rootCtx.settings.YreadComments, true)
     setupRootCtx(this.settings ++ settings, rootCtx)
   }
 
-  private def setupRootCtx(settings: Array[String], rootCtx: Context) = {
+  private def setupRootCtx(settings: Array[String], rootCtx: Context): Context = {
     setup(settings, rootCtx) match
       case Some((files, ictx)) => inContext(ictx) {
         shouldStart = true
@@ -224,7 +222,7 @@ class DottyReplDriver(settings: Array[String],
     else op
   }
 
-  private def newRun(state: State, reporter: StoreReporter = newStoreReporter) = {
+  private def newRun(state: State, reporter: StoreReporter = DottyRandomStuff.newStoreReporter) = {
     val run = compiler.newRun(rootCtx.fresh.setReporter(reporter), state)
     state.copy(context = run.runContext)
   }

core/src/main/scala/replpp/InteractiveShell.scala

@@ -5,20 +5,23 @@ import dotty.tools.repl.State
 import scala.util.control.NoStackTrace
 
 object InteractiveShell {
-  def run(config: Config): Unit = {
 
-    val predefCode = allPredefCode(config)
-    val compilerArgs = replpp.compilerArgs(config)
+  def run(config: Config): Unit = {
     import config.colors
+    val config0 = precompilePredefFiles(config)
+
+    val compilerArgs = replpp.compilerArgs(config0)
+
     val replDriver = new ReplDriver(
       compilerArgs,
-      onExitCode = config.onExitCode,
-      greeting = config.greeting,
-      prompt = config.prompt.getOrElse("scala"),
-      maxHeight = config.maxHeight
+      onExitCode = config0.onExitCode,
+      greeting = config0.greeting,
+      prompt = config0.prompt.getOrElse("scala"),
+      maxHeight = config0.maxHeight
     )
 
     val initialState: State = replDriver.initialState
+    val predefCode = DefaultPredef
     val state: State = {
       if (verboseEnabled(config)) {
         println(s"compiler arguments: ${compilerArgs.mkString(",")}")
@@ -35,4 +38,5 @@ object InteractiveShell {
 
     replDriver.runUntilQuit(using state)()
   }
-}
+  
+}

core/src/main/scala/replpp/ReplDriver.scala

@@ -11,14 +11,14 @@ import scala.annotation.tailrec
 import scala.jdk.CollectionConverters.*
 import scala.util.{Failure, Success, Try}
 
-class ReplDriver(args: Array[String],
+class ReplDriver(compilerArgs: Array[String],
                  out: PrintStream = scala.Console.out,
                  onExitCode: Option[String] = None,
                  greeting: Option[String],
                  prompt: String,
                  maxHeight: Option[Int] = None,
                  classLoader: Option[ClassLoader] = None)(using Colors)
-  extends ReplDriverBase(args, out, maxHeight, classLoader) {
+  extends ReplDriverBase(compilerArgs, out, maxHeight, classLoader) {
 
   /** Run REPL with `state` until `:quit` command found
     * Main difference to the 'original': different greeting, trap Ctrl-c

core/src/main/scala/replpp/ReplDriverBase.scala

@@ -1,33 +1,17 @@
 package replpp
 
-import dotty.tools.MainGenericCompiler.classpathSeparator
-import dotty.tools.dotc.Run
-import dotty.tools.dotc.ast.{Positioned, tpd, untpd}
-import dotty.tools.dotc.classpath.{AggregateClassPath, ClassPathFactory}
-import dotty.tools.dotc.config.{Feature, JavaPlatform, Platform}
-import dotty.tools.dotc.core.Comments.{ContextDoc, ContextDocstrings}
-import dotty.tools.dotc.core.Contexts.{Context, ContextBase, ContextState, FreshContext, ctx, explore}
-import dotty.tools.dotc.core.{Contexts, MacroClassLoader, Mode, TyperState}
-import dotty.tools.io.{AbstractFile, ClassPath, ClassRepresentation}
 import dotty.tools.repl.*
-import org.jline.reader.*
 import replpp.shaded.fansi
 
 import java.io.PrintStream
 import java.lang.System.lineSeparator
-import java.net.URL
 import java.nio.file.{Files, Path}
-import javax.naming.InitialContext
-import scala.annotation.tailrec
-import scala.collection.mutable
-import scala.jdk.CollectionConverters.*
-import scala.util.{Failure, Success, Try}
 
-abstract class ReplDriverBase(args: Array[String],
+abstract class ReplDriverBase(compilerArgs: Array[String],
                               out: PrintStream,
                               maxHeight: Option[Int],
                               classLoader: Option[ClassLoader])(using Colors)
-  extends DottyReplDriver(args, out, maxHeight, classLoader) {
+  extends DottyReplDriver(compilerArgs, out, maxHeight, classLoader) {
 
   protected def interpretInput(lines: IterableOnce[String], state: State, currentFile: Path): State = {
     val parsedLines = Seq.newBuilder[String]

core/src/main/scala/replpp/UsingDirectives.scala

@@ -13,7 +13,7 @@ object UsingDirectives {
   def findImportedFilesRecursively(path: Path, visited: Set[Path] = Set.empty): Seq[Path] = {
     val rootDir: Path =
       if (Files.isDirectory(path)) path
-      else path.getParent
+      else path.toAbsolutePath.getParent
 
     val importedFiles = findImportedFiles(util.linesFromFile(path), rootDir)
     val recursivelyImportedFiles = importedFiles.filterNot(visited.contains).flatMap { file =>
@@ -24,7 +24,7 @@ object UsingDirectives {
 
   def findImportedFiles(lines: IterableOnce[String], rootPath: Path): Seq[Path] =
     scanFor(FileDirective, lines).iterator.map(resolveFile(rootPath, _)).toSeq
-    
+
   def findImportedFilesRecursively(lines: IterableOnce[String], rootPath: Path): Seq[Path] = {
     val results = Seq.newBuilder[Path]
     val visited = mutable.Set.empty[Path]