Skip to content
This repository was archived by the owner on Sep 1, 2020. It is now read-only.

Fix .check files for SIP-23 #87

Merged
merged 70 commits into from
Nov 17, 2014
Merged

Fix .check files for SIP-23 #87

merged 70 commits into from
Nov 17, 2014

Conversation

folone
Copy link

@folone folone commented Nov 16, 2014

This should fix what #86 has broken. /cc @propensive

som-snytt and others added 30 commits November 16, 2014 15:05
People expect to change the class path midstream.

Let's disabuse them by removing the broken command.

The internals are deprecated.
The reset and replay commands take arbitrary command line args.
When settings args are supplied, the compiler is recreated.

For uniformity, the settings command performs only the usual
arg parsing: use -flag:true instead of +flag, and clearing a
setting is promoted to the command line, so that -Xlint: is not
an error but clears the flags.

```
scala> maqicode.Test main null
<console>:8: error: not found: value maqicode
              maqicode.Test main null
              ^

scala> :reset -classpath/a target/scala-2.11/sample_2.11-1.0.jar
Resetting interpreter state.
Forgetting all expression results and named terms: $intp

scala> maqicode.Test main null
Hello, world.

scala> val i = 42
i: Int = 42

scala> s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.

scala> :replay -classpath ""
Replaying: maqicode.Test main null
Hello, world.

Replaying: val i = 42
i: Int = 42

Replaying: s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.

scala> :replay -classpath/a ""
Replaying: maqicode.Test main null
<console>:8: error: not found: value maqicode
              maqicode.Test main null
              ^

Replaying: val i = 42
i: Int = 42

Replaying: s"$i is the loneliest numbah."
res1: String = 42 is the loneliest numbah.
```

Clearing a clearable setting:
```
scala> :reset -Xlint:missing-interpolator
Resetting interpreter state.

scala> { val i = 42 ; "$i is the loneliest numbah." }
<console>:8: warning: possible missing interpolator: detected interpolated identifier `$i`
              { val i = 42 ; "$i is the loneliest numbah." }
                             ^
res0: String = $i is the loneliest numbah.

scala> :reset -Xlint:
Resetting interpreter state.
Forgetting this session history:

{ val i = 42 ; "$i is the loneliest numbah." }

scala> { val i = 42 ; "$i is the loneliest numbah." }
res0: String = $i is the loneliest numbah.

```
* Errors are red
* Warnings are yellow
Added some documentation explaining what the role of `end` is.
Under load on Jenkins, we've been seeing:

```
% diff /localhome/jenkins/a/workspace/scala-nightly-auxjvm-2.12.x/jdk/jdk7/label/auxjvm/test/files/run/t4542-run.log /localhome/jenkins/a/workspace/scala-nightly-auxjvm-2.12.x/jdk/jdk7/label/auxjvm/test/files/run/t4542.check
@@ -2,75 +2,14 @@ Type in expressions to have them evaluated.
 Type :help for more information.

 scala> @deprecated("foooo", "ReplTest version 1.0-FINAL") class Foo() {
 java.util.concurrent.TimeoutException: Futures timed out after [60 seconds]
 	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
 	at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
 	at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
 	at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:95)
 	at scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
 	at scala.concurrent.Await$.ready(package.scala:95)
 	at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:431)
 	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:457)
 	at scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:875)
```

This commit bumps the timeout up be a factor of ten to try to
restore that comforting green glow to https://scala-webapps.epfl.ch/jenkins/view/2.N.x
Given that we'll switch to GenBCode in 2.12, the test case
showing the bug is fixed under that option is all I plan to
offer for this bug.

The flags file contains `-Ynooptimize` to stay locked into
`GenBCode`.
Previously, abstract type members were allowed in objects only when inherited,
but not when declared directly. This inconsistency was not intended. In dotty,
abstract type members are allowed in values and represent existentials; so upon
discussion, it was decided to fix things to conform to dotty and allow such type
members. Adriaan also asked to keep rejecting abstract type members in methods
even though they would conceivably make sense.

Discussions happened on scala#3407, scala/scala-dist#127.
This code is improved from scala#3442, keeps closer to the current logic, and passes tests.

Existing tests that have been converted to `pos` tests show that
this works, and a new test has been added to show that local
aliases (ie term-owned) without a RHS are still rejected.
... which can be introduced by `memberType` for methods
with parameter types dependent on class type parameters.

Here's an example of such a type:

```
scala> class Bippy { trait Foo[A] }
defined class Bippy

scala> final class RichBippy[C <: Bippy with Singleton](val c1: C) {
     |   def g[A](x: A)(ev: c1.Foo[A]): Int = 2
     | }
defined class RichBippy

scala> :power
** Power User mode enabled - BEEP WHIR GYVE **
** :phase has been set to 'typer'.          **
** scala.tools.nsc._ has been imported      **
** global._, definitions._ also imported    **
** Try  :help, :vals, power.<tab>           **

scala> val g = typeOf[RichBippy[_]].member(TermName("g"))
g: $r.intp.global.Symbol = method g

scala> val c = new Bippy
c: Bippy = Bippy@92e2c93

scala> val memberType = typeOf[RichBippy[c.type]].memberType(g)
memberType: $r.intp.global.Type = ([A](x: A)(ev: _7.c1.Foo[A])Int) forSome { val _7: RichBippy[c.type] }

```

In this example, if we were to typecheck the selection
`new RichBippy[c.type].g` that existential type would be short lived.

Consider this approximation of `Typer#typedInternal`:

```scala
val tree1: Tree = typed1(tree, mode, ptWild)
val result = adapt(tree1, mode, ptPlugins, tree)
```

Given that `tree1.tpe` is not an overloaded, adapt will find its
way to:

```
  case tp if mode.typingExprNotLhs && isExistentialType(tp) =>
    adapt(tree setType tp.dealias.skolemizeExistential(context.owner, tree), mode, pt, original)
```

Which would open the existential as per:

```
scala> memberType.skolemizeExistential
res2: $r.intp.global.Type = [A](x: A)(ev: _7.c1.Foo[A])Int
```

However, if do have overloaded alternatives, as in the test case,
we have to remember to call `adapt` again *after* we have picked
the winning alternative.

We actually don't have a centralised place where overload resolution
occurs, as the process differs depending on the context of the
selection. (Are there explicit type arguments? Inferred type
arguments? Do we need to use the expected type to pick a winner?)

This commit finds the existing places that call adapt after
overloade resolution and routes those calls through a marker
method. It then adds one more call to this in `inferPolyAlternatives`,
which fixes the bug.
This reverts commit 8986ee4.

Scaladoc seems to work reliably for 2.11.x. We are using it in the IDE builds and haven't noticed any
flakiness, so we'd like to get reinstate this test.
This pattern of code is typically a bug:

    if (f(tp.typeSymbol)) {
       g(tp.typeArgs)
    }

Intead, one needs to take the base type of `tp` wrt `tp.typeSymbol`.

This commit does exactly that when formatting the `@implicitNotFound`
custom error message.

Patch found on the back of an envelope in the handwriting of @adriaanm
Fixes non-determinism within the DPLL algorithm and disallows
infeasible counter examples directly in the formula.

The function to compute all solutions was flawed and thus only
returned a subset of the solutions. The algorithm would stop too soon
and thus depending on the ordering of the symbols return more or less
solutions. I also added printing a warning when the search was stopped
because the max recursion depth was reached. This is very useful as an
explanation of spuriously failing regression tests, since less counter
examples might be reported. In such a case the recursion depth should
be set to infinite by adding `-Ypatmat-exhaust-depth off`.

The mapping of the solutions of the DPLL algorithm to counter examples
has been adapted to take the additional solutions from the
solver into account:

Consider for example `t8430.scala`:

```Scala
sealed trait CL3Literal
case object IntLit extends CL3Literal
case object CharLit extends CL3Literal
case object BooleanLit extends CL3Literal
case object UnitLit extends CL3Literal

sealed trait Tree
case class LetL(value: CL3Literal) extends Tree
case object LetP extends Tree
case object LetC extends Tree
case object LetF extends Tree

object Test {
  (tree: Tree) => tree match {case LetL(CharLit) => ??? }
}
```

This test contains 2 domains,  `IntLit, CharLit, ...` and `LetL, LetP, ...`,
the corresponding formula to check exhaustivity looks like:

```
    V1=LetC.type#13    \/ V1=LetF.type#14 \/    V1=LetL#11     \/  V1=LetP.type#15    /\
 V2=BooleanLit.type#16 \/  V2=CharLit#12  \/ V2=IntLit.type#17 \/ V2=UnitLit.type#18  /\
      -V1=LetL#11      \/ -V2=CharLit#12  \/                   \/
```

The first two lines assign a value of the domain to the scrutinee (and
the correponding member in case of `LetL`) and prohibit the counter
example `LetL(CharLit)` since it's covered by the pattern match. The
used Boolean encoding allows that scrutinee `V1` can be equal to
`LetC` and `LetF` at the same time and thus, during enumeration of all
possible solutions of the formula, such a solution will be found,
since only one literal needs to be set to true, to  satisfy that
clause. That means, if at least one of the literals of such a clause
was in the `unassigned` list of the DPLL procedure, we will get
solutions where the scrutinee is equal to more than one element of the
domain.

A remedy would be to add constraints that forbid both literals
to be true at the same time. His is infeasible for big domains (see
`pos/t8531.scala`), since we would have to add a quadratic number of
clauses (one clause for each pair in the domain). A much simpler
solution is to just filter the invalid results. Since both values for
`unassigned` literals are explored, we will eventually find a valid
counter example.
to initial implementation).

Assuming that the DPLL procedure does not run into max recursion
depth, that workaround is not needed anymore, since the non-
determinism has been fixed.
Let the AbstractFileClassLoader override just the usual suspects.

Normal delegation behavior should ensue.

That's instead of overriding `getResourceAsStream`, which was intended
that "The repl classloader now works more like you'd expect a classloader to."
(Workaround for "Don't know how to construct an URL for something which exists
only in memory.")

Also override `findResources` so that `getResources` does the obvious thing,
namely, return one iff `getResource` does.

The translating class loader for REPL only special-cases `foo.class`: as
a fallback, take `foo` as `$line42.$read$something$foo` and try that class file.
That's the use case for "works like you'd expect it to."

There was a previous fix to ensure `getResource` doesn't take a class name.
The convenience behavior, that `classBytes` takes either a class name or a resource
path ending in ".class", has been promoted to `ScalaClassLoader`.
For matches with two or fewer cases, @switch is ignored. This should
not happen silently.
The pattern matcher phase (conceivably, among others) can generate
code that binds local `Ident`s symbolically, rather than according
to the lexical scope. This means that a lambda can capture more than
one local of the same name.

In the enclosed test case, this ends up creating the following
tree after delambdafy

    [[syntax trees at end of                delambdafy]] // delambday-patmat-path-dep.scala
    matchEnd4({
      case <synthetic> val x1: Object = (x2: Object);
      case5(){
        if (x1.$isInstanceOf[C]())
          {
            <synthetic> val x2#19598: C = (x1.$asInstanceOf[C](): C);
            matchEnd4({
              {
                (new resume$1(x2#19598, x2#19639): runtime.AbstractFunction0)
              };
              scala.runtime.BoxedUnit.UNIT
            })
          }
        else
          case6()
      };
      ...
    })
    ...
    <synthetic> class resume$1 extends AbstractFunction0 {
      <synthetic> var x2: C = null;
      <synthetic> var x2: C = null;
      ...
    }

After this commit, the var members of `resume$1` are given fresh
names, rather than directly using the name of the captured var:

    <synthetic> var x2$3: C = null;
    <synthetic> var x2$4: C = null;
Incorporate review comments by Som Snytt.
@serialversionuid is special-cased, the warning doesn't apply.

Related to SI-7041.
Note that I removed the check to ignore @deprecated:
- @deprecated extends StaticAnnotation, so they aren't
  supposed to show up in the RuntimeInvisibleAnnotation
  attribute anyway, and the earlier check for "extends
  ClassfileAnnotationClass" makes this check superflous
  anyway.
- Otherwise, if @deprecated was extending
  ClassfileAnnotationClass it would seem inconsistent
  that we don't emit @deprecated, but would do so for
  @deprecatedOverriding, @deprecatedInheritance, etc.
  Anyway, due to ClassfileAnnotation not working in
  Scala, and the additional check which only allows
  Java-defined annotations, this is pretty pointless
  from every perspective.
The transformation of applications to specialized methods
relies on the owner of said method having had the specialization
info transform run which stashes a bunch of related data into
per-run caches such as `SpecializeTypes#{typeEnv}`.

Recently, we found that per-run caches didn't quite live up
to there name, and in fact weren't being cleaned up before
a new run. This was remedied in 00e11ff.

However, no good deed goes unpunished, and this led to a
regression in specialization in the REPL and FSC.

This commit makes two changes:

  - change the specialization info tranformer to no longer
    directly enter specialized methods into the `info` of whatever
    the current phase happens to be. This stops them showing up
    `enteringTyper` of the following run.
  - change `adaptInfos` to simply discard all but the oldest
    entry in the type history when bringing a symbol from one
    run into the next. This generalizes the approach taken to
    fix SI-7801. The specialization info transformer will now
    execute in each run, and repopulate `typeEnv` and friends.

I see that we have a seemingly related bandaid for this sort
of problem since 08505bd. In a followup, I'll try to revert
that.
08505bd added a workaround for an undiagnosed interaction
between resident compilation and specialzation. This is no
longer needed after the previous commit.
Classic bait-and-switch: `isTupleType` dealiases, but `typeArgs` does not.
When deciding with `isTupleType`, process using `tupleComponents`.
Similar for other combos. We should really enforce this using extractors,
and only decouple when performance is actually impacted.
Inserted missing word "bounds".
Corrected the claimed outcome of the example in the section on repeated parameters. The example method sum sums up the _squares_ of the arguments.
Inserted two missing instances of the word "the".
Corrected "invokeDynamic" to "applyDynamic".
Elided superfluous "a".
Corrected "no" to "not".
lrytz and others added 22 commits November 16, 2014 15:08
When parsed from source, java annotation class symbol are lacking the
`@Retention` annotation. In mixed compilation, java annotations are
therefore emitted with visibility CLASS.

This patch conservatively uses the RUNTIME visibility in case there is
no @retention annotation.

The real solution is to fix the Java parser, logged in SI-8928.
The observed bug is probably solved (or should have a ticket), this 4-year-old thread dump does not seem to belong here.
This should happen every time we cut a new release. We don't touch
versions.properties files because that file specifies dependencies and
we don't want to depend on broken Scala 2.11.3 release.
  -Ywarn-unused-import: creates a bogues warning, see SI-7750

  -Ywarn-unused: creates a lot of noise & has bugs, see SI-7707, SI-7712
showCode used to print nothing when the only modifier was a change in
visibility scope (i.e. no flags but privateWithin is set).
Scala 2.11.4 release has been staged, we should start publishing snapshots
for Scala 2.11.5 now.
The code that "aligns" patterns and extractors assumes that it can
look at the type of the unapply method to figure the arity of the
extractor.

However, the result type of a whitebox macro does not tell the
whole story, only after expanding an application of that macro do
we know the result type.

During regular compilation, this isn't a problem, as the macro
application is expanded to a call to a synthetic unapply:

    {
      class anon$1 {
        def unapply(tree: Any): Option[(Tree, List[Treed])]
      }
      new anon$1
    }.unapply(<unapply selector>)

In the presentation compiler, however, we now use
`-Ymacro-expand:discard`, which expands macros only to compute
the type of the application (and to allow the macro to issue
warnings/errors). The original application is retained in the
typechecked tree, modified only by attributing the potentially-sharper
type taken from the expanded macro.

This was done to improve hyperlinking support in the IDE.

This commit passes `sel.tpe` (which is the type computed by the
macro expansion) to `unapplyMethodTypes`, rather than using
the `finalResultType` of the unapply method.

This is tested with a presentation compiler test (which closely
mimics the reported bug), and with a pos test that also exercises
`-Ymacro-expand:discard`. Prior to this patch, they used to fail with:

    too many patterns for trait api: expected 1, found 2
The test test/files/run/global-showdef.scala was outputting
to the cwd instead of the test output dir.

Good behavior is now inherited from DirectTest.

Test frameworks, of any ilk or capability, rock.
You can only show one class or object at a time,
but we can show one of each to reduce the compilations
for this test.

It seems the original issue happened because the test
started to create class files after SI-8217.

So, also stop compile after typer, because why stress the kitteh.
When we name an implicit class, `enterImplicitWrapper` is called,
which enters the symbol for the factory method into the
owning scope. The tree defining this factory method is stowed into
`unit.synthetics`, from whence it will be retrieved and incorporated
into the enclosing tree during typechecking (`addDerivedTrees`).
The entry in `unit.synthetics` is removed at that point.

However, in the presentation compiler, we can typecheck a unit
more than once in a single run. For example, if, as happens
in the enclosed test, a call to ask for a type at a given
position interrupts type checking of the entire unit, we
can get into a situation whereby the first type checking
invocation has consumed the entry from `unit.synthetics`,
and the second will crash when it can't find an entry.

Similar problems have been solved in the past in
`enterExistingSym` in the presentation compiler. This method
is called when the namer encounters a tree that already has
a symbol attached. See 0b78a01 / 148736c.

This commit takes a two pronged approach.

First, `enterExistingSym` is extended to handle implicit classes.
Any previous factory method in the owning scope is removed, and
`enterImplicitWrapper` is called to place a new tree for the factory
into `unit.synthetics` and to enter its symbol into the owning scope.

Second, the assertions that could be tripped in `addDerivedTrees`
and in `ImplicitClassWrapper#derivedSym` have been converted to
positioned errors.

The first change is sufficient to fix this bug, but the second
is also enough to make the enclosed test pass, and has been retained
as an extra layer of defence.
A retrospective test case which covers typechecking idempotency which
was introduced in 0b78a01 / 148736c. It also tests the
implicit class handling, which was fixed in the previous commit.

It is difficult to test this using existing presentation compiler
testing infrastructure, as one can't control at which point during
the first typechecking the subesquent work item will be noticed.

Instead, I've created a test with a custom subclass of
`interactive.Global` that allows precise, deterministic control
of when this happens. It overrides `signalDone`, which is called
after each tree is typechecked, and watches for a defintion with
a well known name. At that point, it triggers a targetted typecheck
of the tree marked with a special comment.

It is likely that this approach can be generalized to a reusable
base class down the track. In particular, I expect that some of
the nasty interactive ScalaDoc bugs could use this single-threaded
approach to testing the presentation compiler.
When a case class is type checked, synthetic methods are added,
such as the `hashCode`/`equals`, implementations of the `Product`
interface. At the same time, a case accessor method is added for
each non-public constructor parameter. This the accessor for a
parameter named `x` is named `x$n`, where `n` is a fresh suffix.

This is all done to retain universal pattern-matchability of
case classes, irrespective of access. What is the point of allowing
non-public parameters if pattern matching can subvert access? I
believe it is to enables private setters:

```
case class C(private var x: String)

scala> val x = new C("")
x: C = C()

scala> val c = new C("")
c: C = C()

scala> val C(x) = c
x: String = ""

scala> c.x
<console>:11: error: variable x in class C cannot be accessed in C
              c.x
                ^

scala> c.x = ""
<console>:13: error: variable x in class C cannot be accessed in C
val $ires2 = c.x
               ^
<console>:10: error: variable x in class C cannot be accessed in C
       c.x = ""
         ^
```

Perhaps I'm missing additional motivations.

If you think scheme sounds like a binary compatiblity nightmare,
you're right: https://issues.scala-lang.org/browse/SI-8944

`caseFieldAccessors` uses the naming convention to find the right
accessor; this in turn is used in pattern match translation.

The accessors are also needed in the synthetic `unapply` method
in the companion object. Here, we must tread lightly to avoid
triggering a typechecking cycles before; the synthesis of that method
is not allowed to force the info of the case class.

Instead, it uses a back channel, `renamedCaseAccessors` to see
which parameters have corresonding accessors.

This is pretty flaky: if the companion object is typechecked
before the case class, it uses the private param accessor directly,
which it happends to have access to, and which duly gets an
expanded name to allow JVM level access. If the companion
appears afterwards, it uses the case accessor method.

In the presentation compiler, it is possible to typecheck a source
file more than once, in which case we can redefine a case class. This
uses the same `Symbol` with a new type completer. Synthetics must
be re-added to its type.

The reported bug occurs when, during the second typecheck, an entry
in `renamedCaseAccessors` directs the unapply method to use `x$1`
before it has been added to the type of the case class symbol.

This commit clears corresponding entries from that map when we
detect that we are redefining a class symbol.

Case accessors are in need of a larger scale refactoring. But I'm
leaving that for SI-8944.
As suggested during code review, this test checks that
the tailcalls phase recurses appropriately below a method
that doesn and does not itself tail call. The test case is
based on the pattern of code that to trigger super-linear
performance in this transform.
Another excellent test suggestion by Dear Reviewer.

After tail calls, the transform results in:

```
def tick(i: Int): Unit = {
      <synthetic> val _$this: Test.type = Test.this;
      _tick(_$this: Test.type, i: Int){
        if (i.==(0))
  ()
else
  if (i.==(42))
    {
      Test.this.tick(0);
      _tick(Test.this, i.-(1).asInstanceOf[Int]())
    }
  else
    _tick(Test.this, i.-(1).asInstanceOf[Int]()).asInstanceOf[Unit]()
      }
    };
```

We test this by mostly exercising the tail-recursive code path with
a level of recursion that would overflow the stack if we weren't
using jumps.
The latest version of the Scala plugin for IntelliJ IDEA introduces
a new project structure

http://blog.jetbrains.com/scala/2014/10/30/scala-plugin-update-for-intellij-idea-14-rc-is-out/

Due to a bug (https://youtrack.jetbrains.com/issue/SCL-7753), you
currently need to install the latest nightly build from here:
http://confluence.jetbrains.com/display/SCA/Scala+Plugin+Nightly+Builds+for+Cassiopeia

The new format doesn't allow scala compiler options per-module, so
the `-sourcepath src/libarary` is used for all modules.
This requires -Xexperimental in 2.11.x.

Update spec.

TODO:
 - test!
 - include SIP as spec addendum
 - don't predicate on -Xexperimental -- introduce -Xsip:23?

Notes:
 - `()` is not a literal, neither are `Symbol`s
@folone
Copy link
Author

folone commented Nov 16, 2014

@propensive
Copy link

Thanks @folone!

propensive pushed a commit that referenced this pull request Nov 17, 2014
Fix .check files for SIP-23
@propensive propensive merged commit f308ab0 into typelevel:2.11.x Nov 17, 2014
@folone folone deleted the fix-sip23 branch November 17, 2014 08:49
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.