A symmetric meta programming framework #3634
Conversation
There was a problem hiding this comment.
Hello, and thank you for opening this PR! 🎉
All contributors have signed the CLA, thank you! ❤️
Commit Messages
We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).
Please stick to these guidelines for commit messages:
- Separate subject from body with a blank line
- When fixing an issue, start your commit message with
Fix #<ISSUE-NBR>:- Limit the subject line to 72 characters
- Capitalize the subject line
- Do not end the subject line with a period
- Use the imperative mood in the subject line ("Added" instead of "Add")
- Wrap the body at 80 characters
- Use the body to explain what and why vs. how
adapted from https://chris.beams.io/posts/git-commit
Have an awesome day! ☀️
|
I will integrate the TASTY reification from the prototype in https://github.com/dotty-staging/dotty/tree/tasty-quote |
| @@ -49,7 +49,8 @@ class Compiler { | |||
| List(new LinkAll), // Reload compilation units from TASTY for library code (if needed) | |||
| List(new FirstTransform, // Some transformations to put trees into a canonical form | |||
| new CheckReentrant, // Internal use only: Check that compiled program has no data races involving global vars | |||
| new ElimJavaPackages), // Eliminate syntactic references to Java packages | |||
| new ElimPackagePrefixes, // Eliminate syntactic references to Java packages | |||
| new ReifyQuotes), // Eliminate syntactic references to Java packages | |||
There was a problem hiding this comment.
We might need to do this before FirstTranform, when we will pickle the contents of the quotes we should probably give that pickler a tree that is in the same shape as the tree was in Pickler.
There was a problem hiding this comment.
FirstTransforms does some simplifications. It eliminates NamedArgs and Imports and replaces all trees representing types by TypeTree nodes. This is arguably good, since it simplifies the trees to reify quite a bit. The reason we keep the more complicated trees in the TASTY is for the IDE, so that all elements of source files are represented. But for macros and staging that should not be necessary.
There was a problem hiding this comment.
On second thought, I now see you were right. It does become easier if we reify exactly in the pickled format.
tests/pos/quoteTest.scala
Outdated
| @@ -0,0 +1,15 @@ | |||
| import scala.meta._ | |||
There was a problem hiding this comment.
Is the plan to use this for the public API or only tests? scala.meta.{Tree,Type,Term,...} are already defined in https://github.com/scalameta/scalameta/blob/v2.1.3/scalameta/trees/shared/src/main/scala/scala/meta/Trees.scala and get ~50,000 monthly downloads.
There was a problem hiding this comment.
We could use scala.meta.quote instead.
There was a problem hiding this comment.
We could use scala.meta.quote instead.
Good idea.
There was a problem hiding this comment.
Still seems a bit too easy to confuse with scala.meta. How about one of: scala.quote, scala.reify, scala.staging
There was a problem hiding this comment.
How about scala.quoted? staging and reify is both expert-speak.
There was a problem hiding this comment.
Also, it's a stretch to call the macro part of this "staging".
|
test performance please |
|
performance test scheduled: 1 job(s) in queue, 1 running. |
|
Performance test finished successfully: Visit http://dotty-bench.epfl.ch/3634/ to see the changes. Benchmarks is based on merging with master (cb758bd) |
odersky
force-pushed
the
add-meta
branch
2 times, most recently
from
9cd6432 to
7a1e933
Compare
|
I found a small blocker when inlining quotes in separate compilations object Macros {
inline def f(): Unit = '(1)
}class Test {
Macros.f()
}We are probably missing a case in the |
|
@odersky, @smarter, @OlivierBlanvillain, @allanrenucci @liufengyun we need to rebase all open all PRs on top of #3665. Without this, the CI will try to execute a test command that doesn't exist anymore. |
tests/pos/liftable.scala
Outdated
| @@ -17,7 +17,7 @@ object Test { | |||
| if (b) '(true) else '(false) | |||
| } | |||
|
|
|||
| implicit def ListIsQuotable[T: Type: Quotable]: Quotable[List[T]] = new { | |||
| implicit def ListIsQuotable[T: Quotable]: Quotable[List[T]] = new { | |||
There was a problem hiding this comment.
Quotable -> Liftable
In several places of this file
There was a discrepancy in that value parameters had a Deferred flag set before pickling but not after unpickling. This triggered a check that the (missing) rhs of an inline parameter was a constant. This commit changes the condition of the test to also exclude parameters. It also aligns frontend and unpickler in that the frontend will no longer mark term parameters as Deferred.
Previous condition did not take into account private static members. Their "enclosing class" is the enclosing package, but we really want to count them as static members of their lexical enclosing class here.
With the cleaned-up local scope handling we don't need them anymore.
Allow `~ stableId` as a type, where `stableId` is a stable term identifier. This is not the same as the second part of SIP 33, which proposed `~` as an operator over types.
Just came by this accidentally and noticed that it could be more idiomatic and terminology could be improved.
That's a better description of what the miniphase does.
Can be used like this:
<compute>
.reporting(res => println(s"result is $res"))
To achieve this previously, we'd need to store result in a val and then
return the val. This is more effort, in particular if we want to just quickly
add some debug output, to be removed later.
No phase consistency principle yet, we compute just the basic types.
The body of reify is still missing, for @nicolasstucki can fill in. For the moment, we just replace the tree to reify with its "_.show" output.
This is still somewhere between a paper draft and true reference docs. But since it's the only thing we have right now, it's worthwhile to integrate it now and change it later.
Rename ' to a unicode quote character so that syntax highlighters don't get confused.
Also, add AsFunction implicit class to Expr.
| # Symmetric Meta Programming | ||
|
|
||
| Symmetric meta programming is a new framework for staging and certain | ||
| forms of macros. It is is expressed as strongly and statically typed |
|
|
||
| object Macros { | ||
|
|
||
| def mapImpl[T, U](u: Type[U], arr: Expr[Array[T]], op: Expr[T => U])(implicit ctx: Context): Expr[Array[U]] = ’{ |
There was a problem hiding this comment.
We do not need the implicit context for this one.
| # Symmetric Meta Programming | ||
|
|
||
| Symmetric meta programming is a new framework for staging and for some | ||
| forms of macros. It is is expressed as strongly and statically typed |
| takes expressions of type `Expr[T]` to expressions of type `T` and it | ||
| takes expressions of type `Type[T]` to types `T`. | ||
|
|
||
| The two types can be are defined in package `scala.quoted` as follows: |
| The phase consistency principle can be motivated as follows: First, | ||
| suppose the result of a program `P` is some quoted text `’{ ... x | ||
| ... }` that refers to a free variable `x` in `P` This can be | ||
| represented only by referring to original the variable `x`. Hence, the |
There was a problem hiding this comment.
referring to the original the variable
|
|
||
| Here’s an application of `map` and how it rewrites to optimized code: | ||
|
|
||
| genSeq[Int]().map(x => x + 1) |
There was a problem hiding this comment.
There are some inconsistencies: you map with x => x + 1 here but with x => x * x below
| `Macro.assertImpl` becomes phase correct even if macro library and | ||
| program are conceptualized as local definitions. | ||
|
|
||
| But what about the call from `assert` to `assertImpl? Here, we need a |
There was a problem hiding this comment.
Missing ` after asserImpl
| contains a splice operation outside an enclosing quote is called a | ||
| _macro_. Macros are supposed to be expanded in a subsequent phase, | ||
| i.e. in a quoted context. Therefore, they are also type checked as if | ||
| they were in a quoted context, For instance, the definition of |
| representation of `Expr` trees. For instance, here is a possible | ||
| instance of `Liftable[Boolean]`: | ||
|
|
||
| implicit def BooleanIsLiftable: Liftable[Boolean] = new { |
| possible implementation of `Liftable[Int]` that does not use the underlying | ||
| tree machinery: | ||
|
|
||
| implicit def IntIsLiftable: Liftable[Int] = new { |
There was a problem hiding this comment.
no, that's actually legal syntax now.
| Since `Liftable` is a type class, instances can be conditional. For instance | ||
| a `List` is liftable if its element type is: | ||
|
|
||
| implicit def ListIsLiftable[T: Liftable]: Liftable[List[T]] = new { |
|
|
||
| This would allow constructing applications from lists of arguments | ||
| without having to match the arguments one-by-one with the | ||
| corresponding formal parameter types of the function. We need then "at |
| trees. So we should wait with this addition until we have more | ||
| use-cases that help us decide whether the loss in type-safety is worth | ||
| the gain in flexibility. In this context, it seems that deconstructing types is | ||
| less error-prone than deconstructing tersm, so one might also |
|
|
||
| ~(’t) --> t | ||
|
|
||
| t --> t’ |
There was a problem hiding this comment.
We should avoid using both ' for quote and ' prime. Maybe we should use unstead if we use t --> t2.
odersky
changed the title
|
With separate compilation, this code produces an unexpected error message. import scala.quoted._
object Macros {
inline def assert(expr: => Boolean): Unit = ~ assertImpl('(expr))
def assertImpl(expr: Expr[Boolean]) = '{ () }
}class Test {
import Macros._
val x = 1
assert(x != 0)
}8 |
| ^^^^
| value of type scala.quoted.Type[Boolean] does not take parameters |
| @@ -225,7 +227,7 @@ object TastyFormat { | |||
|
|
|||
| final val header = Array(0x5C, 0xA1, 0xAB, 0x1F) | |||
| val MajorVersion = 2 | |||
| val MinorVersion = 0 | |||
| val MinorVersion = 1 | |||
There was a problem hiding this comment.
We do not need this since we have not published 2.0 yet
There was a problem hiding this comment.
No sure what you mean by "published". We do have nightly builds out that use 2.0 and we should bump the version when we change something because it's good practice and because it's valuable information to anyone wanting to make a separate implementation of tasty.
| /** A Macrotransform that maintains the necessary infrastructore to support | ||
| * contxtual implicit searches (type-scope implicits are supported anyway). | ||
| */ | ||
| abstract class MacroTransformWithImplicits extends MacroTransform { |
There was a problem hiding this comment.
Should this really go in if we do not use it? What are the other use cases where it might be needed?
There was a problem hiding this comment.
someone might want to fix the typos in the doc-comment, too...
Otherwise unpickler gets confused when it tries to resolve references to these methods.
It's easier to handle this in PostTyper since then we do not need to distinguish between inlined and normal code. Also, need to record quotes as well as splices since ReifyQuotes should also do macro expansion. Also, add a missing recursive call to `transform` in ReifyQuotes.
This currently fails with "splice outside quote". Once macro expansion is implemented it should compile.
| @@ -1086,7 +1074,6 @@ class Typer extends Namer with TypeAssigner with Applications with Implicits wit | |||
| } | |||
|
|
|||
| def typedQuote(tree: untpd.Quote, pt: Type)(implicit ctx: Context): Tree = track("typedQuote") { | |||
| ctx.compilationUnit.containsQuotes = true | |||
There was a problem hiding this comment.
We do need ctx.compilationUnit.containsQuotesOrSplices = true in case there are only quotes in the tree. I can add the fix with the test case in #3662.
| import scala.quoted._ | ||
| object Macros { | ||
| inline def assert(expr: => Boolean): Unit = ~ assertImpl('(expr)) | ||
| def assertImpl(expr: Expr[Boolean]) = '{ () } |
There was a problem hiding this comment.
Indeed, this one works fine as a run test in #3662
We did not use lazy evaluation for static objects, because these are usually evauated lazily by the backend. But that does not hold if the static module is an (accessor) def instead of a val.
Fix #3624: Lazily evaluate static module accessors
A first implementation of the compiler parts of symmetric meta programming.
Still to be done in separate PRs:
Liftablenativelyquotedreside?