Skip to content

Commit 5172c69

Browse files
lrhnlrhn
authored
Add feature specification and implementation plan for small features for Q1 of 2021 (#1417)
1 parent 75d7d47 commit 5172c69

File tree

2 files changed

+270
-0
lines changed

2 files changed

+270
-0
lines changed

‎accepted/future-releases/small-features-21Q1/feature-specification.md

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Dart "Small Features" 21Q1
2+
3+
## Changelog
4+
5+
- 2021-02-03: Initial version
6+
7+
## Specification
8+
We have a [list](https://github.com/dart-lang/language/issues/1077) of smaller language enhancement features which are expected to be:
9+
10+
* Non-breaking.
11+
* Localized (no expected cross-cutting concerns with other features).
12+
* Already designed to a point where we expect no surprises.
13+
* Fairly easy and cheap to implement (little-to-no back-end work expected).
14+
15+
For the first quarter of 2021, we schedule three of these:
16+
17+
1. Allow type arguments on annotations ([#1297](https://github.com/dart-lang/language/issues/1297)).
18+
2. Allow generic function types as type arguments ([#496](https://github.com/dart-lang/language/issues/496)).
19+
3. Allow `>>>` as an overridable operator ([#120](https://github.com/dart-lang/language/issues/120)).
20+
21+
Allowing type arguments for annotations removes an unnecessary historical restriction on annotation syntax, and the VM team have a vested interest in the feature for use in `dart:ffi`. This feature was chosen because of a pressing need for it.
22+
23+
Allowing generic functions as type arguments is a restriction originally introduced as a precautionary measure, because it wasn't clear that the type system wouldn't become undecidable without it. We don't *think* that's a problem, and it's been a tripwire for people writing, e.g., lists of generic functions, where the type inference would infer a type argument that the compiler then reported as invalid. This feature was chosen because it is related to the previous change, and is expected to be very minor in scope.
24+
25+
Reintroducing the `>>>` operator was intended for Dart 2.0, but was repeatedly postponed as not important. We do want it for the unsigned shift of integers, and it's been mostly implemented on some of our platforms already. This feature was chosen because it is already half-way implemented.
26+
27+
**Nothing new!** All the chosen changes remove non-orthogonal restrictions on existing features or introduce features analog to other existing features, which means that we do not expect to need a lot of new documentation or educational material. That leaves those who'd write such resources free to deal with the launch of null safety instead.
28+
29+
The changes, in more detail, are detailed below.
30+
31+
## Allow type arguments on annotations
32+
33+
We currently allow metadata to be a call to a constant constructor:
34+
35+
```dart
36+
@Deprecated("Do not use this thing")
37+
```
38+
39+
However, the *grammar* does not allow type arguments, meaning that it's not possible to write:
40+
41+
```dart
42+
@TypeHelper<int>(42, "The meaning")
43+
```
44+
45+
There is no technical reason for this restriction. It was just simpler, and probably didn't seem necessary at the time metadata was introduced. It does now.
46+
47+
The only change is in the grammar, from
48+
49+
```
50+
<metadatum> ::= \gnewline{}
51+
<identifier> | <qualifiedName> | <constructorDesignation> <arguments>
52+
```
53+
54+
to
55+
56+
```
57+
<metadatum> ::= \gnewline{}
58+
<identifier> | <qualifiedName> | <constructorDesignation> <argumentPart>
59+
```
60+
61+
The corresponding constructor invocation must still be valid, now including the type arguments.
62+
63+
The constructed constant, if accessible in any way, will contain the provided type arguments, exactly like if it had been created by a <code>\`const' \<constructorDesignation> \<argumentPart></code> production, and is canonicalized correspondingly.
64+
65+
The largest expected effort for this implementation is the analyzer adding a place to store the type arguments to its public AST. The remaining changes should be using existing functionality.
66+
67+
If type arguments are allowed and omitted, the types are inferred from the types of the arguments to the constructor, as for any other constant invocation. This already happens (checked in VM with `dart:mirrors`), so no change is necessary.
68+
69+
## Allow generic function types as type arguments
70+
71+
The language disallows generic function types as type arguments.
72+
73+
```dart
74+
List<T Function<T>(T)> idFunctions; // INVALID
75+
var callback = [foo<T>(T value) => value]; // Inferred as above, then invalid.
76+
```
77+
78+
We remove that restriction, so a type argument *can* be a generic function type.
79+
80+
This requires no new syntax, and in some cases only the removal of a single check. There might be some platforms where the implementation currently assumes that generic function types cannot occur as the value of type variables (an proof-of-concept attempt hit an assert in the VM). Such assumptions will need to be flushed out with tests and fixed.
81+
82+
Because we already infer `List<T Function<T>(T)>` in the code above, this change will not affect type inference, it will just make the inferred type not be an error afterwards.
83+
84+
We do not expect the removal of this restriction to affect the feasibility of type inference. After all, it's already possible to have a generic function type occurring covariantly in a type argument, like `List<T Function<T>(T) Function()>`.
85+
86+
## Allow `>>>` as overridable operator
87+
88+
We reintroduce the `>>>` operator where it originally occurred in the Dart grammar (it's`\gtgtgt`):
89+
90+
```latex
91+
<shiftOperator> ::= `\ltlt'
92+
\alt `\gtgtgt'
93+
\alt `\gtgt'
94+
```
95+
96+
Because this makes `>>>` an `<operator>` and a `<binaryOpartor>`, it directly enables `#>>>` to be written a `Symbol` literal, and it allows declaring `operator >>>` as an instance member with a single positional argument. As for any other `<operator>`, you can do composite assignment as `x >>>= y`.
97+
98+
Further, the `Symbol` constructor must accept the string `">>>"` as argument and create a symbol equal to `#>>>` (identical if `const` invoked).
99+
100+
Some tests have already been committed, and the feature is currently under the `triple-shift` experiment flag.
101+
102+
Very little actual change is expected since the `>>>` operator behaves equivalently to `>>` and `<<`, so the same code paths should apply as soon as we are past lexical analysis.
103+
104+
When the operator has been enabled, we'll quickly (in the same dev-release series if possible, possibly early Q2) introduce:
105+
106+
```dart
107+
int operator >>>(int shift);
108+
```
109+
110+
on the `int` class, which will work similarly to `>>`, but will zero-extend instead of sign-extend.
111+
112+
At that point, the `>>>` operator on `int` must be **valid in constant and potentially constant expressions**, so `0x40 >>> 3` is a compile-time constant expression with the value `8`, and `const C(int x) : y = 0xFFFFFFFF >>> x;` is valid (although potentially throwing if `x > 63`) as a constant constructor.
113+
114+
Backends may want to optimize this to use available bitwise shift operations (like `>>>` in JavaScript), and intrinsify the function if possible. This can be done at any later time, though.

‎accepted/future-releases/small-features-21Q1/implementation-plan.md

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# Implementation Plan for "Small Features 21Q1"
2+
3+
Owner: [email protected] ([@lrhn](https://github.com/lrhn/) on GitHub)
4+
5+
Relevant links:
6+
7+
* [Tracking issue](https://github.com/dart-lang/language/issues/)
8+
* [Proposal](https://github.com/dart-lang/language/blob/master/working/small-features-21q1/feature-specification.md)
9+
10+
## Phase 0 (Prerequisite)
11+
12+
### "generic-metadata" Experimental flag
13+
14+
The implementation of parts of this feature ("generic metadata" and "generic function type arguments") should be developed behind an [experiment
15+
flag][]. Tools must be passed the flag
16+
`--enable-experiment=generic-metadata` to enable those features.
17+
18+
[experiment flag]: https://github.com/dart-lang/sdk/blob/master/docs/process/experimental-flags.md
19+
20+
While this feature is under development, individual tools may have incomplete or
21+
changing implementations behind the flag. When all tools have completely
22+
implemented the feature, the the feature will be enabled by default, and the
23+
flag removed in a stable release.
24+
25+
### "triple-shift" Experimental flag
26+
27+
The existing "triple-shift" experiment flag is already used for the partly implemented `>>>` operator. The flag is retained until we release the feature.
28+
29+
### Tests
30+
31+
The language team adds tests for the feature.
32+
33+
## Phase 1 (Foundation)
34+
35+
### CFE
36+
37+
The CFE implements parsing the new syntax, type checking it, and compiling it to
38+
Kernel. Since the CFE performs constant evaluation, it also
39+
implements the evaluation of metadata annotations.
40+
41+
The generic metadata feature can likely be implemented entirely in the front end and analyzer, with no back-end
42+
support needed. Since the front-end already evaluates the constants, and might infer type arguments, back-ends are likely used to seeing the filled-in result anyway. The analyzer deals with *source*, and must be able to make the distinction between an omitted-and-inferred type argument and an explicit type argument on a metadata constructor invocation.
43+
44+
The generic function type argument feature can likely be *implemented* entirely in the front-end, by not disallowing generic function types as type arguments, but back-ends might need to find places where they assume that a type argument cannot be a generic function type.
45+
46+
The `>>>` feature is already implemented in the CFE.
47+
48+
### Analyzer
49+
50+
For generic metadata, the analyzer needs to represent the *source*, and therefore must be able to make the distinction between an omitted-and-inferred type argument and an explicit type argument on a metadata constructor invocation. This is important for other source-manipulating tools like the formatter. The AST for metadata invocations need to have a place to store the type arguments, which isn't there now.
51+
52+
If the analyzer assumes, and relies on the assumption, that type arguments cannot be generic function types, then it might need to fix that.
53+
54+
The analyzer also needs to support `const Symbol(">>>")`, but otherwise seems to support the `>>>` feature (`operator>>>`  and `#>>>` works).
55+
56+
Otherwise the analyzer is unlikely to need significant change for any of the features.
57+
58+
## Phase 2 (Tool Implementation)
59+
60+
### dart2js
61+
62+
If the feature is handled by the front end, there may be no dart2js work.
63+
Otherwise, dart2js may need to handle any Kernel changes
64+
65+
Dart2js supports `>>>` except for `new Symbol(">>>")`. This can be fixed by a single RegExp update.
66+
67+
### Dartfmt
68+
69+
The only new syntax is metadata type arguments. They need to be supported, but should likely be treated like any other constructor invocation. Define and implement formatting rules for the new syntax. Add formatting tests.
70+
71+
### DDC
72+
73+
If these features can be implemented entirely in the front end with no Kernel
74+
changes, then no DDC changes may be needed.
75+
Otherwise, DDC may need to handle any Kernel changes or otherwise add support
76+
for this.
77+
78+
### IntelliJ
79+
80+
Update the IntelliJ parser to support the new syntax forms.
81+
82+
### Grok
83+
84+
Update Grok to handle the new AST.
85+
86+
### Analysis server
87+
88+
Update to use the latest analyzer with support for the feature.
89+
90+
The analyzer should likely recognize the grammar even without the experiment flag, and report errors stating that the feature is not available yet, rather than trying to parse `>>>` as `>>` followed by `>`, which is never valid as an operator.
91+
92+
There are no new quick-fixes needed.
93+
94+
### Atom plug-in
95+
96+
The [Dart Atom plug-in][atom] has a grammar for syntax highlighting Dart code in
97+
Atom. This same grammar is also used for syntax highlighting on GitHub. Update
98+
this to handle the new syntax.
99+
100+
[atom]: https://github.com/dart-atom/dart
101+
102+
### VS Code
103+
104+
Update the syntax highlighting grammar to support the control flow syntax (and,
105+
likely apply a very similar diff to the Atom grammar above).
106+
107+
### VM
108+
109+
If the feature is handled by the front end, there may be no VM work. The VM already fully supports `>>>`, but it is possible that generic functions as type arguments may trigger some unused code paths.
110+
111+
### Co19 tests
112+
113+
The co19 team can start implementing tests early using the experimental flags.
114+
Those tests should not be run by default until the feature has been released.
115+
116+
### Usability validation
117+
118+
No usability testing is planned. There is nothing *new* in these features (deliberately), just allowing existing syntax in new places, or one more operators which also exist in other languages.
119+
120+
## Phase 3 (Release)
121+
122+
### Enabling
123+
124+
The language team updates the experimental flags `generic-metadata` and `triple-shift` to
125+
always be enabled and no longer be available to users, and releases this update
126+
in the next stable Dart release.
127+
128+
### Use
129+
130+
The Dart library team introduces `int.operator>>>` as soon as possible.
131+
132+
### Documentation
133+
134+
The language team adds the feature to the CHANGELOG. They write some sort of
135+
announcement email or blog post.
136+
137+
The language tour needs to be updated to mention `>>>` as a user definable
138+
operator.
139+
140+
## Phase 4 (Clean-up)
141+
142+
### Remove flag
143+
144+
All tools may now remove the dependencies on the flag in the experiments flag
145+
definition file. When all SDK tools have done so, the flag is removed from the
146+
experiments flag definition file.
147+
148+
## Timeline
149+
150+
Completion goals for the phases:
151+
152+
* Phase 0 (Prerequisite): Mostly done, tests by end of January.
153+
* Phase 1 (Foundation): Early February 2021.
154+
* Phase 2 (Tool Implementation): Mid March, 2021
155+
* Phase 3 (Release): TODO
156+
* Phase 4 (Clean-up): TODO

0 commit comments

Comments
 (0)