Records: Should functional update be allowed to change the type?

[eernstg]

Issue #1292 seems to gather broad support for a mechanism where an existing record/tuple whose static type is a concrete record type is used as a template in the creation of a new one, just differing at the components that are mentioned. The syntax could for instance be as follows:

void main() {
  (int, int, {Color color}) x = (0, 1, color: Color.red);
  var x2 = x.with(7); // Just changes x[0].
  var x3 = x.with(_, 3); // Just changes x[1].
  var x4 = x.with(1: 3); // Also just changes x[1].
  var x5 = x.with(color: Color.Blue); // Just changes x.color.
}

This mechanism gives rise to a language design decision with respect to the type of the result:

We may require that each actual argument of the with construct has a type that is a subtype of the static type of the corresponding component of the receiver. This allows the result to be re-assigned to a variable:

void main() {
  (num, double, {Color color}) x = (3.1, 1.3, color: Color.red);
  x = x.with(7); // Require that `7` has type `num`.
  x with= 7, 4.2; // Haha, perhaps. Would require `num`, `double`.
}

Alternatively, we could relax the requirement such that it must be a subtype or a supertype, or we could leave the type entirely unconstrained:

void main() {
  (num, int, {Color color}) x = (0, 1, color: Color.red);
  var y = x.with('Hello!'); // OK, yielding type `(String, int, {Color color})`.
}

Pro:

• This creates a systematic abstraction that relates concrete record/tuple types with the same shape, even in the case where there is no componentwise subtype relationship. This allows developers to write code which is more compact and consistent, because they can directly rely on all components which are not updated to keep the same position and type, even in the case where the updated ones have new types. A class can have a similar property (Pair<T1, T2> and Pair<S1, S2> can be seen as related because both have Pair<Object?, Object?> as a supertype), but that connection does not allow for abstractions over transfer of state, or any other operations involving "all components", so this would bring something new to Dart.

Con:

• If there are no constraints from the context on the result, "wrong-typed components" (anything that is not a subtype of the statically known type) would be allowed, and this may be considered as too permissive.

We could also consider changing the shape of the result, but that seems less attractive: It would require a non-trivial amount of syntactic support to allow components to be dropped or added. It's probably better to require that new shapes are expressed using a record/tuple literal, where we already have well-known syntax for specifying the shape.

[lrhn]

Since functional update creates a new object, there is no issue with it creating something of a new type. You might not be able to assign it back to the original variable, but that's not necessarily a problem.

Also, the type of the original object isn't as interesting as its static type. If we have: (num, num) p = (1, 2); then the static type of p is two doubles, but the run-time type is two integers. Replacing one of those integers with a double would be completely reasonable within the static type. It would be wrong to prevent you from replacing an int by double if the target you're aiming for is num. So, static type is all that really matters, but should the static type of the original matter more than the static type that the resulting value is assigned to? I see no reason for this.

However, if you are allowed to "replace as single value" from a tuple to have a completely different type, then there is no longer any type relation between the original and the new value. So, why can't you also add more values or remove values entirely from the original? In short, why are you not just building a new value from scratch instead of "replacing" in the existing value?

The answer will likely be "it's shorter", which suggests that we might just want shorter syntax for more general projections, so instead of x.with("hello!") you can write ("hello!", ...x[1, color]) to project the positions 1 and color of x into a tuple of type (int, {Color color}) and then spread that into the new tuple (which the compiler should be able to do without an intermediate tuple).

Is "replacing specific values" just a special case of building a new tuple containing some values of an existing tuple? Should we support the more general case instead?
If we have that, will we need the "replace specific values with" operation at all?

[leafpetersen]

FWIW, F# has no problem with this:

let myRecord2 = {| X = 1; Y = 2; Z = 3 |}
let myRecord3 = {| myRecord2 with X = "hello" |}

Ocaml and Haskell records are (I think) always named, so this would mostly come up with polymorphic records. Ocaml at least doesn't seem to allow changing the type of a polymorphic field with a with clause.

[eernstg]

@lrhn wrote:

However, if you are allowed to "replace as single value" from a tuple
to have a completely different type, then there is no longer any type
relation between the original and the new value.

This is exactly the point I'm making: The ability to express and maintain a connection between such types is new to Dart, because they are unrelated according to the existing type system. In other words, this enhances the expressive power of the type system.

The benefit derived from this feature is consistency and abstraction: if a record/tuple r has type (T1, T2, T3) then r.with(1) has type (int, T2, T3). If the code is updated (say, we do pub upgrade and import a new version of lib.dart) such that r now has type (T1, T2, T4, {T5 foo}) then r.with(1) will have type (int, T2, T4, {T5 foo}).

The context may then have to be edited such that it works with the new structure of data (this is a good thing), or the new type may propagate implicitly through the context because it abstracts away from the details that differ (this is even better).

In contrast, if we use a record/tuple literal (1, r[1], r[2]) rather than r.with(1) then the update would not be propagated consistently: It does get propagated that the type of r[2] is now T4 rather than T3, but r.foo has been dropped silently.

If the change goes from (T1, T2, T3) to (T1, T2) then (1, r[1], r[2]) would be a compile-time error: The change does get propagated, but this creates a need to edit code to fix the error, whereas r.with(1) would just keep working correctly.

So, why can't you also add more values or remove values entirely from the original?

I mentioned this possibility, noting that the ability to build a new shape would be somewhat costly: We'd need syntax and language mechanisms to specify record/tuple shape incrementally. If we can come up with an elegant design for doing this then the benefits described above would just apply to a broader range of situations, which is great!

In short, why are you not just building a new value from scratch instead of "replacing" in the existing value?

Because that new value would not be as abstract, and we would miss out on the ability to maintain consistency as described above.

Conciseness is another benefit, but that is much, much less important.

Is "replacing specific values" just a special case of building a new tuple
containing some values of an existing tuple? Should we support the more
general case instead?

This is the incremental shape specification I just mentioned, and it would indeed be more powerful. But the shape preserving with operator is already useful, and it is of course possible to start with that and then enhance it with shape changing language mechanisms.

@tatumizer wrote:

Question: Are we allowed to write x.with(color: red) if all the compiler knows about x is that it's a Record

My preference would be that the with operator is only available for receivers whose static type is a concrete record type, so the answer would be "no".

With a concrete record type the shape of the receiver is statically known. This ensures that the operation can have good performance (a with expression can be desugared to a record/tuple literal at compile-time) and we avoid the reflection-ish features associated with dynamically looking up named fields and creating similarly named fields in a new record type.

[eernstg]

@munificent, as far as I can see we aren't actually doing anything like with. True? Move to records-later?

[munificent]

Yes, we aren't going to get any kind of record update in the initial release.

[eernstg]

Sounds good that it might come later on! ;-)