Possible solution for #107 - implicit constructors

[kevmoo]

Some rough ideas. CC @leafpetersen @efortuna @jakemac53

[ds84182]

What about something like an into expression, a from factory constructor, an into method, and an into type parameter constraint:

// package:http
Future<Response> get<T into Uri>(T url, {Map<String, String> headers}) => _client.get(url into Uri, headers: headers);

// Uri
class Uri {
  // ...
  from String uri => Uri.parse(uri);
}

// User defined class
class MyUri {
  // ..
  into Uri => Uri.parse("http://example.com/$path");
}

Every type implements into T => this;, where T is a supertype of the current type, and it cannot be overridden.

Interface conflicts won't be an issue, since the most specific into conversion is selected for a type, just like virtual methods. In fact, into could be implemented as mangled methods (at least initially).

Good questions:
Should type hierarchy be taken into account? If a type X has into int declared, can you do x into num? What if both into int and into double are defined? into Object is already implemented, so that isn't a problem.

[efortuna]

cc @filiph

[lrhn]

This is implicit conversion. It does not have to be a constructor, but I can see how it would be convenient to allow the operation to act like a constructor in some cases. I fear that it might be too limiting if only the class itself can define conversion to it.

I might want to define a conversion in the source type, not the target type, because that's the one I'm in control of.
I might want to define a conversion even if I'm not in control of either type.

There are three general issues here:

• Recognizing that a conversion is needed. If expression is not assignable, then it's easy. While we have implicit down-casts, we may want an implicit conversion to be used even if an implicit down-cast is possible. The conversion should likely be based on static type, but it's not clear whether there are pitfalls with that.
• Finding the correct conversion. You can look at the target type (what is proposed here), or at the source type, or you could even define conversions statically outside of either class (which, I guess, is effectively a static extension method for one of the other two cases). It will not work for dynamic invocations. It will not work for up-cast values. If I have a conversion from Foo to Bar, it will not get triggered by if I assign a Foo to a variable with type Object, and then assign that variable to Bar. Is that a problem?
• Since it's based on static type, we might be looking at static declarations (constructors are static-y too).
  Static declarations are not inherited. If I need to convert Foo to num, will I find a Foo-to-int conversion? (Not if I look in num, maybe if I look in Foo).
• Handling conflicting conversions. If I can convert both num and int to Foo, which one should I use for 42. Likely the most specific. What if there is no most-specific? What if I can convert int and Future<int> to Foo, what should I do with an expression with static type FutureOr<int>? Likely a compile-time error. There are details to nail down, but probably not very surprising ones.

The into approach is new syntax, I'd probably just an extension of the operator syntax for it (since it's an operation that is not invoked directly). Let's go wild:

class C {
  final value;
  C(this.value);
  // Implicit cast from instance of C to Foo. Inherited by sub-classes.
  // (Then it's part of the interface!)
  Foo operator cast() => Foo(value);
  // Implicit cast from C to Baz.
  static Baz operator cast(C c) => Baz(c.value);
  // Implicit cast from Foo to C.
  static C operator cast(Foo foo) => C(foo.value);
  // Implicit cast from Bar to C (using constructor behavior to create the C)
  static C.operator cast(Bar bar) : this.value = bar.value;
}
// Implicit cast from Baz to C, when declaration is in scope.
// (So, not needed if we have static extension methods, then just declare it on C or Baz).
C operator cast(Baz baz) => C(baz.value);

The, when you need to assign an expression of type B to type A:

• If B is a sub-type of A, just assign it.
• Check for top-level cast operations from B (or a super-type) to A (or to a sub-type). If found, use it.
• Check class A for cast operations from B (or a super-type) to A (or to a sub-type). If found, use it.
• Check class B for cast operations from B (or a super-type) to A (or to a sub-type). If found, use it.
• if B is a super-type of A, insert implicit down-cast.

If any step finds more than one viable cast, and none of them are most specific, make it a compile-time error.

[jakemac53]

What happens if P has a static type of dynamic? It seems like this could easily end up causing issues with tree shaking (not being able to treeshake any implicit constructor if dynamic is ever passed to a function that takes that type).

Maybe we could say this only happens at compile time, so you have to explicitly cast to get the implicit conversion? That would also cause some weird issues potentially but might also make it faster?

Making it only a compile-time thing would also mean it can be implemented in the FE only most likely.

[jakemac53]

Oh I should have read the other comments first haha, @lrhn brought up essentially the same question.

[leafpetersen]

Let's go wild:

Question: are any of the extended versions not already expressible if we have just the basic feature (implicit constructor) + scoped extension methods? It seems to me that all of the static and top level casts can just be added as instance casts in a scoped extension method for the appropriate type. e.g.

  // Implicit cast from instance of C to Foo. Inherited by sub-classes.
  // (Then it's part of the interface!)
  // Foo operator cast() => Foo(value);
  extend Foo with {
    implicit Foo(C c) => Foo(c);
  }

 // static Baz operator cast(C c) => Baz(c.value);
  extend Baz with {
    implicit Baz(C c) => Baz(c.value);
  }

etc?

[kevmoo]

I agree w/ @leafpetersen . The flexibility offered by @lrhn's proposal is interesting, but it scares me, too.

[leafpetersen]

I'd say land this for iteration, discussion, and future reference (no promises... :). Can you:

• File an issue stating the problem you're trying to solve (e.g. "APIs that can take one of two different types (one of which is constructed from the other) are very common in Flutter and hard to express")
• File an issue for discussion of your solution (this) referencing the first issue
• move this to working/#of the first issue - implicit_constructors/
• land this, then add a link to your doc to the first comment of the second issue filed.

[kevmoo]

Note all:

• Problem definition/discussion has moved to Make it easy to use type B in place of type A when there is an obvious/explicit conversion from B -> A #107
• The proposed solution discussion has moved to Implicit Constructor proposal #108

Cheers!

[lrhn]

@leafpetersen It's quite possible that we can get everything with static extension cast operations.

Static extension methods are "inheritable" (they're not virtual, but subtypes do get them unless they override the member with something else on a subtype).

However, constructors are used on a type, not on an object, so they differ from static extension methods.
We'd have to introduce extension static methods (classes extended with static methods) which is different from static extension methods (instances extended with methods that are statically resolved).
The former does not require an instance.

That will allow externally adding constructors to a class (maybe even generative ones, as long as they are redirecting).

However, Constructors are never inherited (the typing doesn't make sense), and we don't want to check an unbounded sub-type tree for implicit constructors, so I'm not sure using constructors is the way to go. I'd prefer to search for the conversion on the source type because it makes sense to check for casts in its (limited) superinterfaces

So, we could also define cast operators as instance methods on the source type, and allow adding or overriding them as static extension methods.

extend class String with {
  operator Uri() => Uri.parse(this);   // C++ cast operator syntax, for example.
}

All our other operators are instance methods working on the operator.

The reason this feels slighlty odd is that String is a more fundamental class than Uri. The String class doesn't/shouldn't even know what a Uri is. In terms of a stratified type dependency hierarchy, String is more fundamental than Uri, so dependencies should go from the latter to the former.
Putting the cast operation from String to Uri on the String class breaks that ordering.
(Which is why we add it as an extension method).

tl;dr: I don't think constructors are the way to go, and they don't fit well with static extension methods.

[jmesserly]

The reason this feels slighlty odd is that String is a more fundamental class than Uri. The String class doesn't/shouldn't even know what a Uri is.

yeah, agreed.

FWIW, C# allows conversion operators to be defined in either direction, you can do it on the same type. Here's an example from the C# spec (Conversion operators section):

using System;

public struct Digit
{
    byte value;

    public Digit(byte value) {
        if (value < 0 || value > 9) throw new ArgumentException();
        this.value = value;
    }

    public static implicit operator byte(Digit d) {
        return d.value;
    }

    public static explicit operator Digit(byte b) {
        return new Digit(b);
    }
}

(The example is illustrating that implicit conversions should only be used for cases where they don't cause data loss or throw exceptions, otherwise an explicit conversion should be used. But it also illustrates how Digit can define both directions of the conversion, without the byte class knowing about it.)

edit: in Dart that would probably look more like:

class Uri {
  ...
  // Add implicit conversions from String <-> Uri in both directions.
  static operator String(Uri uri) => uri.ToString();
  static operator Uri(String str) => Uri.parse(str);
}

[munificent]

One thought: Anything we do here will impact profoundly with overloading if we ever have any intention of supporting overloading by type. I have read numerous horror stories from C++, C#, and Swift about the interactions between overload resolution and implicit conversions. Just implicit conversions is easy because you only have a single linear path of conversions your are trying to solve. But overloading introduces argument lists which means the solution space you are exploring to select conversions and overloads is combinatorial.

Putting that aside...

I really like Lasse's suggestion to make conversions instead methods on the source object. It's natural in that you must have some object you are converting from in the first place. It composes very nicely with extension methods. The latter also lets you do things like define conversions from specific types, not just classes. For example, I could define a conversion from List<double> to Matrix, without being able to convert arbitrary lists of other types to them.

I'm not worried that this forces you to put the conversion on the "wrong" type in that you end up declaring it on the more fundamental type (String) instead of the more domain-specific (Uri). I think that presumes a mental model where extension methods are monkey-patching or interfering with the extended type in some way. But that's not my model. Extension methods are syntactically called from the type, but aren't related to it in any other way. They don't break its encapsulation, access its internal state, define invariants for it, or override behavior. In other words, they can't break anything the original author of the class expects to be true. So I think of them as being "outside" of the type.

Pragmatically, a nice thing about always defining conversions as instance/extension methods on the source type is that it lets us reuse the existing rules for method collisions to also cover colliding conversions. If a class can define multiple conversions to different types (which I think they should support), then a syntax like C++'s that uses the destination type name as a name is probably a good hint that those conversions don't collide. If we do something like operator cast(), then it looks like multiple operators would have the same name and rely on some sort of weird return type overloading to disambiguate.

C# allows conversion operators to be defined in either direction, you can do it on the same type.

This is true, but that feature is also older than extension methods. If they had extension methods first, I wonder if they would have still felt the need to support defining conversions on both the source or destination type.

[jmesserly]

One thought: Anything we do here will impact profoundly with overloading if we ever have any intention of supporting overloading by type. I have read numerous horror stories from C++, C#, and Swift about the interactions between overload resolution and implicit conversions. Just implicit conversions is easy because you only have a single linear path of conversions your are trying to solve.

C# overload resolution is NP-hard (Eric Lippert's 2007 blog post about it) but it's due to lambda expressions. I don't recall offhand what the worst case is with only implicit conversions+overloads. In practice, the worst case basically never comes up.

(Besides, we already have a much more common NP-complete problem in the Dart ecosystem... package resolution 😉)

This is true, but that feature is also older than extension methods. If they had extension methods first, I wonder if they would have still felt the need to support defining conversions on both the source or destination type.

fair point! It does simplify the search for implicit conversions to only look at the types involved, but you're right, defining them via extension methods would be quite nice too:

operator String cast(this Uri uri) => uri.ToString();
operator Uri cast(this String str) => Uri.parse(str);

[leafpetersen]

However, Constructors are never inherited (the typing doesn't make sense), and we don't want to check an unbounded super-type tree for implicit constructors, so I'm not sure using constructors is the way to go. I'd prefer to search for the conversion on the source type because it makes sense to check for casts in its (limited) superinterfaces

I don't understand this comment, can you elaborate? I don't understand why there's any searching at all.

  Bar x = ...
  Foo t = x;

Assuming Bar is not assignable to Foo: does Foo have an implicit constructor that accepts Bar yes/no? If yes, call it, if not error. Where does searching come in?

You do immediately get into the question of overloading if you want to have multiple implicit constructors, but I'm not sure any of the options avoid something like this (again, assuming you want to support multiple implicit constructors).

[leafpetersen]

So, we could also define cast operators as instance methods on the source type, and allow adding or overriding them as static extension methods.

So something like:

class A {
  operator int() => 3;
}
class B {
  operator double() => 3;
}
class C extends A with B {
}
num x = new A(); // Produces 3?
num x = new C(); // Static error?

class D extends A {
  operator num() => 3.0;
}

int x = new D(); // Produces 3?
num x = new D(); // Produces 3.0?
num x = new D() as A; // Produces 3?

It feels unpleasantly obscure to me on first read. The criticism that I hear of implicit conversions in other languages is that it's hard to understand what the compiler is doing. I'm not sure that throwing virtual dispatch into the mix really improves that situation.

[munificent]

I'm not sure that throwing virtual dispatch into the mix really improves that situation.

Is virtual dispatch involved in your example, or is it more about how subtyping on the destination type gets involved? My (simple) mental model for this is that each conversion operator has a different name — the name of the destination type. So in C, there is no overriding or collision. operator int() and operator double() are two totally unrelated members.

Then the question is whether either of those matches when the destination type is num. A simple answer is to say "no" and that it must be an exact match:

class A {
  operator int() => 1;
}
class B {
  operator double() => 2.0;
}
class C extends A with B {}

num x = new A(); // Error. A has no "operator num()".
num x = new C(); // Error. C has no "operator num()".

class D extends A {
  operator num() => 3.0;
}

int x = new D(); // 1.
num x = new D(); // 3.0
num x = new D() as A; // Error. A has no "operator num()".

You're probably right that virtual dispatch doesn't add much value. One, perhaps terrible idea: allow defining implicit conversions only as extension methods on the source type. That sidesteps inheritance and overriding issues.

[lrhn]

I don't understand this comment, can you elaborate? I don't understand why there's any searching at all.

 Bar x = ...
 Foo t = x;

Assuming Bar is not assignable to Foo: does Foo have an implicit constructor that accepts Bar yes/no? If yes, call it, if not error. Where does searching come in?

First: My bad, I meant to say "unbounded subtype hierarchy".

The question is that if Foo2 extends Foo and has a conversion Bar->Foo2, should you use it here?
If yes, then you have to look through Foo's entire sub-type hierarchy for the conversion.
If not, the feature becomes much less usable. You can only convert Bar to exactly Foo. Any wiggling in the static types, and the conversion no longer works. You need to add exact conversion to every type you ever want to convert to, even if they are all supertypes of one particular type.

[lrhn]

For the virtual override, I think the biggest issue would be not knowing whether a conversion is virtual.
If we allow both proper virtual cast operations and static extension method based operations, then one is virtual and the other is not. That will be confusing.
For your examples:

num x = new A(); // Produces 3?
num x = new C(); // Static error?

I'd be fine with that result, yes. The former checks the static type A for a valid cast to num, finds only operator int, and it works.
The latter checks the static type C for valid castas to num and finds two, with neither being more preferable than the other (in whatever way we would do that), so it's a static error - duplicate definition, ambiguous invocation, whatever.

int x = new D(); // Produces 3?
num x = new D(); // Produces 3.0?
num x = new D() as A; // Produces 3?

The first one I'm OK with, there are conversions to int and num, but only int is sound, so we use that.
The second one finds two applicable conversions, one to int and one to num. It's either an error or we pick one of them by some kind of preference (most specific/least specific, not sure which is best).
The third one is a problem, then because the static lookup only finds operator int, and unless operator num somehow overrides that (and that's probably not meaningful to say), we will get 3.

There is no virtual dispatch here, only signature based overloading. It's not the virtuality that's a problem, it's the overloading plus inheritance which makes it possible to use an object at a super-type where not all of the operations are (statically know to be) available.

The overloading is necessary, we will want to be able to convert int to both double and String.
The inheritance is the questionable thing then. if D does not inherit the int conversion from A, then ... well, not much changes. Statically casting to A will still make a difference in the result unless the method is virtual. I'm not completely sure non-inheritance + virtuality combines, it should be possible even if it sounds crazy, but it will probably only work for the exact type. So operator int wouldn't be useful to convert to num, which is sad.

[leafpetersen]

The question is that if Foo2 extends Foo and has a conversion Bar->Foo2, should you use it here?

No.

If not, the feature becomes much less usable.

I need some justification for this claim. Constructors in Dart are non-virtual. All this is doing is giving you a convenient way to automatically call a constructor. From the perspective of the use cases in mind, it seems to me that searching the subtype hierarchy would be an explicit anti-feature. It's also anti-modular. Moving an additional subtype of Foo into scope suddenly breaks existing code (since that suddenly makes it ambiguous).

So I don't get the argument. Virtual would be bad, non-virtual has no issues. What's the problem?