Can we have a generic Type[C]?

[NYKevin]

The type object occupies a rather strange place in the type hierarchy:

>>> type(type) is type
True

(I'm pretty sure that's a flat lie, since you can't instantiate something from itself, but regardless...)

>>> isinstance(type, object)
True
>>> isinstance(object, type)
True
>>> isinstance(type, type)
True

In Java, the (very rough) equivalent is a class, specifically Class<T>. It's also generic; the type variable refers to the instance. Java has it easy because they don't support metaclasses. Classes are not first class in Java, so their type hierarchy doesn't have to deal with the strange loops shown above.

I realize metaclasses in their full generality are out of scope (at least for now), but a generic Type[T] like Java's would be nice to have. So far as I can tell from the Mypy documentation, it doesn't currently exist.

Here's some example code which might like to have this feature:

def make_foo(class_: Type[T]) -> T:
    # Instantiate and return a T

[gvanrossum]

Unless this is trivial to add to mypy I propose to punt this to a future version of type hints.

FWIW type(X) is a shorthand for X.__class__ and type.__class__ is indeed type. Not a lie.

[NYKevin]

Yeah, but I'm pretty sure you have to reassign type.__class__ at some point to get there...

Anyway, I recognize that this is probably a lot more complex than it looks, and that the first 3.5 beta is rather alarmingly close, but I do feel this is essential for anyone using first-class classes in any significant way. Otherwise, you just have a type object, which is basically Callable[[???], object] (any idea what to put for the args?). You'd need a lot of casting to make that useful. I mean, I suppose you can treat it like Type[Any]/Callable[Any, Any], but then you're just being too permissive instead of too strict.

[gvanrossum]

Any is your friend. :-)

[agronholm]

Here's an example use case, straight from the code of my new framework:

@typechecked
def register_extension_type(ext_type: str, extension_class: type, replace: bool=False):
    """
    Adds a new extension type that can be used with a dictionary based configuration.

    :param ext_type: the extension type identifier
    :param extension_class: a class that implements IExtension
    :param replace: ``True`` to replace an existing type
    """

    assert_subclass('extension_class', extension_class, IExtension)
    if ext_type in extension_types and not replace:
        raise ValueError('Extension type "{}" already exists'.format(ext_type))

    extension_types[ext_type] = extension_class

I would like to declare the second argument as extension_class: Type[IExtension] (or Class[IExtension], doesn't matter to me). Likewise, the type hint for extension_types should be Dict[str, Type[IExtension]].

[ilevkivskyi]

I understand the time pressure but without Type[T] typehinting system looks a bit incomplete. I hope it will be added soon.

[gvanrossum]

Could you submit a patch to mypy? That would make a world of difference.

Until then, incompleteness of the typesystem doesn't bother me that much -- unlike statically-typed languages, Python programs don't have to be fully specified, you can always use Any.

[ilevkivskyi]

I totally agree, one of the good points of gradual typing is the possibility to introduce it gradually :-) IMO it is perfectly pythonic - one could take exactly as much typing as one wants, practicality beats purity here. And yes I will try to make a patch.

[NYKevin]

@ilevkivskyi If you're serious about making a patch, make sure type inference of this line doesn't crash or break things too badly:

x = type

You could infer x to be a Type[Type[Type[...]]], but that's probably not a Good Idea. I'd just make it Type[object] or Type[Any]. Not sure whether this is an actual problem without looking at mypy's source, but it's something to watch out for.

[JukkaL]

Another issue related to Type[X] is whether a value with such type should be callable. Since __init__ is special (it is often overridden with an incompatible signature) just taking the signature of __init__ is not safe, unless there is a way to annotate __init__ to require a compatible override in all subclasses.

For example, consider x with type Type[object]. If it would be callable, x() should probably be accepted, as object() is fine. Now the runtime valueof x could be slice, as it's a subclass of object -- like any class is. However, in this case the call would fail at runtime since slice() is not valid.

[NYKevin]

@JukkaL It gets worse:

class Foo:
    def __init__(self, args):
        # do something
    @classmethod
    def bar(cls, args):
        # cls is Type[Foo]
        self = cls(args)  # type: Foo
        # do something else
        return self

I agree with Jukka that this code is wrong. We can easily fix it by splitting the "do something" off into a separate (underscore-prefixed) method and calling super().__new__() from bar(). But unlike the examples we've been discussing so far, this is something the average user would actually run into. If we talk about Type[Foo] in the error message, we'll induce exploding head syndrome in the end user. It would be nice if we could avoid that.

[gvanrossum]

The pattern (helper class methods that create and return instances) is indeed very popular. I think it's usually done for classes that won't be subclassed, or where the subclasses don't override __init__ (at least not in an incompatible way). In the former case one could use a static method and name the class explicitly, but the second use case is popular in some circles.

I wonder if a good type checker should be complaining only about an __init__ override that changes the signature, and only when the base class constructs instances of itself through a class method's cls argument.

[agronholm]

Frankly I don't understand what the fuss is about. Type[Foo] would not guarantee anything beyond the type being a subclass of Foo. It would not guarantee compatibility of method signatures. Just like when annotating an argument as Foo, it would still accept an instance of any Foo's subclass, which may have arbitrary method overrides, yes?

[NYKevin]

@agronholm That's why we have the Liskov substitution principle. Usually, constructors and initializers are exempt from that rule since they're (effectively) static, but when you start to allow covariant first-class classes, that decision becomes a bit more questionable. Should static/class methods be exempt from LSP at all?