Move some types from _typeshed to typing_extensions

[srittau]

I wonder if we should have a third-party module on PyPI with only useful type definitions / aliases that can be more cleanly used in user code than importing from _typeshed. Unlike typing_extensions, these wouldn't introduce any new type system features -- it would all be just for convenience. And unlike typing_extensions, we could add arbitrary things not in typing.

Originally posted by @JukkaL in #5910 (comment)

[hauntsaninja]

This is a little bit tricky if we're hoping to have stdlib types depend on this. E.g., we run into the same issues that caused us to pretend typing_extensions is part of the stdlib (what happens if users don't install _typeshed?).

[srittau]

Type checkers would need to depend on this package.

[hauntsaninja]

It could still lead to problems, e.g. with mypy's --python-executable option where it checks an environment other than its own, or if users pin a types-typeshed version that is incompatible with the type checker's. The analogy to typing_extensions still holds, since mypy does depend on typing_extensions (/ briefly depended on types-typing-extensions).

[srittau]

Since I'd expect type checkers to depend on the correct types-typeshed version, pinning it by the user should conflict when pip installing, so this would lead to an impossible situation.

[JukkaL]

typing_extensions is a bit different in that mypy has special treatment of some of the definitions, which means that an incompatible version can break things. This package would only contain "ordinary" definitions so having an incompatible version would be less bad.

Typeshed stubs don't have to use this package -- we could continue to have _typeshed that we'd use internally. This way version mismatches would be more easily avoided for stdlib stubs. It would make sense to have the PyPI package mostly mirror definitions in _typeshed, but I don't think that we need to make them match each other 100%. _typeshed can use definitions that are more complex/general/special-purpose than the PyPI package. The PyPI package should probably be designed more for usability and clarity. One approach would be to promote definitions from _typeshed to the PyPI package once there's enough evidence that the definition is stable and widely useful. After we make a definition officially public, it will hard to make any changes to it, ever.

Another option would be to use the public package in third-party stubs but not in stdlib stubs. Here the motivation would be that type checkers may ship a too old version of the _typeshed package. I think that it's more important to avoid breaking stdlib stubs, since third-party library stubs are usually not necessary for basic type checking. Also, if the user has to install a types- package anyway for a third-party library, having an extra dependency is no big deal. And users have to be prepared to deal with library version conflicts with third-party packages in any case.

[MapleCCC]

+1 for this. It feels more sane to write from typeutils import StrOrBytesPath than writing if TYPE_CHECKING: from _typeshed import StrOrBytesPath every time.

[hauntsaninja]

Can we re-use typing_extensions for this? I feel like that could be less likely to cause confusion for users / typing_extensions already has things like SupportsIndex.

[srittau]

I think using typing-extensions would be the best idea.

[JelleZijlstra]

Yes, using typing-extensions makes sense to me. In the docs we should make a clear distinction between the backports and utilities parts of typing-extensions though.

[srittau]

We could put the custom types in a sub-package. typing_extensions.ext or so. We should also reconsider the typing-extensions version scheme in that case and probably decouple it from Python versions.

[tony]

StrOrBytesPath may be a good candidate for this, if it's stable: re conversations at https://stackoverflow.com/a/68027757/1396928

Could StrOrBytesPath also be compatible back to python 3.6? This type has been discussed since PEP 519

[AlexWaygood]

This now exists as https://github.com/hauntsaninja/useful_types