Removal policy

[srittau]

This came up in #4606.

I think we should have a clear policy on when to remove packages from typeshed. My suggestion:

• If a package ships with a py.typed file in the latest released version, or
• if a package supports no Python version released by typeshed officially.

[JelleZijlstra]

Deleting the stubs as soon as a version with py.typed is released seems a bit extreme to me, since users may still be using older versions. I'd prefer waiting for some time.

[srittau]

Sounds reasonable. What do you think is a good time? Six months?

We also need to review this policy, when typeshed supports multiple versions at once.

[JelleZijlstra]

Six months or a year sounds reasonable, though it may also depend on individual libraries and how likely users are to use an old version.

[srittau]

I am reopening this discussion due to modular typeshed and because it became relevant due to all pallets projects now shipping their own stubs.

Do we still keep a package around for 6-12 months, even with modular typeshed? After all, the package is not removed from pypi, even if we remove it from the repository and is still usable indefinitely.

Also I think it would be useful to create one final upload before we remove a package from the repository, where we indicate that the package is now obsolete and the upstream package ships its own stubs.

[srittau]

We can now mark packages with obsolete_since and a version number. See this example and the corresponding pypi page. We should do this for each package we remove. Alternatively, for packages that we remove, but don't ship types (for example old, unsupported packages), we can use the extra_description field.

This only leaves the question whether we should keep packages around for 6-12 months.

[Akuli]

Instead of a fixed time, maybe look at how popular the new version of the package is compared to older versions that lack type hints? If half (or some other amount) of all downloads is for new versions that come with type hints, then I think it's fine to delete the stubs in typeshed. As pointed out before, this shouldn't break pip install types-foo in anyone's CI.

[jakebailey]

For the editor case, I know it's probably going to be useful to have these stubs to support users who are still on old versions of libraries (which will likely be the case for a good while). I know the Flask stubs help current users quite a bit, and I've sent some targeted fixes to members we saw people using but were untyped.

My impression is that to conform to PEP 561, type checkers are going to be ignoring stub packages / typeshed in favor of the library itself if the library claims to be py.typed anyway (which is the case for new Flask et al), so I don't know if this needs to go super quickly. (Pending fixes to microsoft/pylance-release#1197, microsoft/pyright#1764, in our case; next week.)

[srittau]

For the editor case, I know it's probably going to be useful to have these stubs to support users who are still on old versions of libraries (which will likely be the case for a good while). I know the Flask stubs help current users quite a bit, and I've sent some targeted fixes to members we saw people using but were untyped.

Please note that the package won't be removed from pypi, just because it's removed from typeshed. So the package will be available indefinitely, we just can't update it anymore.

[jakebailey]

Yes, certainly, but we vendor the typeshed repo in pyright/pylance, as the average python user isn't really expected to understand what a stub is (let alone that they need to install them).

[jakebailey]

But, we also bundle stubs sourced from other packages, so could continue to bundle the old Flask stubs and such for a while if typeshed drops them.

[srittau]

Makes sense. So let's just keep the status quo for now.