[CI] Run mypy tests with a fixed Python version

[srittau]

Since mypy is currently considering to drop Python 3.9: We run most Python-based tasks in CI using a fixed Python version, even if we check against different versions, e.g.:

foo:
    strategy:
        matrix:
            python-version: ["3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
    steps:
    - uses: actions/setup-python@v6
      with:
          python-version: 3.13
    - run: bar --py ${{ matrix.python-version }}

The exception is mypy. I think this has mostly historical reason. I suggest we change this to always run mypy with a fixed version as well, and just use the --python-version flag. This removes a bit of churn for us, makes the Python versions used in the tests more consistent overall, and shouldn't have any adverse effects.

[AlexWaygood]

I think this would lead to worse results.

In general, we run our mypy tests in a more principled way than we run our pyright tests. For pyright, we install all third-party dependencies for all stubs into a single "virtual environment soup", and then test all third-party dependencies using that single virtual environment. But for mypy, we create a separate, isolated virtual environment for each third-party stub and install only that stub's third-party dependencies into the environment.

Some non-stub dependencies such as numpy have already dropped support for Python 3.10, so we won't be able to install a 3.10-compatible version of numpy unless we actually use Python 3.10 to create the virtual environment into which we install numpy.

[srittau]

Well, my suggestion was not to use 3.9 for mypy, but the latest version that works, currently probably 3.14.

[AlexWaygood]

Doesn't that run the risk that we might add a non-stubs dependency, which doesn't support Python 3.9, to a stubs package that does support 3.9, and none of our tests would flag that?

[srittau]

In that case it's very likely that the runtime package would also depend on that package. Honestly, these dangers sound fair negligible to me, while the advantage of not battling outdated Python versions in CI is a plus to me. (Especially with our quite limited maintainer resources.) We could also get rid of workarounds like this:

typeshed/.github/workflows/tests.yml

Lines 66 to 68 in 37dceef

	# python-version can sometimes be pinned to a specific version or to "-dev", but
	# mypy understands only X.Y version numbers.
	MYPY_PY_VERSION=$(echo ${{ matrix.python-version }} | cut -d - -f 1 | cut -d . -f 1-2)

That said, the status quo mostly works, and I'm not super invested in this.

[AlexWaygood]

I don't think there's any reason to ever use -dev suffixes in our setup-python version matrixes, because we already have this setting enabled for that action:

typeshed/.github/workflows/tests.yml

Line 52 in 37dceef

allow-prereleases: true

When we start running Python 3.15 in CI, it should work just fine to add 3.15 to the matrix (not 3.15-dev). See https://github.com/actions/setup-python/blob/main/docs/advanced-usage.md#allow-pre-releases. I think it would be fine to just get rid of this hack?

typeshed/.github/workflows/tests.yml

Lines 66 to 68 in 37dceef

	# python-version can sometimes be pinned to a specific version or to "-dev", but
	# mypy understands only X.Y version numbers.
	MYPY_PY_VERSION=$(echo ${{ matrix.python-version }} | cut -d - -f 1 | cut -d . -f 1-2)

[srittau]

We at least need the hack to pin releases to a specific version, for when a point release breaks CI, and there's no quick fix/no maintainer resources available. This happened in the past.