pip 22.1.2 breaks test_venv of CPython 3.11+

[stratakis]

Description

After trying to update pip from 22.0.4 to 22.1.2 in Fedora and doing an impact check, the cpython test suite error's out:

test_with_pip (test.test_venv.EnsurePipTest.test_with_pip) ... /tmp/tmpuz5z3j31/bin/python: No module named pip

ERROR

======================================================================
ERROR: test_with_pip (test.test_venv.EnsurePipTest.test_with_pip)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/cpython/Lib/test/test_venv.py", line 625, in test_with_pip
    self.do_test_with_pip(False)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/cpython/Lib/test/test_venv.py", line 574, in do_test_with_pip
    out, err = check_output([envpy, '-W', 'ignore::DeprecationWarning',
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/cpython/Lib/test/test_venv.py", line 52, in check_output
    raise subprocess.CalledProcessError(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
subprocess.CalledProcessError: Command '['/tmp/tmpuz5z3j31/bin/python', '-W', 'ignore::DeprecationWarning', '-W', 'ignore::ImportWarning', '-I', '-m', 'pip', '--version']' returned non-zero exit status 1.

----------------------------------------------------------------------
Ran 1 test in 1.549s

FAILED (errors=1)
test test_venv failed

After testing it a bit more, it only happens on Python 3.11 and later.

Expected behavior

No response

pip version

22.1.2

Python version

3.11+

OS

Fedora Linux

How to Reproduce

Get the CPython source code.

From Lib/ensurepip/_bundled remove the 22.0.4 pip wheel and replace it with the latest version.

Make sure that the Lib/ensurepip/__init__.py is updated to reflect the new version:

-_PIP_VERSION = "22.0.4"
+_PIP_VERSION = "22.1.2"

Compile CPython and run the tests:
$ ./configure && make && ./python -m test -vvw test_venv

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[uranusjr]

It seems like the command is raising unexpected DeprecationWarning. Would you be able to investigate a bit and find out what exactly is the warning causing the failure?

[hroncok]

Installed Python 3.11.0b3 and pip 22.1.2 wheel that ensurepip will use to Fedora 37 chroot:

$ mock -r fedora-rawhide-x86_64 install python3.11-devel https://kojipkgs.fedoraproject.org//work/tasks/7505/88887505/python-pip-wheel-22.1.2-1.fc37.noarch.rpm

Inside that chroot:

<mock-chroot> sh-5.1$ python3.11 -m venv venv
<mock-chroot> sh-5.1$ . venv/bin/activate
(venv) <mock-chroot> sh-5.1$ python -m pip --version
/builddir/venv/bin/python: No module named pip
(venv) <mock-chroot> sh-5.1$ python -m ensurepip
Looking in links: /tmp/tmpq77hrovr
Requirement already satisfied: setuptools in /tmp/tmpq77hrovr/setuptools-62.6.0-py3-none-any.whl (62.6.0)
Requirement already satisfied: pip in /tmp/tmpq77hrovr/pip-22.1.2-py3-none-any.whl (22.1.2)
(venv) <mock-chroot> sh-5.1$ deactivate

<mock-chroot> sh-5.1$ tree venv/
venv/
├── bin
│   ├── Activate.ps1
│   ├── activate
│   ├── activate.csh
│   ├── activate.fish
│   ├── python -> python3.11
│   ├── python3 -> python3.11
│   └── python3.11 -> /usr/bin/python3.11
├── include
│   └── python3.11
├── lib
│   └── python3.11
│       └── site-packages
├── lib64 -> lib
└── pyvenv.cfg

7 directories, 8 files

<mock-chroot> sh-5.1$ python3.11 -m ensurepip
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /tmp/tmpu2ouxnzq
Requirement already satisfied: setuptools in /tmp/tmpu2ouxnzq/setuptools-62.6.0-py3-none-any.whl (62.6.0)
Requirement already satisfied: pip in /tmp/tmpu2ouxnzq/pip-22.1.2-py3-none-any.whl (22.1.2)
<mock-chroot> sh-5.1$ python3.11 -m pip --version
/usr/bin/python3.11: No module named pip

I think the problem is not a DeprecationWarning (the testes are explicitly ignoring those after all) but a false Requirement already satisfied problem.

I think that ensurepip adds the pip.whl to sys.path and installs pip. That previously worked, now it is reported as installed.

[pfmoore]

I think the problem is not a DeprecationWarning (the testes are explicitly ignoring those after all) but a false Requirement already satisfied problem.

If that's the case, it shouldn't be specific to Python 3.11. Does this happen with older versions of Python as well?

[hroncok]

Python 3.11:

$ mkdir 22.1.2
$ mkdir 22.0.4
$ mv ~/.../pip-22.0.4-py3-none-any.whl 22.0.4/
$ mv ~/.../pip-22.1.2-py3-none-any.whl 22.1.2/
$ python3.11 -m venv --without-pip venv
$ . venv/bin/activate

(venv) $ python -m pip
/.../tmp/venv/bin/python: No module named pip

(venv) $ PYTHONPATH=$PWD/22.1.2/pip-22.1.2-py3-none-any.whl python -m pip list
Package Version
------- -------
pip     22.1.2
(venv) $ PYTHONPATH=$PWD/22.1.2/pip-22.1.2-py3-none-any.whl python -m pip install 22.1.2/pip-22.1.2-py3-none-any.whl
Processing ./22.1.2/pip-22.1.2-py3-none-any.whl
pip is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.

(venv) $ PYTHONPATH=$PWD/22.0.4/pip-22.0.4-py3-none-any.whl python -m pip list
(venv) $ PYTHONPATH=$PWD/22.0.4/pip-22.0.4-py3-none-any.whl python -m pip install 22.0.4/pip-22.0.4-py3-none-any.whl
Processing ./22.0.4/pip-22.0.4-py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-22.0.4
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/.../tmp/venv/bin/python -m pip install --upgrade pip' command.

[uranusjr]

That previously worked, now it is reported as installed.

Ah this is probably related to the importlib.metadata transition. I think there’s an issue on that.

[uranusjr]

Yup #11183. Let’s merge these two then.

[hroncok]

Python 3.10:

$ python3.10 -m venv --without-pip venv 
$ . venv/bin/activate
(venv) $ PYTHONPATH=$PWD/22.1.2/pip-22.1.2-py3-none-any.whl python -m pip list
(empty output)
(venv) $ PYTHONPATH=$PWD/22.0.4/pip-22.0.4-py3-none-any.whl python -m pip list
(empty output)