get_installed_distributions() on --target

[kevjumba]

Environment

• pip version: 20.2.2
• Python version: 2.7.17
• OS: MacOSX 10.15.5

Description
When downgrading a python package that is already installed by reinstalling it with using pip install --upgrade to a directory, the pip.get_installed_distributions() function gets confused and returns the later version. For example, if Flask 1.1.2 is installed and we change it so using pip install flask==1.1.0 --upgrade, the installation proceeds correctly and the flask version is updated but pip.get_installed_distributions() still returns Flask 1.1.2.

How to Reproduce
Steps to reproduce the behavior:

1. Create a new directory and a requirements.txt file with Flask version 1.1.2.
2. Install using pip install --target lib -r requirements.txt.
3. Change directories into lib and use the python shell.
4. Import get_installed_distributions from pip._internal.utils.misc import get_installed_distributions.
5. Run it on the current directory by using get_installed_distributions(skip=[], paths=['/path/to/lib'])
6. Version should be correct.
7. Now change the requirements.txt file to any version of Flask that is earlier than 1.1.2 (e.g 1.1.0).
8. Run pip install --target lib -r requirements.txt --upgrade.
9. Repeat steps 3-5.
10. Version is still 1.1.2 on Flask.

Expected behavior
The version should be correct. I made sure by importing the flask library and printing out flask.version and confirmed that the installation proceeded correctly. However, the get_installed_distribution() function does not return the correct version.

Output

Detailed in the "To Reproduce" section.

[brainwane]

Thank you for the bug report! Could you please try the beta of the new resolver that is available in pip 20.2 and see whether that helps?

[uranusjr]

I think this is not related to the resolver, but how --target works. If you inspect the directory content after step 8. you’d see both Flask-1.1.0.dist-info and Flask-1.1.2.dist-info are installed. This does not generally happen for pip install, but --target is different because it is not an uninstallable target, so pip has no way to clean up Flask-1.1.2.dist-info when you install 1.1.0.

See #5595, or more generally #4575 for discussion. I’ll close this one since this is a special case to a more general problem, and would be fixed (and can only be fixed) by fixing the more general problem.