--ignore-installed not working with pip install -e

[asmodehn]

• Pip version: 9.0.1
• Python version: 2.7
• Operating system: Ubuntu

It seems the recent new behavior of pip install -e tries to remove existing installed packages.
Ref : https://github.com/pypa/pip/pull/3898/files

However this doesn't seems to honor the --ignore-installed flag.

I had this error when using--prefix. I was expecting the pip install to ignore whatever was installed on the system, but it tried to uninstall and didn't have permissions.

More detailed information here.

However I am doing this in a custom tool with a somehow esoteric environment.
It would be good if someone could confirm/infirm in a "usual" python environment...

[pfmoore]

--ignore-installed just ignores installed packages for the purposes of deciding what to install, not for actually doing the install. When installing a new version of a package, pip will always remove existing versions (unless you are using --target) as Python won't allow multiple versions of a package to be installed at the same time.

[asmodehn]

Oh interesting to know. I m using --prefix and I understood it was the proper way to install in a separate directory ( --target works in some strange ways as far as I can tell and I thought I should favor using prefix ) However when I use --prefix and ignore install I cannot install a package that already exists on the system. I would think that the same exception as for --target would also apply here...

…

On Jan 20, 2017 6:34 PM, "Paul Moore" ***@***.***> wrote: --ignore-installed just ignores installed packages for the purposes of deciding *what* to install, not for actually doing the install. When installing a new version of a package, pip will always remove existing versions (unless you are using --target) as Python won't allow multiple versions of a package to be installed at the same time. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4243 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANgSLRGlsk85dU6NTqmR7nMXPCLMgWWks5rUH-egaJpZM4LpGNH> .

[pfmoore]

--prefix is for when you want to install in a non-standard site-packages location, so your "prefix" directory contains the standard structure underneath, and is expected to be part of your sys.path. So clashes with what is currently on sys.path are a problem, and hence if you're installing a newer copy of something already present, that older copy should be uninstalled first.

--target is for just placing a package somewhere in the filesystem. It may or may not end up on your sys.path (it's often used for vendoring or for bundling dependencies in a zipapp) and what's on your sys.path is expected to be irrelevant (because you won't be using the installed package in the runtime environment you currently have).

There's also --root which is somehow similar to but different from prefix. I freely admit I don't understand the distinction (I don't personally use --prefix or --root).

Hopefully this helps you - from what I can see the "catkin pip" environment you hit this in may be doing nonstandard things with sys.path and so you may have to work to get the behaviour you want from standard pip.

[asmodehn]

Thank you. very useful indeed. catkin_pip is an attempt at modernising catkin ( old cmake-based way of managing python packages for http://www.ros.org/ ) by bringing pip in, and letting pip do the heavy lifting. However the tricky part is not breaking user expectations. The problematic expectation here is that catkin allows multiple versions of the same package to be installed on the system, one eventually overriding another one, depending on the order of the paths in PYTHONPATH. This is usually used for development, where version N+1 will be developped and worked on while version N ( and all dependencies ) are installed on the system ( in various locations ). In that context, one can say that one element of the PYTHONPATH is an "environment". Coming from a python background, it is as if you can "stack up" multiple virtualenvs... I achieved that previously by using --prefix and fiddling with PYTHONPATH, but since an editable install now will attempt to remove the installed package, I need to find another way to do that, or provide a decent alternative. As far as I know, such a feature (stacking virtualenvs) is not available, and deemed too confusing for users and therefore unsuitable. Preventing user to have the same package installed multiple times in the same system is probably the saner option, however I ll need to come up with a workflow that make sense in a context where people are used to install one package multiple time from different places into different places. Anyway thanks again for the explanation. Thanks also for any advice you may have, and feel free to close this issue I ll need to do more thinking about what would be the best way for catkin_pip to move to pip > 9.0 -- AlexV

…

On Sat, Jan 21, 2017 at 7:33 PM, Paul Moore ***@***.***> wrote: --prefix is for when you want to install in a non-standard site-packages location, so your "prefix" directory contains the standard structure underneath, and is expected to be part of your sys.path. So clashes with what is currently on sys.path are a problem, and hence if you're installing a newer copy of something already present, that older copy should be uninstalled first. --target is for just placing a package somewhere in the filesystem. It may or may not end up on your sys.path (it's often used for vendoring or for bundling dependencies in a zipapp) and what's on your sys.path is expected to be irrelevant (because you won't be using the installed package in the runtime environment you currently have). There's also --root which is somehow similar to but different from prefix. I freely admit I don't understand the distinction (I don't personally use --prefix or --root). Hopefully this helps you - from what I can see the "catkin pip" environment you hit this in may be doing nonstandard things with sys.path and so you may have to work to get the behaviour you want from standard pip. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#4243 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AANgSLPheIwz0DHdZutoVWLzY6Nvnnnnks5rUd7qgaJpZM4LpGNH> .

[pfmoore]

The problematic expectation here is that catkin allows multiple versions of
the same package to be installed on the system, one eventually overriding
another one, depending on the order of the paths in PYTHONPATH.

Yes, I thought that was what you were doing. This is not really how PYTHONPATH is meant to be used, so you will hit issues - it's not specifically a pip problem, it's how core Python handles sys.path. For example, if version 1 of package foo exists on sys.path before version 2, but version 2 introduced a subpackage foo.bar, you could end up importing version 1 of foo but version 2 of foo.bar (as there's no version 1 foo.bar to shadow it). That's very rare in practice, and so isn't likely to be a practical issue for you, but for general tools like pip we need to consider the rare corner cases.

Setuptools used to have a system for supporting multi-versioned packages via eggs. AFAIK, it's rarely if ever used today, and is viewed as an experiment that didn't work out. You may find some information by digging into the history of that approach (your specific needs may well mean that you can overcome the difficulties they had, so don't assume that the fact that their approach didn't take off means you're doomed!)

As per your request, I'll close this issue, but feel free to open new ones (or even reopen this one) if you have other questions or think you've found problems. I'm sure there are places where pip does the wrong thing, as this isn't a frequently used are of the code. And if nothing else, the discussion could suggest ways we could improve the docs!