pip help * prints keyring log on GNU/Linux

[McSinyx]

Environment

• pip version: reproducible on master as well as Debian downstream (20.0.2)
• Python version: reproducible on 2.7, 3.7 and 3.8, but I think it applies to others as well
• OS: Debian GNU/Linux testing on amd64

The log seems to be from keyring, since without the package installed the bug disappears.

Edit: This is not reproducible on latest keyring, i.e. version 21.2.0. However the last version supporting Python 2 (keyring 18.0.1, which is also the one shipped for Debian testing for Python 3 at the moment) shows this defect if anyone wants to investigate a bit futher.

Description

This only seems to happen to some pip help <subcommand>, not pip <subcommand> -h or any operation by subcommands:

$ pip help list
Loading KWallet
...
Loading pyfs

Usage:   
  pip list [options]
...
$ pip list -h

Usage:   
  pip list [options]
...
$ pip list
Package                       Version        Location
----------------------------- -------------- ----------------------
alabaster                     0.7.12
...

Affected subcommands include install, download, uninstall, list, search and wheel.

Expected behavior

The loading of keying backends should not be logged, at least at the normal level.

How to Reproduce

1. Install Python package keyring
2. Run pip help {install,download,uninstall,list,search,wheel}

After bisecting I found that the log is printed within the call to commands.create_commands in commands.help. However, the log seems to be printed from another process since wrapping contextlib[2].redirect_stdout (the log is in stdout) around that line has no effect.

[deveshks]

I was able reproduce this on MacOS Catalina with python 3.8 and the latest master by using pip help list -v, and not pip help list, I also saw a few VCS backend logs being printed

$ python --version
Python 3.8.2
$ pip --version
pip 20.1.dev1 from /Users/devesh/pip/src/pip (python 3.8)

$ pip help list

Usage:   
  pip list [options]
....

$ pip help list -v
Registered VCS backend: bzr
Registered VCS backend: git
Registered VCS backend: hg
Registered VCS backend: svn
Loading KWallet
Loading SecretService
Loading Windows
Loading chainer
Loading macOS

Usage:   
  pip list [options]

Description:
  List installed packages, including editables.
.....

[McSinyx]

Thanks for the additional output. Now we need someone with Windows to complete the trilogy (to find out if it only happens on GNU/Linux, or that it only doesn't occur on macOS).

[NoahGorny]

I can approve @deveshks on wsl ubuntu, but honestly if someone does pip help {command} -v it means he wants verbose output and thats okay. I think that this issue was fixed by moving some logs into "debug", so I think this is pretty much resolved.

[McSinyx]

I can approve deveshks on wsl ubuntu

How about on normal Windows? I'm curious if it's Debian-environment-specific bug, though I myself don't consider behavior on WSL trust worthy. Just to restate the issue, it's with that the supposed-to-be-part-of-verbose-log is leaked into normal log on pip help *, and the logging of that output is not even from pip, but keyring, so we might need something more specific than the general idea of

moving some logs into "debug"

Anyway, since you're on Windows, would love to see pip help * behavior there.

[pfmoore]

I don't see this behaviour on Windows. I did

pew mktmpenv
pip install keyring
pip help list

The "Loading KWallet" output shows up in pip help list -v but not in plain pip help list.

[pfmoore]

The message seems to come from keyring.backend in _load_plugins. But it's a debug-level log, so I don't see why it would be different.

Also, I just tried on Fedora in docker:

python3 -m ensurepip
python3 -m pip install keyring
python3 -m pip help list

and don't see this issue. Silly question, but could there be a PIP_VERBOSE environment variable or config file entry around somewhere?

[pfmoore]

I also can't reproduce on Ubuntu in docker.

[McSinyx]

Thank you for the quick response, it made me suspected my results an investigate a bit further

# apt show python3-keyring
Package: python3-keyring
Version: 18.0.1-2
$ pip3 help install
Loading ...
# aptitude purge python3-keyring
$ pip3 install keyring
$ pip3 show keyring
Name: keyring
Version: 21.2.0
$ pip3 help install
#no more loading
$ pip2 install keyring
$ pip2 show keyring
Name: keyring
Version: 18.0.1
$ pip2 install keyring==21
...
ERROR: Could not find a version that satisfies the requirement keyring==21.* (from versions: 0.1, ..., 18.0.1)
ERROR: No matching distribution found for keyring==21.*
$ pip2 help install
Loading ...

Seems like the bug is fixed for current keyring, so if other agree, I'll go ahead and close it now. However, somehow pip2 only recognize keyring<=18.0.1 although up to 21.1.0 provide py2.py3 wheel, I have no idea.

Edit: I took a look at keyring installation log using and keyring>=19 require Python 3.5, so there's nothing we can do, closing this now. Thanks everyone for your attention and sorry for the noise.

[pfmoore]

Nice bit of analysis :-)

[matheussouza9]

Uninstall python3-keyring may remove important packages.

The following packages will be REMOVED:
  python3-entrypoints{u} python3-keyring{p} python3-secretstorage{u}
0 packages upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 237 kB will be freed.
The following packages have unmet dependencies:
 python3-launchpadlib : Depends: python3-keyring (>= 0.5) but it is not going to be installed
The following actions will resolve these dependencies:

     Remove the following packages:
1)     apport [2.20.11-0ubuntu27.10 (focal-updates, now)]
2)     python3-apport [2.20.11-0ubuntu27.10 (focal-updates, now)]
3)     python3-launchpadlib [1.10.13-1 (focal, now)]
4)     ubuntu-server [1.450.2 (focal-updates, now)]
5)     xserver-xorg [1:7.7+19ubuntu14 (focal, now)]

     Leave the following dependencies unresolved:
6)     apport-symptoms recommends apport
7)     gdm3 recommends xserver-xorg

Instead doing that, try this solution: #8485 (comment)

[McSinyx]

Thanks for the heads-up, @matheussouza9.

Instead doing that, try [...]

Since this seems to be directed at me, FYI xserver-xorg depending on apport depending on python3-launchpadlib depending on python3-keyring is Ubuntu-specific (and IMHO absurd) and I use Debian. Ubuntu users stumbling upon this thread may very well benefit from your comment though.