Tox/Pyenv failure upon creating virtualenv

[berdario]

even if skip_missing_interpreters=True

This didn't use to be the case. The exact error message is:

 ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.5: b"pyenv: python3.5: command not found\\n\\nThe `python3.5\' command exists in these Python versions:\\n  3.5\\n  3.5.3\\n\\n"',)

See:

https://travis-ci.org/berdario/pew/jobs/281757929

I unfortunately don't have time to investigate much into it, but the TravisCI build link should have everything you need: you can see that previous builds passed just fine when the interpreter didn't exist, and in the repository you can find the tox.ini

[berdario]

I updated the title, this might not actually be due to a missing interpreter, but it's due to a different regression

[obestwalter]

Hi @berdario thanks for the report, since when is that failing? Did anything change on your side?

[obestwalter]

Looking at the error and the project there are several things that leave me veryconfused.

The travis configuration does use pytest directly, so I have no idea why this is running tox at all. Can you explain?

Even stranger: when I look at the current master build, my expectation fits what is going on in the build: pytest is used directly - no tox: https://travis-ci.org/berdario/pew/jobs/281829400

The error above contains the original exception message , which is

pyenv: python3.5: command not found\\n\\nThe python3.5 command exists in these Python versions: 3.5  3.5.3

This is a known problem with pyenv that you need to use either tox-pyenv or register all python versions to make them accesssible from tox. But what does pyenv have to do with all of this? If Travis uses pyenv under the hood to provide the python versions, than there might be something going wrong.

I have to admit that this is the weirdest thing I have seen in a long time, so I guess that there are things happening under the hood I don't understand.

[berdario]

Sorry for the confusion, the .travis.yml has just been updated. This was its state as of yesterday:

https://github.com/berdario/pew/blob/97f9a78c2dd6615cbb1ce29238327ff900b19dad/.travis.yml

Yeah, it looks quite weird, since the build always used to succeed. I never used pyenv, and by seeing the error I thus thought that it was tox that was using pyenv

I guess it might be possible that in the last 2 months Travis CI itself might have switched to use pyenv (from whatever solution they might have been previously using).

Anyhow, I'd be fine on such Pythons to not be registered and available to Tox, but ideally it should then preserve the behavior of skip_missing_interpreters, and avoid failing the invokation... Do you think it would be feasible to special case this?

(it's not a big deal for me anymore, anyhow.... Since as you've seen we're not using Tox on TravisCI anymore)

Thank you!

[obestwalter]

Thanks for the clarification.

I just had a look and Travis is actually using pyenv since 2014: https://blog.travis-ci.com/2014-04-28-upcoming-build-environment-updates/

Summerized Build Environment Updates
[...]
Python:
Pythons now installed with pyenv
3.4.0 is now available

So you might have triggered a corner case in their setup, as I never even have seen that they use pyenv under the hood, so this must work quite smoothly usually.

skip_missing_interpreters is a different story. What happens here is an actual error that needs to be resolved. Don't know how yet though. It would be handy to have a reproducer for that that is independent from Travis first.

[asottile]

This is a duplicate to an issue I opened!

travis-ci/travis-ci#8363

[obestwalter]

Ah, so a Travis/pyenv config bug it is - seems to affect a lot of builds.

[schinckel]

I don't think it's a travis bug: I'm hitting the same issue with a local tox setup. Still trying to figure out what is actually happening.

[schinckel]

Okay, in my case, I had a .python-version file at my project root: I'm starting to prepare to migrate to Python 3, and I needed to add 3.6.2 to the end of that file. Then generating a virtualenv that uses that interpreter works.

[obestwalter]

Hi @schinckel - thanks for following up on that. Juggling with different versions and using pyenv has its own pitfalls I am afraid.

[gauravv7]

Hi all,

This errors occurs for me too over travis. but locally tox runs fine. I have not checked running travis' docker image locally.
I have 3.5.2 python installed as well as 2.7 and both test-env runs fine with tox no error. but on travis it fails.. since I think travis doesn't prepare images with multiple versions of python in 1 machine run.
My travis is failing for 2.7 only when running 3.5 test, although there is no 'interpreter not found' error but python3.5 is not picked up while running p35 test sessions.

[obestwalter]

Hi @gauravv7, this is the same upstream problem that @asottile linked I guess: see travis-ci/travis-ci#8363

[obestwalter]

Reopened because the upstream bug at travis-ci/travis-ci#8363 is also still open.

[gauravv7]

@obestwalter thanks for notifying here, I tried a fix for the same, and can put up a PR. It would be an honor to contribute further if you would allow me to :)

[asottile]

@obestwalter ah, I usually err on closing issues that are not actionable here and just upstream problems (so our tracker isn't even more cluttered than it already is 😆). Up to you though :)

[obestwalter]

Hey @asottile I think it is better to have them marked as upstream and keep them open, so that people looking for open bugs with that problem here hopefully find the open upstream bug :)

[obestwalter]

Hi @gauravv7, what kind of fix would you have in mind?

[ssbarnea]

I think that this issue had nothing to do with travis itself because I found a way to reproduce it with my own rtox setup. Mainly I am randomly getting this error while running tox remotely on a Linux (RHEL-7.4) machine where I have pyenv installed.

My main machine is MacOS where I also have tox and pyenv, a setup that works quite well as I have 6-7 different versions of python which I am testing with.

To be clear: the chance of triggering the bug is like 1/5 executions, even if I wipe the .tox folder between each execution.

The funny part is that his is never happening if I am manually running the command on the target machine, but when I run tox remotely (ssh) which makes me think that TTY type, environment var or shell can have an impact.

[cicorias]

i was able to fix this in a docker build here
https://github.com/Microsoft/agogosml/blob/7a41e4d136534c9b8caf93dc1a87cb52a62aa845/agogosml_cli/Dockerfile

The "gist" of it all is below:

RUN curl https://pyenv.run | bash
#see https://www.loganasherjones.com/2018/03/setting-up-your-python-environment/

RUN pyenv install 3.5.6
RUN pyenv install 3.6.8
RUN pyenv install 3.7.2
RUN pip install pytest tox
RUN pip install tox-globinterpreter

WORKDIR /usr/src/agogosml_cli

RUN echo "3.5.6" >> .python-version
RUN echo "3.6.8" >> .python-version
RUN echo "3.7.2" >> .python-version

GitHub

Microsoft/agogosml

agogosml is a flexible data processing pipeline that addresses the common need for operationalizing ML models at scale - Microsoft/agogosml

[asottile]

@cicorias another fix is to use pyenv global ... (that way those interpreters are available wherever and you don't need to write a bunch of .python-version files just to use them 👍

[gaborbernat]

I don't see any actionable item on our side, so I'll close this for now.