Only use hashes in requirements.txt if there are no VCS repositories

[sumanthratna]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option). (n/a)

• OS version and name: macOS 10.5.5 Beta

• Poetry version: 1.0.5

• Contents of pyproject.toml file:

[tool.poetry]
name = "test-poetry-proj"
version = "0.1.0"
description = "desc"
authors = ["Sumanth Ratna <sumanthratna@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.6.1"
numpy = "^1.18.4"
hyperopt = {git = "https://github.com/hyperopt/hyperopt.git"}


[tool.poetry.dev-dependencies]
pytest = "^5.4.2"

[tool.poetry.scripts]

[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

Issue

When I run poetry export -f requirements.txt > requirements.txt, the resulting requirements.txt file has a bunch of hashes. Unfortunately, that means I can't run pip install -r requirements.txt on another machine, because it returns this error:

ERROR: Can't verify hashes for these requirements because we don't have a way to hash version control repositories:
    hyperopt from git+https://github.com/hyperopt/hyperopt.git#egg=hyperopt (from -r requirements.txt (line 192))

I'm labeling this as a bug report instead of a feature request because this seems like a fault in the way Poetry behaves when exporting.

This is still an issue. Sounds simple enough to fix.

[ryancausey]

Just ran into this while trying to export a requirements.txt for a two-stage docker build. Is there something that needs to be done to help the open PR along?

[hterik]

There is a --without-hashes option for poetry export one can use if you know beforehand there are git requirements in your pyproject.toml.

[neersighted]

This issue has been migrated as the underlying code has moved to a first-party plugin.

[shai4lt]

In my opinion, whatever is done here should be user-controllable. My ideal is, when there are mixed (index and VCS) requirements

• If --without-hashes was specified, export without hashes, no problem
• Else, there should be options that say to export "only vcs" (implies without-hashes) or "only index" requirements
• If none of the above was specified, error out

I believe these options are useful, because in situations where one may rely on the VCS repo (say, the repo is owned by the owners of the project we're exporting), it is possible to export both separately, pip install --without-deps the VCS requirements, and then pip install the index requirements; relying on poetry's locking to make sure all the required dependencies are present.

[shangxiao]

So with a little grepping we can actually resolve this ourselves.

This works for me (alter as desired):

poetry export -f requirements.txt > requirements-base.txt
grep 'git\+' requirements-base.txt > requirements-vcs.txt
grep 'git\+' -v requirements-base.txt > requirements-hashed.txt

pip install -r requirements-vcs.txt
pip install -r requirements-hashed.txt

This is due to pip install implicitly setting --require-hashes if any of the requirements in a file has --hash:

pip install -h
  ...
  --require-hashes            Require a hash to check each requirement against, for repeatable installs. This
                              option is implied when any package in a requirements file has a --hash option.

Edit: I don't know whether poetry will export a line-wrapped vcs requirement but if it did then we'd need to be a little more clever than simply grepping. Ideally it would be something Poetry does.

[shai4lt]

Just one note:

poetry export -f requirements.txt > requirements-base.txt
grep 'git\+' requirements-base.txt > requirements-vcs.txt
grep 'git\+' -v requirements-base.txt > requirements-hashed.txt

pip install -r requirements-vcs.txt
pip install -r requirements-hashed.txt

When doing this, you need to be careful -- when installing the vcs requirements, if you do it as you wrote above, you open yourself up to sneaking in uncontrolled (2nd-order) dependencies. To prevent this, use

pip install --without-deps -r requirements-vcs.txt
pip install -r requirements-hashed.txt

After locking, the exported requirements (requirements-base.txt) should already include all the dependencies for the VCS dependencies; and that's how you want them installed.

Avoid adding --without-deps to the command for requirements-hashed.txt. Running without it serves as a sort of "sanity check" -- if all is well, no dependency needs to be added and the flag is meaningless; if any dependency does need to be added, it will not be hashed and cause the installation to fail. But needing to add a dependency at that stage indicates a failure in locking or exporting, and then you're better off failing the installation and investigating.