Add auto-publishing to PyPI #487
Conversation
| on: | ||
| tags: true | ||
| user: $PYPI_USER | ||
| password: $PYPI_PASSWORD |
There was a problem hiding this comment.
The Travis docs mention "It is also possible, but not recommended, to use PyPI user and password, instead of token" -- though they don't actually mention the advantage of one over the other.
There was a problem hiding this comment.
They are more secure because:
- Are longer so they probably take longer to brute force
- And if they are hacked in some way, they are limited to specific project and can be revoked more easily
With this PR, it is possible to use password or token, because when using token you just set username to __token__ and password to actual token and this can be done with environment variables.
| provider: pypi | ||
| skip_existing: true | ||
| on: | ||
| tags: true |
There was a problem hiding this comment.
on: tags: true will publish from any tag on any branch, no?
Maybe it would be more desirable to only publish when something on master is tagged? https://docs.travis-ci.com/user/deployment#conditional-releases-with-on. Also, not sure if possible, but Travis may offer a way to match the tag to only a final release (e.g. 1.2.3, not 1.2.3.dev0) as defined by PEP 440.
There was a problem hiding this comment.
on: tags: truewill publish from any tag on any branch, no?
Yes it will publish on any tag on any branch. But because tags are almost always published on master, it shouldn't cause much problems. And even it is would trigger on multiple branches for same tag, it won't deploy it multiple times because of skip_existing: true.
I think that Travis CI allows conditional deployments to be either on all tags on all branches or commits on specific branch, so publishing only on tags on master could be harder. It may be possible with custom condition but I'm now sure how exactly will this work.
Also, not sure if possible, but Travis may offer a way to match the tag to only a final release (e.g. 1.2.3, not 1.2.3.dev0) as defined by PEP 440.
It's probably also possible with custom condition. But I also don't know if this is really needed because pip and PyPI will handle pre-releases and not install them unless specifically required by user.
There was a problem hiding this comment.
I think that Travis CI allows conditional deployments to be either on all tags on all branches or commits on specific branch, so publishing only on tags on master could be harder. It may be possible with custom condition but I'm now sure how exactly will this work.
So, I am not a Travis expert at all, but reading between the lines here,
it looks like this may be satisfied with:
on:
branch: staging
tags: true
It's probably also possible with custom condition. But I also don't know if this is really needed because pip and PyPI will handle pre-releases and not install them unless specifically required by user.
For this, I was more getting at the possibility that a tag is pushed that doesn't represent a PEP-440 compliant version at all, such as git tag -a "this-is-not-a-version" ... git push ... --follow-tags. I agree with you that it is fine to upload development, pre-release, and post-release tags because PyPI very much can handle them as you point out.
There was a problem hiding this comment.
I found this:
tags: true: deployment is triggered if and only if$TRAVIS_TAGis set. Depending on your workflow, you may set$TRAVIS_TAGexplicitly, even if this is a non-tag build when it was initiated. This causes the branch condition to be ignored.
However, I don't know if this means that branch condition is ignored only if you manually set $TRAVIS_TAG or in any case when $TRAVIS_TAG is set.
| tags: true | ||
| user: $PYPI_USER | ||
| password: $PYPI_PASSWORD | ||
| distributions: "sdist bdist_wheel" |
There was a problem hiding this comment.
As a bonus here, you could add a call to check-wheel-contents on the resulting wheel artifact - in this case it might look like:
check-wheel-contents --toplevel=toolz,tlz dist/toolz-*.whl
There was a problem hiding this comment.
This is harder to to because Travis will build source and wheel distributions and then immediately publish it to PyPI, without any place to run this check. But it may be possible to solve it with manually building source and wheels, checking wheels and publishing them using normal commands.
There was a problem hiding this comment.
True, I did not consider that. I guess at best it would be a "post-upload" sanity check that you could remedy by fixing and releasing a subsequent patch version that addresses whatever issue was found in the wheel. The alternative would be to execute a job prior to deploy that builds the wheel and runs check-wheel-contents on that, then only deploy s if check-wheel-contents exits 0. But I concede that's kind of a hassle to build the wheel twice.
There was a problem hiding this comment.
I though about adding check-wheel-contents to after_deploy section. However, "after_deploy does not affect the status of the build" so you will have to manually check build output.
| @@ -34,5 +34,15 @@ script: | |||
| after_success: | |||
| - if [[ $TRAVIS_PYTHON_VERSION != pypy* ]]; then pip install coveralls --use-mirrors ; coveralls ; fi | |||
|
|
|||
| # publish new version to pypi | |||
| deploy: | |||
| provider: pypi | |||
There was a problem hiding this comment.
If desired, you could test this first with Test PyPI:
https://docs.travis-ci.com/user/deployment/pypi/#releasing-to-a-self-hosted-pypi
E.g. I think that would be:
server: https://test.pypi.org/simple/
|
All of this looks great. Much appreciated. Thanks for educating me. This looks ready to go. As a matter of convention or protocol, we already only tag the master branch for releases, so I don't think this will be a problem. I'll fill out the username and password field with a secured token after merging. As a note to self, the PyPI token is too large for the |
This completes the work from: pytoolz#487 Hopefully this works!
This adds auto-publishing to PyPI. It also adds publishing wheels and closes #466.
Currently, it seems that new versions are published manually, but this change will automatically publish new version on every Git tag and will contain both source and wheels. Because module is pure-Python, wheels can be for more Python versions so there isn't any need for complicated build scripts.
To enable this,
PYPI_USERandPYPI_PASSWORDshould be set in Travis CI config as secret/private environment variables. First option is to just set it to PyPI username and password, the second one (which is recommended and more secure), is to set username to__token__and then set password to PyPI token as described here.