Skip to content

uv migration #725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 15 commits into from
10 changes: 5 additions & 5 deletions .github/workflows/build-binaries.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,24 +43,24 @@ jobs:
uses: Swatinem/rust-cache@v2
with:
workspaces: temporalio/bridge -> target

- uses: astral-sh/setup-uv@v5
# Prepare
# Using fixed Poetry version until
# https://github.com/python-poetry/poetry/issues/7611 and
# https://github.com/python-poetry/poetry/pull/7694 are fixed
- run: python -m pip install --upgrade wheel "poetry==1.3.2" poethepoet
- run: poetry install --no-root --all-extras
- run: uv sync --all-extras

# Add the source dist only for Linux x64 for now
- if: ${{ matrix.package-suffix == 'linux-amd64' }}
run: poetry build --format sdist
run: uv build --sdist

# Build and fix the wheel
- run: poetry run cibuildwheel --output-dir dist
- run: uv run cibuildwheel --output-dir dist
- run: poe fix-wheel

# Simple test
- run: poe test-dist-single
- run: uv run pytest -k test_activity_hello

# Upload dist
- uses: actions/upload-artifact@v4
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,9 @@ jobs:
# TODO(cretz): Can upgrade proto when https://github.com/arduino/setup-protoc/issues/99 fixed
version: "23.x"
repo-token: ${{ secrets.GITHUB_TOKEN }}
- run: python -m pip install --upgrade wheel poetry poethepoet
- run: poetry install --no-root --all-extras
- uses: astral-sh/setup-uv@v5
- run: python -m pip install --upgrade wheel poethepoet
- run: uv sync --all-extras
- run: poe bridge-lint
if: ${{ matrix.clippyLinter }}
- run: poe lint
Expand Down Expand Up @@ -82,8 +83,8 @@ jobs:
env:
TEMPORAL_TEST_PROTO3: 1
run: |
poetry add --python 3.9 "protobuf<4"
poetry install --no-root --all-extras
uv add --python 3.9 "protobuf<4"
uv sync --all-extras
poe gen-protos
poe format
[[ -z $(git status --porcelain temporalio) ]] || (git diff temporalio; echo "Protos changed"; exit 1)
Expand All @@ -107,9 +108,9 @@ jobs:

# Runs the sdk features repo tests with this repo's current SDK code
features-tests:
uses: temporalio/features/.github/workflows/python.yaml@main
uses: temporalio/features/.github/workflows/python.yaml@uv
with:
python-repo-path: ${{github.event.pull_request.head.repo.full_name}}
version: ${{github.event.pull_request.head.ref}}
version-is-repo-ref: true
features-repo-ref: python-version-upgrade
features-repo-ref: uv
5 changes: 3 additions & 2 deletions .github/workflows/run-bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,10 @@ jobs:
version: "23.x"
repo-token: ${{ secrets.GITHUB_TOKEN }}

- uses: astral-sh/setup-uv@v5
# Build
- run: python -m pip install --upgrade wheel poetry poethepoet
- run: poetry install --no-root --all-extras
- run: python -m pip install --upgrade wheel poethepoet
- run: uv sync --all-extras
- run: poe build-develop-with-release

# Run a bunch of bench tests. We run multiple times since results vary.
Expand Down
49 changes: 20 additions & 29 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1390,28 +1390,26 @@ The Python SDK is built to work with Python 3.9 and newer. It is built using

To build the SDK from source for use as a dependency, the following prerequisites are required:

* [Python](https://www.python.org/) >= 3.9
* Make sure the latest version of `pip` is in use
* [uv](https://docs.astral.sh/uv/)
* [Rust](https://www.rust-lang.org/)
* [Protobuf Compiler](https://protobuf.dev/)
* [poetry](https://github.com/python-poetry/poetry) (e.g. `python -m pip install poetry`)
* [poe](https://github.com/nat-n/poethepoet) (e.g. `python -m pip install poethepoet`)

macOS note: If errors are encountered, it may be better to install Python and Rust as recommended from their websites
instead of via `brew`.
Use `uv` to install `poe`:
```bash
uv tool install poethepoet
```

With the prerequisites installed, first clone the SDK repository recursively:
Now clone the SDK repository recursively:

```bash
git clone --recursive https://github.com/temporalio/sdk-python.git
cd sdk-python
```

Use `poetry` to install the dependencies with `--no-root` to not install this package (because we still need to build
it):
Install the dependencies:

```bash
poetry install --no-root --all-extras
uv sync --all-extras
```

#### Build
Expand All @@ -1422,7 +1420,7 @@ Now perform the release build:
environment](#local-sdk-development-environment) for the quicker approach to local development).

```bash
poetry build
uv build
```

The compiled wheel doesn't have the exact right tags yet for use, so run this script to fix it:
Expand Down Expand Up @@ -1487,22 +1485,15 @@ It should output:

### Local SDK development environment

For local development, it is often quicker to use debug builds and a local virtual environment.

While not required, it often helps IDEs if we put the virtual environment `.venv` directory in the project itself. This
can be configured system-wide via:

```bash
poetry config virtualenvs.in-project true
```
For local development, it is quicker to use a debug build.

Now perform the same steps as the "Prepare" section above by installing the prerequisites, cloning the project,
installing dependencies, and generating the protobuf code:
Perform the same steps as the "Prepare" section above by installing the prerequisites, cloning the project, and
installing dependencies:

```bash
git clone --recursive https://github.com/temporalio/sdk-python.git
cd sdk-python
poetry install --no-root --all-extras
uv sync --all-extras
```

Now compile the Rust extension in develop mode which is quicker than release mode:
Expand Down Expand Up @@ -1535,14 +1526,14 @@ poe test -s --log-cli-level=DEBUG -k test_sync_activity_thread_cancel_caught
#### Proto Generation and Testing

To allow for backwards compatibility, protobuf code is generated on the 3.x series of the protobuf library. To generate
protobuf code, you must be on Python <= 3.10, and then run `poetry add "protobuf<4"` +
`poetry install --no-root --all-extras`. Then the protobuf files can be generated via `poe gen-protos`. Tests can be run
for protobuf version 3 by setting the `TEMPORAL_TEST_PROTO3` env var to `1` prior to running tests.
protobuf code, you must be on Python <= 3.10, and then run `uv add "protobuf<4"` + `uv sync --all-extras`. Then the
protobuf files can be generated via `poe gen-protos`. Tests can be run for protobuf version 3 by setting the
`TEMPORAL_TEST_PROTO3` env var to `1` prior to running tests.

Do not commit `poetry.lock` or `pyproject.toml` changes. To go back from this downgrade, restore both of those files
and run `poetry install --no-root --all-extras`. Make sure you `poe format` the results.
Do not commit `uv.lock` or `pyproject.toml` changes. To go back from this downgrade, restore both of those files and run
`uv sync --all-extras`. Make sure you `poe format` the results.

For a less system-intrusive approach, you can (note this approach [may have a bug](https://github.com/temporalio/sdk-python/issues/543)):
For a less system-intrusive approach, you can:
```shell
docker build -f scripts/_proto/Dockerfile .
docker run --rm -v "${PWD}/temporalio/api:/api_new" -v "${PWD}/temporalio/bridge/proto:/bridge_new" <just built image sha>
Expand All @@ -1552,7 +1543,7 @@ poe format
### Style

* Mostly [Google Style Guide](https://google.github.io/styleguide/pyguide.html). Notable exceptions:
* We use [Black](https://github.com/psf/black) for formatting, so that takes precedence
* We use [ruff](https://docs.astral.sh/ruff/) for formatting, so that takes precedence
* In tests and example code, can import individual classes/functions to make it more readable. Can also do this for
rarely in library code for some Python common items (e.g. `dataclass` or `partial`), but not allowed to do this for
any `temporalio` packages (except `temporalio.types`) or any classes/functions that aren't clear when unqualified.
Expand Down
Loading
Loading