Skip to content

feat: allow custom platform when overriding #2880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
May 29, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,10 @@ END_UNRELEASED_TEMPLATE
* (pypi) Starlark-based evaluation of environment markers (requirements.txt conditionals)
available (not enabled by default) for improved multi-platform build support.
Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it.
* (toolchains) Arbitrary python-build-standalone runtimes can be registered
and activated with custom flags. See the [Registering custom runtimes]
docs and {obj}`single_version_platform_override()` API docs for more
information.

{#v0-0-0-removed}
### Removed
Expand Down
19 changes: 19 additions & 0 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,25 @@ dev_python.override(
register_all_versions = True,
)

# Necessary so single_platform_override with a new version works
dev_python.toolchain(python_version = "3.13.3")

# For testing an arbitrary runtime triggered by a custom flag.
# See //tests/toolchains:custom_platform_toolchain_test
dev_python.single_version_platform_override(
platform = "linux-x86-install-only-stripped",
python_version = "3.13.3",
sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd",
target_compatible_with = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
],
target_settings = [
"@@//tests/support:is_custom_runtime_linux-x86-install-only-stripped",
],
urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"],
)

dev_pip = use_extension(
"//python/extensions:pip.bzl",
"pip",
Expand Down
73 changes: 73 additions & 0 deletions docs/toolchains.md
Original file line number Diff line number Diff line change
Expand Up @@ -236,6 +236,79 @@ existing attributes:
* Adding additional Python versions via {bzl:obj}`python.single_version_override` or
{bzl:obj}`python.single_version_platform_override`.

### Registering custom runtimes

Because the python-build-standalone project has _thousands_ of prebuilt runtimes
available, rules_python only includes popular runtimes in its built in
configurations. If you want to use a runtime that isn't already known to
rules_python then {obj}`single_version_platform_override()` can be used to do
so. In short, it allows specifying an arbitrary URL and using custom flags
to control when a runtime is used.

In the example below, we register a particular python-build-standalone runtime
that is activated for Linux x86 builds when the custom flag
`--//:runtime=my-custom-runtime` is set.

```
# File: MODULE.bazel
bazel_dep(name = "bazel_skylib", version = "1.7.1.")
bazel_dep(name = "rules_python", version = "1.5.0")
python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(python_version="3.13.3")
python.single_version_platform_override(
platform = "my-platform",
python_version = "3.13.3",
sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd",
target_compatible_with = [
"@platforms//os:linux",
"@platforms//cpu:x86_64",
],
target_settings = [
"@@//:runtime=my-custom-runtime",
],
urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"],
)

# File: //:BUILD.bazel

load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
string_flag(
name = "custom_runtime",
build_setting_default = "",
)

config_setting(
name = "is_custom_runtime_linux-x86-install-only-stripped",
flag_values = {
":custom_runtime": "linux-x86-install-only-stripped",
},
)
```

Notes:
- While any URL and archive can be used, it's assumed their content looks how
a python-build-standalone archive looks.
- `python.toolchain()` is required if the version is unknown; if the version
is already known, it can be omitted.
- A "version aware" toolchain is registered, which means the Python version flag
must also match (e.g. `--@rules_python//python/config_settings:python_version=3.13.3`
must be set -- see `minor_mapping` and `is_default` for controls and docs
about version matching and selection).
- The labels in `target_settings` must be absolute; `@@` refers to the main repo.
- The `target_settings` are `config_setting` targets, which means you can
customize how matching occurs.

:::{seealso}
See {obj}`//python/config_settings` for flags rules_python already defines
that can be used with `target_settings`. Some particular ones of note are:
{flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others.
:::

:::{versionadded} VERSION_NEXT_FEATURE
Added support for custom platform names, `target_compatible_with`, and
`target_settings` with `single_version_platform_override`.
:::

### Using defined toolchains from WORKSPACE

It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example
Expand Down
1 change: 1 addition & 0 deletions python/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,7 @@ bzl_library(
name = "versions_bzl",
srcs = ["versions.bzl"],
visibility = ["//:__subpackages__"],
deps = ["//python/private:platform_info_bzl"],
)

# NOTE: Remember to add bzl_library targets to //tests:bzl_libraries
Expand Down
6 changes: 6 additions & 0 deletions python/private/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -241,11 +241,17 @@ bzl_library(
],
)

bzl_library(
name = "platform_info_bzl",
srcs = ["platform_info.bzl"],
)

bzl_library(
name = "python_bzl",
srcs = ["python.bzl"],
deps = [
":full_version_bzl",
":platform_info_bzl",
":python_register_toolchains_bzl",
":pythons_hub_bzl",
":repo_utils_bzl",
Expand Down
34 changes: 34 additions & 0 deletions python/private/platform_info.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
"""Helper to define a struct used to define platform metadata."""

def platform_info(
*,
compatible_with = [],
flag_values = {},
target_settings = [],
os_name,
arch):
"""Creates a struct of platform metadata.

This is just a helper to ensure structs are created the same and
the meaning/values are documented.

Args:
compatible_with: list[str], where the values are string labels. These
are the target_compatible_with values to use with the toolchain
flag_values: dict[str|Label, Any] of config_setting.flag_values
compatible values. DEPRECATED -- use target_settings instead
target_settings: list[str], where the values are string labels. These
are the target_settings values to use with the toolchain.
os_name: str, the os name; must match the name used in `@platfroms//os`
arch: str, the cpu name; must match the name used in `@platforms//cpu`

Returns:
A struct with attributes and values matching the args.
"""
return struct(
compatible_with = compatible_with,
flag_values = flag_values,
target_settings = target_settings,
os_name = os_name,
arch = arch,
)
Loading