Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources #10939

MeitarR · 2025-01-24T15:40:43Z

Summary

Add an option to overwrite the list of available Python downloads from a local JSON file by using the environment variable UV_PYTHON_DOWNLOADS_JSON_URL

as an experimental support for providing custom sources for Python distribution binaries #8015

related #10203

I probably should make the JSON to be fetched from a remote URL instead of a local file.
please let me know what you think and I will modify the code accordingly.

Test Plan

normal run

root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.14.0a4+freethreaded-linux-x86_64-gnu    <download available>
cpython-3.14.0a4-linux-x86_64-gnu                 <download available>
cpython-3.13.1+freethreaded-linux-x86_64-gnu      <download available>
cpython-3.13.1-linux-x86_64-gnu                   <download available>
cpython-3.12.8-linux-x86_64-gnu                   <download available>
cpython-3.11.11-linux-x86_64-gnu                  <download available>
cpython-3.10.16-linux-x86_64-gnu                  <download available>
cpython-3.9.21-linux-x86_64-gnu                   <download available>
cpython-3.8.20-linux-x86_64-gnu                   <download available>
cpython-3.7.9-linux-x86_64-gnu                    <download available>
pypy-3.10.14-linux-x86_64-gnu                     <download available>
pypy-3.9.19-linux-x86_64-gnu                      <download available>
pypy-3.8.16-linux-x86_64-gnu                      <download available>
pypy-3.7.13-linux-x86_64-gnu                      <download available>

empty JSON file

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{}
root@75c66494ba8b:/# /code/target/release/uv python list
root@75c66494ba8b:/#

JSON file with valid version

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json 
root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL 
{
  "cpython-3.11.9-linux-x86_64-gnu": {
    "name": "cpython",
    "arch": {
      "family": "x86_64",
      "variant": null
    },
    "os": "linux",
    "libc": "gnu",
    "major": 3,
    "minor": 11,
    "patch": 9,
    "prerelease": "",
    "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz",
    "sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9",
    "variant": null
  }
}
root@75c66494ba8b:/# /code/target/release/uv python list
cpython-3.11.9-linux-x86_64-gnu    <download available>
root@75c66494ba8b:/#

Remote Path

root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json 
root@75c66494ba8b:/# /code/target/release/uv python list
error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix)

zanieb · 2025-01-24T22:32:17Z

crates/uv-python/src/downloads.rs

-            // linked), so we pretend they don't exist. https://github.com/astral-sh/uv/issues/4242
-            .filter(|download| download.key.libc != Libc::Some(target_lexicon::Environment::Musl))
+        PYTHON_DOWNLOADS.get_or_init(|| {
+            if let Ok(json_path) = std::env::var("UV_PYTHON_DOWNLOADS_JSON_PATH") {


Note this variable will need to be defined in crates/uv-static/src/env_vars.rs

I wonder if we should call it _URL or _URI? We can assume a local path and error with an informative message about the lack of remote file support for now, but we should have a forward-looking name.

We also need to be considerate about how the fetch is implemented, e.g., if we're making a network request it should probably be async which would mean this this function needs to be async (which I would be disappointed by). What kind of change would we need to make to avoid that?

(I don't think we need remote file support to add this feature — but I would like to know how invasive it would be)

A remote file also introduces the complexity of caching 🤔 though perhaps we could avoid that since installs don't happen that often.

looks like it won't be that bad to make it async.
we would also need to make the following async:

iter_downloads

from_request

PytestRequest::new

But maybe we could use tokio::runtime::Handle::current().block_on or construct a new runtime for the request only.

zanieb · 2025-01-24T22:34:52Z

crates/uv-python/src/downloads.rs

+                let json_downloads: HashMap<String, JsonPythonDownload> = serde_json::from_reader(file)
+                    .unwrap_or_else(|e| {
+                        panic!("Failed to parse JSON file at {json_path}: {e}");
+                    });


Maybe a bit of a silly question, but is there a reason we shouldn't deserialize directly into ManagedPythonDownload? Why the intermediate type?

TBH, that's what the AI suggested, but I had the same thought as you, so I tried to derive the Deserialize on ManagedPythonDownload and then on PythonInstallationKey. I understood that target_lexicon's enums do not implement the Deserialize trait, and even if I overcome this, the conversion from JSON will not be as is. So the only way to improve it (the way I see it) is to implement Deserialize on our own and define the JSON structs inside the implementation.

If you see a better way, please let me know

It looks like there's a serde feature for target_lexicon, did you try that?

What part is not "as-is"? Like some custom mapping of architectures or something?

It looks like there's a serde feature for target_lexicon, did you try that?

from the code, I understood that they only implemented it for the Triple struct, so it not useful here :(

What part is not "as-is"? Like some custom mapping of architectures or something?

yes, it's small details, like the implementation that will need special implementation, or the arch. the main problem here is the structs of target_lexicon

zanieb · 2025-01-24T22:36:26Z

crates/uv-python/src/downloads.rs

+
+                let json_downloads: HashMap<String, JsonPythonDownload> = serde_json::from_reader(file)
+                    .unwrap_or_else(|e| {
+                        panic!("Failed to parse JSON file at {json_path}: {e}");


A panic is probably not acceptable here. We'd need to handle this gracefully, i.e., by surfacing the error to the user.

MeitarR · 2025-01-30T18:44:52Z

@zanieb is there something left for me to do to get this merged?

zanieb · 2025-02-03T21:55:32Z

Just need to find the time to review it carefully and make sure we're happy maintaining / supporting this in perpetuity.

Gankra · 2025-02-21T16:19:38Z

Apologies for the delay! I pushed up a tweak to the design to make it remain totally static and cheap when the environment variable isn't set. My main hesitation with this solution now is just that we are maintaining two copies of this "raw json" => "digested rust types" logic: One copy in python, and one copy in Rust.

This would be really easy to accept if this digestion code was shared, but it's a lot of work to do that.

The easiest path forward would be drop the python translation script, directly embed the json statically, and just parse it at runtime (still caching it in a Once so it's a one-time-cost). The only barrier to accepting that solution is someone doing profiling to verify the performance impact is negligible.

MeitarR · 2025-02-22T21:00:10Z

Apologies for the delay! I pushed up a tweak to the design to make it remain totally static and cheap when the environment variable isn't set. My main hesitation with this solution now is just that we are maintaining two copies of this "raw json" => "digested rust types" logic: One copy in python, and one copy in Rust.

This would be really easy to accept if this digestion code was shared, but it's a lot of work to do that.

The easiest path forward would be drop the python translation script, directly embed the json statically, and just parse it at runtime (still caching it in a Once so it's a one-time-cost). The only barrier to accepting that solution is someone doing profiling to verify the performance impact is negligible.

@Gankra, I agree with you.

I implemented the "easiest path" and pushed it to this branch.
I'm concerned about the size that is added to the binary (~~810K~~ 501K~).
and about the implementation, it will probably be better to minify the file on compile time and not save a duplicate. another option is to only save the minified version in the repo, I leave it for you to decide.

About the profiling part, I do not know how to do it (would be happy to learn). is it something that you will take care of?

…URL` and moved it to `crates/uv-static/src/env_vars.rs`

…inal behavior

…keys

Gankra

I think this looks good now! Just one minor improvement but I could take it or leave it.

Gankra · 2025-04-04T14:08:58Z

crates/uv-python/src/downloads.rs

+            let implementation = match entry.name.as_str() {
+                "cpython" => LenientImplementationName::Known(ImplementationName::CPython),
+                "pypy" => LenientImplementationName::Known(ImplementationName::PyPy),
+                _ => LenientImplementationName::Unknown(entry.name.clone()),
+            };


If this was just processing our input I'd be strict to make sure that upstream PBS features get proper support in uv. But when taking user input being lenient is probably fair... maybe this function can take a boolean flag so the two differ?

j178 · 2025-04-04T14:42:20Z

crates/uv-python/src/downloads.rs

@@ -7,8 +9,11 @@ use std::task::{Context, Poll};
 use std::time::{Duration, SystemTime};

 use futures::TryStreamExt;
+use itertools::Itertools;
+use once_cell::sync::OnceCell;


Should we use std::cell::OnceCell instead?

I wanted to, but get_or_try_init is nightly-only

Gankra · 2025-04-07T17:56:07Z

Huge kudos for the extreme diligence on landing this!

(security note: i deleted and regenerated the artifacts locally, produced the exact same result)

…es (#10939) Add an option to overwrite the list of available Python downloads from a local JSON file by using the environment variable `UV_PYTHON_DOWNLOADS_JSON_URL` as an experimental support for providing custom sources for Python distribution binaries #8015 related #10203 I probably should make the JSON to be fetched from a remote URL instead of a local file. please let me know what you think and I will modify the code accordingly. ``` root@75c66494ba8b:/# /code/target/release/uv python list cpython-3.14.0a4+freethreaded-linux-x86_64-gnu <download available> cpython-3.14.0a4-linux-x86_64-gnu <download available> cpython-3.13.1+freethreaded-linux-x86_64-gnu <download available> cpython-3.13.1-linux-x86_64-gnu <download available> cpython-3.12.8-linux-x86_64-gnu <download available> cpython-3.11.11-linux-x86_64-gnu <download available> cpython-3.10.16-linux-x86_64-gnu <download available> cpython-3.9.21-linux-x86_64-gnu <download available> cpython-3.8.20-linux-x86_64-gnu <download available> cpython-3.7.9-linux-x86_64-gnu <download available> pypy-3.10.14-linux-x86_64-gnu <download available> pypy-3.9.19-linux-x86_64-gnu <download available> pypy-3.8.16-linux-x86_64-gnu <download available> pypy-3.7.13-linux-x86_64-gnu <download available> ``` ```sh root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL {} root@75c66494ba8b:/# /code/target/release/uv python list root@75c66494ba8b:/# ``` ```sh root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=/code/crates/uv-python/my-download-metadata.json root@75c66494ba8b:/# cat $UV_PYTHON_DOWNLOADS_JSON_URL { "cpython-3.11.9-linux-x86_64-gnu": { "name": "cpython", "arch": { "family": "x86_64", "variant": null }, "os": "linux", "libc": "gnu", "major": 3, "minor": 11, "patch": 9, "prerelease": "", "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20240814/cpython-3.11.9%2B20240814-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", "sha256": "daa487c7e73005c4426ac393273117cf0e2dc4ab9b2eeda366e04cd00eea00c9", "variant": null } } root@75c66494ba8b:/# /code/target/release/uv python list cpython-3.11.9-linux-x86_64-gnu <download available> root@75c66494ba8b:/# ``` ```sh root@75c66494ba8b:/# export UV_PYTHON_DOWNLOADS_JSON_URL=http://a.com/file.json root@75c66494ba8b:/# /code/target/release/uv python list error: Remote python downloads JSON is not yet supported, please use a local path (without `file://` prefix) ``` --------- Co-authored-by: Aria Desires <[email protected]>

## 0.6.14 ### Python versions The following Python versions have been added: - CPython 3.13.3 - CPython 3.12.10 - CPython 3.11.12 - CPython 3.10.17 - CPython 3.9.22 See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250409) for more details. ### Enhancements - Add `uv-build` and `uv_build` aliases to `uv init --build-backend` ([#12776](astral-sh/uv#12776)) - Emit dedicated error message for Conda `environment.yml` files ([#12669](astral-sh/uv#12669)) ### Preview features - Build backend: Check module dir exists for sdist build ([#12779](astral-sh/uv#12779)) - Build backend: Fix sdist with long directories ([#12764](astral-sh/uv#12764)) ### Performance - Avoid querying GitHub on repeated install invocations ([#12767](astral-sh/uv#12767)) ### Bug fixes - Error when `tool.uv.sources` is set in system-level configuration file ([#12757](astral-sh/uv#12757)) - Split workspace members onto their own lines in `uv init` ([#12756](astral-sh/uv#12756)) ### Documentation - Add lockfile note about PEP 751 ([#12732](astral-sh/uv#12732)) - Extend the reference documentation for `uv pip sync` ([#12683](astral-sh/uv#12683)) - Fix mismatched pip interface header / nav titles ([#12640](astral-sh/uv#12640)) ## 0.6.13 ### Enhancements - Add `--show-version` to `uv python find` ([#12376](astral-sh/uv#12376)) - Remove `--no-config` warning from `uv pip compile` and `uv pip sync` ([#12642](astral-sh/uv#12642)) - Skip repeated directories in `PATH` when searching for Python interpreters ([#12367](astral-sh/uv#12367)) - Unset `SCRIPT_PATH` in relocatable activation script ([#12672](astral-sh/uv#12672)) - Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources ([#10939](astral-sh/uv#10939)) - Reject `pyproject.toml` files in `uv pip compile -o` ([#12673](astral-sh/uv#12673)) - Respect the `--offline` flag for Git operations ([#12619](astral-sh/uv#12619)) ### Bug fixes - Warn instead of error if CRC appears to be missing ([#12722](astral-sh/uv#12722)) - Avoid infinite loop in `uv export` with conflicts ([#12726](astral-sh/uv#12726)) ### Rust API - Update MSRV to 1.84 ([#12670](astral-sh/uv#12670)) ## 0.6.12 ### Enhancements - Report the queried executable path in `uv python list` ([#12628](astral-sh/uv#12628)) - Improve archive unpack error messages ([#12627](astral-sh/uv#12627)) ### Bug fixes - Respect `authenticate` when using `explicit = true` ([#12631](astral-sh/uv#12631)) - Normalize extra and group names in `uv add` and `uv remove` ([#12586](astral-sh/uv#12586)) - Enforce CRC-32 checks when unpacking archives ([#12623](astral-sh/uv#12623)) - Fix parsing of `python-platform` in settings files ([#12592](astral-sh/uv#12592)) ### Documentation - Add note about `uv build` to `package = false` ([#12608](astral-sh/uv#12608)) - Add index fallback note to `authenticate = always` documentation ([#12498](astral-sh/uv#12498)) - Fix invalid 'kind' reference in flat index docs ([#12583](astral-sh/uv#12583)) ## 0.6.11 ### Enhancements - Add dependents ("via ..." comments) in `uv export` command ([#12350](astral-sh/uv#12350)) - Bump least-recent non-EOL macOS version to 13.0 ([#12518](astral-sh/uv#12518)) - Support `--find-links`-style "flat" indexes in `[[tool.uv.index]]` ([#12407](astral-sh/uv#12407)) - Distinguish between `-q` and `-qq` ([#12300](astral-sh/uv#12300)) ### Configuration - Support `UV_PROJECT` environment to set project directory. ([#12327](astral-sh/uv#12327)) ### Performance - Use a boxed slice for various requirement types ([#12514](astral-sh/uv#12514)) ### Bug fixes - Add a newline after metadata when initializing scripts with other metadata blocks ([#12501](astral-sh/uv#12501)) - Avoid writing empty `requires-python` to script blocks ([#12517](astral-sh/uv#12517)) - Respect build constraints in `uv sync` ([#12502](astral-sh/uv#12502)) - Respect transitive dependencies in `uv tree --only-group` ([#12560](astral-sh/uv#12560)) ## 0.6.10 ### Enhancements - Add `uv sync --check` flag ([#12342](astral-sh/uv#12342)) - Add support for Python version requests in `uv python list` ([#12375](astral-sh/uv#12375)) - Support `.env` files in `uv tool run` ([#12386](astral-sh/uv#12386)) - Support `python find --script` ([#11891](astral-sh/uv#11891)) ### Preview features - Check all compatible torch indexes when `--torch-backend` is enabled ([#12385](astral-sh/uv#12385)) ### Performance - Use a boxed slice for extras and groups ([#12391](astral-sh/uv#12391)) - Use small string for index name type ([#12355](astral-sh/uv#12355)) ### Bug fixes - Allow virtual packages with `--no-build` ([#12314](astral-sh/uv#12314)) - Ignore `--find-links` entries for pinned indexes ([#12396](astral-sh/uv#12396)) - Omit wheels from lockfile based on `--exclude-newer` ([#12299](astral-sh/uv#12299)) - Retain end-of-line comment position when adding dependency ([#12360](astral-sh/uv#12360)) - Omit fragment when querying for wheels in Simple HTML API ([#12384](astral-sh/uv#12384)) - Error on missing argument in `requirements.txt` ([#12354](astral-sh/uv#12354)) - Support modules with different casing in build backend ([#12240](astral-sh/uv#12240)) - Add authentication policy support for `pip` commands ([#12470](astral-sh/uv#12470)) ## 0.6.9 ### Enhancements - Use `keyring --mode creds` when `authenticate = "always"` ([#12316](astral-sh/uv#12316)) - Fail with specific error message when no password is present and `authenticate = "always"` ([#12313](astral-sh/uv#12313)) ### Bug fixes - Add boolish value parser for `UV_MANAGED_PYTHON` flags ([#12345](astral-sh/uv#12345)) - Make deserialization non-fatal when assessing source tree revisions ([#12319](astral-sh/uv#12319)) - Use resolver-returned wheel over alternate cached wheel ([#12301](astral-sh/uv#12301)) ### Documentation - Add experimental `--torch-backend` to the PyTorch guide ([#12317](astral-sh/uv#12317)) - Fix `#keyring-provider` references in alternative index docs ([#12315](astral-sh/uv#12315)) - Fix `--directory` path in examples ([#12165](astral-sh/uv#12165)) ### Preview changes - Automatically infer the PyTorch index via `--torch-backend=auto` ([#12070](astral-sh/uv#12070)) ## 0.6.8 ### Enhancements - Add support for enabling all groups by default with `default-groups = "all"` ([#12289](astral-sh/uv#12289)) - Add simpler `--managed-python` and `--no-managed-python` flags for toggling Python preferences ([#12246](astral-sh/uv#12246)) ### Performance - Avoid allocations for default cache keys ([#12063](astral-sh/uv#12063)) ### Bug fixes - Allow local version mismatches when validating lockfile ([#12285](astral-sh/uv#12285)) - Allow owned string when deserializing `requires-python` ([#12278](astral-sh/uv#12278)) - Make cache errors non-fatal in `Planner::build` ([#12281](astral-sh/uv#12281)) ## 0.6.7 ### Python - Add CPython 3.14.0a6 - Fix regression where extension modules would use wrong `CXX` compiler on Linux - Enable FTS3 enhanced query syntax for SQLite See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250317) for more details. ### Enhancements - Add support for `-c` constraints in `uv add` ([#12209](astral-sh/uv#12209)) - Add support for `--global` default version in `uv python pin` ([#12115](astral-sh/uv#12115)) - Always reinstall local source trees passed to `uv pip install` ([#12176](astral-sh/uv#12176)) - Render token claims on publish permission error ([#12135](astral-sh/uv#12135)) - Add pip-compatible `--group` flag to `uv pip install` and `uv pip compile` ([#11686](astral-sh/uv#11686)) ### Preview features - Avoid creating duplicate directory entries in built wheels ([#12206](astral-sh/uv#12206)) - Allow overriding module names for editable builds ([#12137](astral-sh/uv#12137)) ### Performance - Avoid replicating core-metadata field on `File` struct ([#12159](astral-sh/uv#12159)) ### Bug fixes - Add `src` to default cache keys ([#12062](astral-sh/uv#12062)) - Discard insufficient fork markers ([#10682](astral-sh/uv#10682)) - Ensure `python pin --global` creates parent directories if missing ([#12180](astral-sh/uv#12180)) - Fix GraalPy abi tag parsing and discovery ([#12154](astral-sh/uv#12154)) - Remove extraneous script packages in `uv sync --script` ([#12158](astral-sh/uv#12158)) - Remove redundant `activate.bat` output ([#12160](astral-sh/uv#12160)) - Avoid subsequent index hint when no versions are available on the first index ([#9332](astral-sh/uv#9332)) - Error on lockfiles with incoherent wheel versions ([#12235](astral-sh/uv#12235)) ### Rust API - Update `BaseClientBuild` to accept custom proxies ([#12232](astral-sh/uv#12232)) ### Documentation - Make testpypi index explicit in example snippet ([#12148](astral-sh/uv#12148)) - Reverse and format the archived changelogs ([#12099](astral-sh/uv#12099)) - Use consistent commas around i.e. and e.g. ([#12157](astral-sh/uv#12157)) - Fix typos in MRE docs ([#12198](astral-sh/uv#12198)) - Fix double space typo ([#12171](astral-sh/uv#12171))

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.6.11` -> `0.6.16` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.6.16`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0616) [Compare Source](astral-sh/uv@0.6.15...0.6.16) ##### Bug fixes - Revert "Properly handle authentication for 302 redirect URLs" ([#13041](astral-sh/uv#13041)) ### [`v0.6.15`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0615) [Compare Source](astral-sh/uv@0.6.14...0.6.15) This release includes preliminary support for the `pylock.toml` file format, as standardized in [PEP 751](https://peps.python.org/pep-0751/). `pylock.toml` is an alternative resolution output format intended to replace `requirements.txt` (e.g., in the context of `uv pip compile`, whereby a "locked" `requirements.txt` file is generated from a set of input requirements). `pylock.toml` is standardized and tool-agnostic, such that in the future, `pylock.toml` files generated by uv could be installed by other tools, and vice versa. As of this release, `pylock.toml` is supported in the following commands: - To export a `uv.lock` to the `pylock.toml` format, run: `uv export -o pylock.toml` - To generate a `pylock.toml` file from a set of requirements, run: `uv pip compile -o pylock.toml -r requirements.in` - To install from a `pylock.toml` file, run: `uv pip sync pylock.toml` or `uv pip install -r pylock.toml` ##### Enhancements - Add PEP 751 support to `uv pip compile` ([#13019](astral-sh/uv#13019)) - Add `uv export` support for PEP 751 ([#12955](astral-sh/uv#12955)) - Accept `requirements.txt` (verbatim) as a format on the CLI ([#12957](astral-sh/uv#12957)) - Add `UV_NO_EDITABLE` environment variable to set `--no-editable` on all invocations ([#12773](astral-sh/uv#12773)) - Add `pylock.toml` to `uv pip install` and `uv pip sync` ([#12992](astral-sh/uv#12992)) - Add a brief sleep before sending `SIGINT` to child processes ([#13018](astral-sh/uv#13018)) - Add upload time to `uv.lock` ([#12968](astral-sh/uv#12968)) - Allow updating Git sources by name ([#12897](astral-sh/uv#12897)) - Cache `which git` in `uv init` ([#12893](astral-sh/uv#12893)) - Enable `--dry-run` with `--locked` / `--frozen` for `uv sync` ([#12778](astral-sh/uv#12778)) - Infer output type in `uv export` ([#12958](astral-sh/uv#12958)) - Make `uv init` resilient against broken git ([#12895](astral-sh/uv#12895)) - Respect build constraints for `uv run --with` dependencies ([#12882](astral-sh/uv#12882)) - Split UV_INDEX on all whitespace ([#12820](astral-sh/uv#12820)) - Support build constraints in `uv tool` and PEP723 scripts. ([#12842](astral-sh/uv#12842)) - Use suffix from `uvx` binary when searching for uv binary ([#12923](astral-sh/uv#12923)) - Update version formatting to use cyan color ([#12943](astral-sh/uv#12943)) - Add debug logs for version file search ([#12951](astral-sh/uv#12951)) - Fix `SourceNotAllowed` error message during Python discovery ([#13012](astral-sh/uv#13012)) - Obfuscate password in credentials debug messages ([#12944](astral-sh/uv#12944)) - Obfuscate possible tokens in URL logs ([#12969](astral-sh/uv#12969)) - Validate that PEP 751 entries don't include multiple sources ([#12993](astral-sh/uv#12993)) ##### Preview features - Build backend: Add reference docs and schema ([#12803](astral-sh/uv#12803)) ##### Bug fixes - Align supported `config-settings` with example in docs ([#12947](astral-sh/uv#12947)) - Ensure virtual environment is compatible with interpreter on sync ([#12884](astral-sh/uv#12884)) - Fix `PythonDownloadRequest` parsing for partial keys ([#12925](astral-sh/uv#12925)) - Fix pre-release exclusive comparison operator in `uv-pep440` ([#12836](astral-sh/uv#12836)) - Forward additional signals to the child process in `uv run` ([#13017](astral-sh/uv#13017)) - Omit PEP 751 version for source trees ([#13030](astral-sh/uv#13030)) - Patch `CC` and `CCX` entries in sysconfig for cross-compiled `aarch64` Python distributions ([#12239](astral-sh/uv#12239)) - Properly handle authentication for HTTP 302 redirect URLs ([#12920](astral-sh/uv#12920)) - Set 4MB stack size for all threads, introduce `UV_STACK_SIZE` ([#12839](astral-sh/uv#12839)) - Show PyPy downloads during `uv python list` ([#12915](astral-sh/uv#12915)) - Add `subdirectory` to Direct URL for local directories ([#12971](astral-sh/uv#12971)) - Prefer stable releases over pre-releases in `uv python install` ([#12194](astral-sh/uv#12194)) - Write requested Python variant to pin file in `uv init` ([#12870](astral-sh/uv#12870)) ##### Documentation - Fix CLI reference with code block ([#12807](astral-sh/uv#12807)) - Fix lockfile note ([#12793](astral-sh/uv#12793)) - Fix typo in a reference ([#12858](astral-sh/uv#12858)) - Improve docs for `uv python list --only-downloads` and `--only-installed` ([#12916](astral-sh/uv#12916)) - Update note on lack of musl distributions to ARM-only ([#12825](astral-sh/uv#12825)) - Add section on shebangs for scripts ([#11553](astral-sh/uv#11553)) - Display aliases for long and short args in the CLI reference ([#12824](astral-sh/uv#12824)) - Fix highlight line in explicit index documentation ([#12887](astral-sh/uv#12887)) - Add explicit source (matching PyTorch guide) ([#12844](astral-sh/uv#12844)) - Fix link to issue ([#12823](astral-sh/uv#12823)) - Fix grammatical error in FastAPI guide ([#12908](astral-sh/uv#12908)) - Add `--locked` to `uv sync` in GitHub Actions guide ([#12819](astral-sh/uv#12819)) - Improve formatting for `"all"` `default-groups` setting documentation ([#12963](astral-sh/uv#12963)) - Replace `--frozen` with `--locked` in Docker integration guide ([#12818](astral-sh/uv#12818)) ### [`v0.6.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0614) [Compare Source](astral-sh/uv@0.6.13...0.6.14) ##### Python versions The following Python versions have been added: - CPython 3.13.3 - CPython 3.12.10 - CPython 3.11.12 - CPython 3.10.17 - CPython 3.9.22 See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250409) for more details. ##### Enhancements - Add `uv-build` and `uv_build` aliases to `uv init --build-backend` ([#12776](astral-sh/uv#12776)) - Emit dedicated error message for Conda `environment.yml` files ([#12669](astral-sh/uv#12669)) ##### Preview features - Build backend: Check module dir exists for sdist build ([#12779](astral-sh/uv#12779)) - Build backend: Fix sdist with long directories ([#12764](astral-sh/uv#12764)) ##### Performance - Avoid querying GitHub on repeated install invocations ([#12767](astral-sh/uv#12767)) ##### Bug fixes - Error when `tool.uv.sources` is set in system-level configuration file ([#12757](astral-sh/uv#12757)) - Split workspace members onto their own lines in `uv init` ([#12756](astral-sh/uv#12756)) ##### Documentation - Add lockfile note about PEP 751 ([#12732](astral-sh/uv#12732)) - Extend the reference documentation for `uv pip sync` ([#12683](astral-sh/uv#12683)) - Fix mismatched pip interface header / nav titles ([#12640](astral-sh/uv#12640)) ### [`v0.6.13`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0613) [Compare Source](astral-sh/uv@0.6.12...0.6.13) ##### Enhancements - Add `--show-version` to `uv python find` ([#12376](astral-sh/uv#12376)) - Remove `--no-config` warning from `uv pip compile` and `uv pip sync` ([#12642](astral-sh/uv#12642)) - Skip repeated directories in `PATH` when searching for Python interpreters ([#12367](astral-sh/uv#12367)) - Unset `SCRIPT_PATH` in relocatable activation script ([#12672](astral-sh/uv#12672)) - Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources ([#10939](astral-sh/uv#10939)) - Reject `pyproject.toml` files in `uv pip compile -o` ([#12673](astral-sh/uv#12673)) - Respect the `--offline` flag for Git operations ([#12619](astral-sh/uv#12619)) ##### Bug fixes - Warn instead of error if CRC appears to be missing ([#12722](astral-sh/uv#12722)) - Avoid infinite loop in `uv export` with conflicts ([#12726](astral-sh/uv#12726)) ##### Rust API - Update MSRV to 1.84 ([#12670](astral-sh/uv#12670)) ### [`v0.6.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0612) [Compare Source](astral-sh/uv@0.6.11...0.6.12) ##### Enhancements - Report the queried executable path in `uv python list` ([#12628](astral-sh/uv#12628)) - Improve archive unpack error messages ([#12627](astral-sh/uv#12627)) ##### Bug fixes - Respect `authenticate` when using `explicit = true` ([#12631](astral-sh/uv#12631)) - Normalize extra and group names in `uv add` and `uv remove` ([#12586](astral-sh/uv#12586)) - Enforce CRC-32 checks when unpacking archives ([#12623](astral-sh/uv#12623)) - Fix parsing of `python-platform` in settings files ([#12592](astral-sh/uv#12592)) ##### Documentation - Add note about `uv build` to `package = false` ([#12608](astral-sh/uv#12608)) - Add index fallback note to `authenticate = always` documentation ([#12498](astral-sh/uv#12498)) - Fix invalid 'kind' reference in flat index docs ([#12583](astral-sh/uv#12583)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

## Summary In #10939 I added the generated `crates/uv-python/src/download-metadata-minified.json` file which is a minified version of `crates/uv-python/download-metadata.json`. The main reason for this PR is to avoid bloating the git objects as this is a single-line file. As a bonus, I also filtered the embed json to include only the versions for the compiled target. Which should improve the binary size and performance by a bit. ## Test Plan

geofft · 2025-07-17T19:22:17Z

I'm adding remote URL support in #14687, for the use case of making rollbacks easier if we break something in python-build-standalone, but likely also useful for maintaining your own mirror of Python builds.

zanieb self-assigned this Jan 24, 2025

zanieb reviewed Jan 24, 2025

View reviewed changes

MeitarR requested a review from zanieb January 25, 2025 21:46

MeitarR force-pushed the uv-python-downloads-json-file branch from 76c6528 to 1af245c Compare January 25, 2025 21:48

MeitarR changed the title ~~custom sources for Python distribution binaries~~ Add UV_PYTHON_DOWNLOADS_JSON_URL to set custom managed python sources Jan 27, 2025

MeitarR force-pushed the uv-python-downloads-json-file branch 2 times, most recently from dedc024 to 93e045c Compare January 30, 2025 19:27

Gankra force-pushed the uv-python-downloads-json-file branch from 93e045c to 1e7b756 Compare February 21, 2025 16:09

Gankra force-pushed the uv-python-downloads-json-file branch from b04d4e2 to c1d4e8a Compare February 21, 2025 16:20

MeitarR force-pushed the uv-python-downloads-json-file branch from 333f594 to 35df49b Compare February 22, 2025 20:49

MeitarR force-pushed the uv-python-downloads-json-file branch from 35df49b to 96aa962 Compare February 28, 2025 21:40

MeitarR marked this pull request as draft February 28, 2025 21:55

MeitarR marked this pull request as ready for review February 28, 2025 23:22

MeitarR force-pushed the uv-python-downloads-json-file branch 3 times, most recently from d0a8e1d to 1c79634 Compare March 4, 2025 21:09

MeitarR temporarily deployed to release March 4, 2025 21:09 — with GitHub Actions Inactive

MeitarR force-pushed the uv-python-downloads-json-file branch from 1c79634 to e9a2f63 Compare March 7, 2025 14:44

MeitarR temporarily deployed to release March 7, 2025 14:44 — with GitHub Actions Inactive

MeitarR force-pushed the uv-python-downloads-json-file branch from e9a2f63 to 94178a6 Compare March 16, 2025 21:47

MeitarR temporarily deployed to release March 16, 2025 21:47 — with GitHub Actions Inactive

MeitarR and others added 12 commits April 3, 2025 22:45

Refactor Python downloads handling to support JSON configuration

5fcd0be

changed UV_PYTHON_DOWNLOADS_JSON_PATH to `UV_PYTHON_DOWNLOADS_JSON_…

8fdfb87

…URL` and moved it to `crates/uv-static/src/env_vars.rs`

surface the errors to the user

277bc5d

avoid needless clones

3413156

embed minified download metadata json inside the binary

624b95f

skip debug versions in the minified json

0b76206

sort PYTHON_DOWNLOADS by version key to hopefully preserve the orig…

5ca192e

…inal behavior

reverse variant sorting to prioritize Default in Python installation …

435a627

…keys

refactor: extract JSON downloads parsing into a separate function

9132949

really skip entry if variant is unknown

eee5931

bring back arch logic from script

56b5eaa

add context for SerdeJson errors

60e6cc7

MeitarR force-pushed the uv-python-downloads-json-file branch from a5f1bae to 60e6cc7 Compare April 3, 2025 19:52

MeitarR temporarily deployed to release April 3, 2025 19:52 — with GitHub Actions Inactive

Gankra approved these changes Apr 4, 2025

View reviewed changes

j178 reviewed Apr 4, 2025

View reviewed changes

Gankra merged commit 2b62f73 into astral-sh:main Apr 7, 2025
99 checks passed

BrewTestBot mentioned this pull request Apr 7, 2025

uv 0.6.13 Homebrew/homebrew-core#218704

Merged

MeitarR mentioned this pull request Apr 11, 2025

Make it easier to deploy a mirror for uv-python command #10203

Closed

MeitarR deleted the uv-python-downloads-json-file branch April 11, 2025 13:45

This was referenced Apr 11, 2025

improvements for creating and using offline mirror for uv-python #12838

Open

minify pythons json on compile time #12967

Merged

zanieb mentioned this pull request May 10, 2025

uv-python: panic when installing Python 3.12 on Windows (called \Option::unwrap() on a None value in downloads.rs) #13376

Closed

Add UV_PYTHON_DOWNLOADS_JSON_URL to set custom managed python sources #10939

Add UV_PYTHON_DOWNLOADS_JSON_URL to set custom managed python sources #10939

Uh oh!

Conversation

MeitarR commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

normal run

empty JSON file

JSON file with valid version

Remote Path

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MeitarR Jan 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanieb Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MeitarR commented Jan 30, 2025

Uh oh!

zanieb commented Feb 3, 2025

Uh oh!

Gankra commented Feb 21, 2025

Uh oh!

MeitarR commented Feb 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gankra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Gankra commented Apr 7, 2025

Uh oh!

geofft commented Jul 17, 2025

Uh oh!

Uh oh!

Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources #10939

Add `UV_PYTHON_DOWNLOADS_JSON_URL` to set custom managed python sources #10939

MeitarR commented Jan 24, 2025 •

edited

Loading

MeitarR Jan 25, 2025 •

edited

Loading

zanieb Jan 24, 2025 •

edited

Loading

MeitarR commented Feb 22, 2025 •

edited

Loading