Add support for PEP 751 lockfiles

[charliermarsh]

PEP 751 is officially accepted.

Today, the PEP 751-style pylock.toml files are not sufficient to replace uv.lock. The biggest limitation is that there's no support for arbitrary entrypoints to the graph, because pylock.toml includes a fixed marker for each package entry rather than recording a graph of dependencies. Specifically, this means uv run -p could not be supported (i.e., we couldn't support installing a specific subset of the graph) within the standard.

Notably, though, unlike requirements.txt, pylock.toml also supports extras and dependency groups by extending the PEP 508 marker syntax. So markers can now express things like "this extra was enabled" or "this dependency group was enabled". As such, we could potentially use PEP 751-style pylock.toml files for single-member workspaces. I don't think this is worth the complexity right now, but it is possible.

So for now, from uv's perspective, we can think of PEP 751-style pylock.toml files as a more modern requirements.txt. A pylock.toml file captures a set of packages to install, along with the source of those packages. Unlike requirements.txt, the format is fully standardized, and it can also model certain pieces of information that requirements.txt cannot (e.g., you can specify a package as coming from a specific registry).

As a first step, we want to support PEP 751-style pylock.toml files as an export format in uv export, and as an installable format in uv pip install. Like uv pip install --group, we may want to see how pip chooses to support these files before implementing uv pip install support. This requires:

• Adding code to translate uv.lock to pylock.toml in uv export.
• Supporting pylock.toml as a dependency source for uv pip install, or maybe just uv pip sync?
• Extending our PEP 508 implementation to understand the new extras and dependency-groups markers.

[Gankra]

• PEP 751 – A file format to record Python dependencies for installation reproducibility
  • Additions to Marker Expression Syntax
• PEP 508 - Dependency specification for Python Software Packages

[BenediktMaag]

Since this PEP contains packages.index may i propose to add functionality for uv add (--script) for the lockfile to solve #11053

[charliermarsh]

Yes, exporting to PEP 751 would solve that issue.

[gaborbernat]

The biggest limitation is that there's no support for arbitrary entrypoints to the graph, because pylock.toml includes a fixed marker for each package entry rather than recording a graph of dependencies.

Any insight why this limitation was deemed out of the scope for that PEP?

[charliermarsh]

Yeah, it was discussed at length but it was ultimately decided that it was adding too much complexity for a first draft / version of the spec. uv is also the only tool that supports this IIUC which made it hard to motivate standardizing around it right now.

[gaborbernat]

Is there an effort to work on a version 2 that would add support for it?😅

Do you have some plans on how to select when both lock files are present or what would UV lock generate by default? In the long term obviously, as this ticket says that we'll only support it via export.

[charliermarsh]

There isn't yet, but there should be eventually. It was made fairly clear in the DPO discussion that if there is a Version 2 with support for these kinds of features, the expectation is that the PEP would need to be authored and driven by tooling authors (uv, Poetry, PDM, etc.).

[polothy]

Supporting pylock.toml as a dependency source for uv pip install, or maybe just uv pip sync?

Please support uv pip install to support the Lambda zip bundling workflow. Thanks!

[benwebber]

We use uv pip compile to compile requirements.txt files instead of using uv lock files. Will it be possible to compile pylock.toml files directly from pyproject.toml, without generating a uv lock file and exporting it?

[charliermarsh]

Yeah I think so. I expect uv pip compile and uv export to both support it as an output format.

[charliermarsh]

The initial uv export support is here: #12955

[charliermarsh]

uv export, uv pip compile, uv pip sync, and uv pip install will support pylock.toml in the next release:

• Add uv export support for PEP 751 #12955
• Add pylock.toml to uv pip install and uv pip sync #12992
• Add PEP 751 support to uv pip compile #13019