Skip to content

Allow an empty metadata cache when adding lower bounds - #21225

Merged
konstin merged 1 commit into
mainfrom
konsti/metadata-free-add-cache-followup
Sep 29, 2026
Merged

konstin merged 1 commit into
mainfrom
konsti/metadata-free-add-cache-followup

Conversation

@konstin

@konstin konstin commented Aug 19, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #21163.

Reusing a metadata-free lock can leave the in-memory distribution index empty. When uv add inserts a lower bound and locks again, invalidating the project's metadata must tolerate an entry that was never cached.

Remove the cache-presence assertion and extend the existing lower-bound regression test to cover adding a redundant marker after reusing a metadata-free lock.

@konstin
konstin force-pushed the konsti/metadata-free-lock-systematic branch 2 times, most recently from 61458ff to fb8e795 Compare August 21, 2026 11:10
@konstin
konstin force-pushed the konsti/metadata-free-add-cache-followup branch from 9bb4385 to 2b3fa70 Compare September 8, 2026 14:16
@konstin
konstin force-pushed the konsti/metadata-free-lock-systematic branch from fb8e795 to 3583307 Compare September 8, 2026 15:52
@charliermarsh
charliermarsh force-pushed the konsti/metadata-free-lock-systematic branch from a08f292 to a9acaab Compare September 15, 2026 10:52
charliermarsh added a commit that referenced this pull request Sep 15, 2026
The previous iterations of `package.metadata`-free lockfiles had missed
a number of cases, and made it hard to ensure that all cases were
actually covered. This PR takes a new approach: Remove the previous
implementation, and then re-add it feature-by-feature in reviewable
commit units. Each commit adds one feature that uv supports.

The new approach is not entirely `package.metadata`-free, instead remote
URL, which can activate other URL source, now have `package.metadata`
which they previously didn't. That's required for offline validation
(which is crucial for `uv lock --check` performance, offline or not) and
IMO the more correct tradeoff: We shouldn't repeat information that's
already available on disk, but we should state what we expect from
remote resources.

`uv.lock` revision 1.4 additionally records empty extras so `--frozen
--extra foo` if the lock wasn't generated with extra `foo` present, but
it was added to `pyproject.toml` afterwards.

This is a lockfile size regression if you're using a lot of remote URLs,
otherwise it's still a much smaller lockfile for large workspaces.
Remote source are usually used sparingly as they are worse for
performance, reliability and auditability than registry sources.

## Validation

This change reflects basically every resolver assumption and form of
dependency specification and also hits a number of preexisting bugs (see
upstack PRs and skipped changes list). Instead of trying to reason
through correctness from the features we're representing, this PR was
tested (and fixed) against large, diverse validation set, checking that
uv's invariants are upheld with lockfile revision 1.4. The PR is
additionally split into one commit per feature the new freshness check
support, to ease reviewing and ensure that all features are covered. The
most import check that needs to always pass is `uv lock --check
--offline --no-cache` on a fresh lockfile. We don't need to check
features such as `uv sync`, as `package.metadata` is only used for fast
path freshness checks, and `uv lock --check --offline --no-cache`
exercises the offline fast path while blocking the slow path if it would
use the internet (which is unacceptable both for performance and for
checks that run in isolated environments). The goal is to cover the
available dependency specification that people use, while not fixing
unrelated bugs in this already large change. There are three of these
larger validations.

The first one is a script finds public GitHub projects with editable
packages, workspaces, Git dependencies or dynamically generated
dependencies and then verifies that both the original lockfile and a
version with package metadata removed pass `uv lock --check --offline
--no-cache` inside an isolated docker container. This dataset contained
254 projects with 14k locked packages.

The second validation uses the check-mode in
https://github.com/astral-sh/uv-ecosystem-testing with
`UV_PREVIEW_FEATURES=lock-without-metadata` .

The third one is running the test suite migrated to lockfile revision
1.4 with
4b98802,
passing 4.8k tests:

```
features=test-python-patch,test-universal,native-auth
cargo insta test --accept --test-runner nextest --workspace --features "$features"
cargo nextest run --workspace --features "$features"
```

The PR handles the test cases and fixes from
#20803 too.

The area with the least coverage wrt to its complexity is conflicts
supports. While the support is implemented, I'd consider its current
support much weaker than the rest of the features.

## Skipped changes

The following independent bugfixes are deliberately excluded from this
stack:

- #21210.
- #21225. The cache assertion remains a TODO here.

The 4.8k-test run above included the add-cache fix and predates this
extraction; it is not a fresh full-suite run of the reduced stack.

---------

Co-authored-by: Charlie Marsh <charlie.r.marsh@gmail.com>
Base automatically changed from konsti/metadata-free-lock-systematic to main September 15, 2026 17:58
@konstin
konstin force-pushed the konsti/metadata-free-add-cache-followup branch from 2b3fa70 to 660c631 Compare September 22, 2026 10:54
Reusing a metadata-free lock can leave the in-memory distribution index empty. Invalidate the project metadata if present instead of asserting that it was cached, and cover adding a redundant marker after reusing the lock.
@konstin
konstin force-pushed the konsti/metadata-free-add-cache-followup branch from 660c631 to 34d8781 Compare September 23, 2026 13:50
@konstin
konstin marked this pull request as ready for review September 23, 2026 13:51
@konstin konstin added the bug Something isn't working label Sep 25, 2026
@konstin
konstin merged commit 6cf775b into main Sep 29, 2026
63 checks passed
@konstin
konstin deleted the konsti/metadata-free-add-cache-followup branch September 29, 2026 13:29

This branch was previously deployed

1 inactive deployment
automations — 34d87817 Deployed Sep 23, 2026 by konstin via review / security review #47460
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants