Conflicting groups should handle conflicting inclusions automatically #12005

jtfmumm · 2025-03-06T14:00:44Z

This adds support for inferring dependency group conflict sets from the directly defined conflicts in configuration. For example, if you declare a conflict between groups alpha and beta and dev includes beta, then we will infer a conflict between dev and alpha. We will also handle a conflict between two groups if they transitively include groups that conflict with each other. See #11232 for more details.

Closes #11232

jtfmumm · 2025-03-06T14:06:07Z

crates/uv-pypi-types/src/dependency_groups.rs

I've moved this file so that I can import DependencyGroups in conflicts.rs. It would have caused a circular dependency in uv-workspace.

jtfmumm · 2025-03-06T14:07:06Z

crates/uv-pypi-types/src/conflicts.rs

+        }
+
+        // Propagate canonical items through the graph and populate substitutions.
+        // FIXME: Have we already done cycle detection before this method was


Can we assume cycle detection before this point? Otherwise, we'll have to return an error

There's a failing test that wasn't running locally for some reason that shows we can't assume it at this point. I'll update.

What I've done now is to just return if there's a Cycle error on toposort. I've left a FIXME comment indicating that we're currently bailing and allowing more detailed include cycle detection downstream. Is this the behavior we want? If so, I'll remove the FIXME tag.

Hmmm cc @zanieb

One thing that strikes me here is that the cycle detection error would only kick in when a relevant conflicting group is declared? What happens if there aren't any conflicts declared? Is there cycle detection happening somewhere else?

Cycle detection is happening somewhere, maybe in the resolver? But that happens after this code is called. With this new version where I just return (no error) when toposort() returns an error, then only the original conflict sets are sent along.

But it does seem weird to silently swallow the error. The reason I don't want it propagated is because better cycle detection happens elsewhere ("there was a cycle" vs. "there was a cycle: dev -> test -> dev"). So I could either warn the user in this method or match at the caller and warn there. Do either of those options make sense?

I think what you're doing now is probably reasonable with a comment explaining what's going on here (basically, what you just said). @zanieb might have a different opinion though!

I don't know what our current behavior for cycles is. Are we talking about cycles in inclusions or cycles in the conflicts?

If dev includes test and test includes dev don't we have a consistent set of dependencies? (just the union of test and dev?)

Similarly, if test is marked as conflicting with some other group foo then dev conflicts with foo too, the cycle doesn't matter here.

If test is marked as conflicting with dev, now the cycle is a problem because... they include each other and can't conflict. This applies to longer chains too, e.g., if we have test -> foo -> dev -> test a conflict between test and dev is "invalid".

If we don't have test coverage for each of these cases, we should add it. In reply to

One thing that strikes me here is that the cycle detection error would only kick in when a relevant conflicting group is declared?

This seems fine, because this form of cycle (3) is the only one that creates an unresolvable situation?

Unless I'm misunderstanding, cycles in dependency groups are an error. Per the PEP:

Dependency Group Includes MUST NOT include cycles, and tools SHOULD report an error if they detect a cycle.

I think we have an DependencyGroupCycle error.

Yeah, on main we definitely error on cycles. That's the behavior I'm relying on when just returning from the conflict expansion method when detecting a cycle during toposort.

Oh, thanks for the correction. Maybe I was just thinking about feedback I wrote on the PEP.

BurntSushi

Nice work!

The failing duplicate_torch_and_sympy_because_of_wrong_inferences is somewhat worrisome, because that doesn't use dependency groups at all. So in theory, it shouldn't be impacted by the work here?

In your code, I wonder if something wonky is happening when no DependencyGroupSpecifier::IncludeGroup are found and thus no edges are added?

crates/uv-pypi-types/Cargo.toml

crates/uv-pypi-types/src/conflicts.rs

crates/uv/tests/it/sync.rs

crates/uv-pypi-types/src/conflicts.rs

BurntSushi · 2025-03-06T15:52:30Z

crates/uv-pypi-types/src/conflicts.rs

+        }
+
+        // Propagate canonical items through the graph and populate substitutions.
+        // FIXME: Have we already done cycle detection before this method was


Hmmm cc @zanieb

One thing that strikes me here is that the cycle detection error would only kick in when a relevant conflicting group is declared? What happens if there aren't any conflicts declared? Is there cycle detection happening somewhere else?

crates/uv-pypi-types/src/conflicts.rs

Cargo.lock

charliermarsh

Looks good from my perspective modulo the Cargo.lock changes which we should verify before merging.

crates/uv-pypi-types/src/conflicts.rs

Cargo.lock

zanieb · 2025-03-07T17:08:51Z

crates/uv/tests/it/sync.rs

+
+    ----- stderr -----
+    Resolved 5 packages in [TIME]
+    error: Groups `dev` and `dev2` are incompatible with the transitively inferred conflicts: {`example:dev`, `example:dev2`}


I don't quite understand this error, are we just repeating the groups here at the end? What's the value of that?

I believe this is identical to the existing error we show in these cases, except "transitively" has been inserted. Perhaps this should be hashed out separately from this PR, since it doesn't seem to be a change? But for reference, the set at the end can be larger than "just" these two items. The set at the end is of arbitrary size, and the groups we list at the start are the activated, conflicting members.

Oh, if it's not changing here that's fine and we can revisit it separately — I just looked at the test case not the implementation.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.6.5` -> `0.6.9` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.6.9`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#069) [Compare Source](astral-sh/uv@0.6.8...0.6.9) ##### Enhancements - Use `keyring --mode creds` when `authenticate = "always"` ([#12316](astral-sh/uv#12316)) - Fail with specific error message when no password is present and `authenticate = "always"` ([#12313](astral-sh/uv#12313)) ##### Bug fixes - Add boolish value parser for `UV_MANAGED_PYTHON` flags ([#12345](astral-sh/uv#12345)) - Make deserialization non-fatal when assessing source tree revisions ([#12319](astral-sh/uv#12319)) - Use resolver-returned wheel over alternate cached wheel ([#12301](astral-sh/uv#12301)) ##### Documentation - Add experimental `--torch-backend` to the PyTorch guide ([#12317](astral-sh/uv#12317)) - Fix `#keyring-provider` references in alternative index docs ([#12315](astral-sh/uv#12315)) - Fix `--directory` path in examples ([#12165](astral-sh/uv#12165)) ##### Preview changes - Automatically infer the PyTorch index via `--torch-backend=auto` ([#12070](astral-sh/uv#12070)) ### [`v0.6.8`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#068) [Compare Source](astral-sh/uv@0.6.7...0.6.8) ##### Enhancements - Add support for enabling all groups by default with `default-groups = "all"` ([#12289](astral-sh/uv#12289)) - Add simpler `--managed-python` and `--no-managed-python` flags for toggling Python preferences ([#12246](astral-sh/uv#12246)) ##### Performance - Avoid allocations for default cache keys ([#12063](astral-sh/uv#12063)) ##### Bug fixes - Allow local version mismatches when validating lockfile ([#12285](astral-sh/uv#12285)) - Allow owned string when deserializing `requires-python` ([#12278](astral-sh/uv#12278)) - Make cache errors non-fatal in `Planner::build` ([#12281](astral-sh/uv#12281)) ### [`v0.6.7`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#067) [Compare Source](astral-sh/uv@0.6.6...0.6.7) ##### Python - Add CPython 3.14.0a6 - Fix regression where extension modules would use wrong `CXX` compiler on Linux - Enable FTS3 enhanced query syntax for SQLite See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250317) for more details. ##### Enhancements - Add support for `-c` constraints in `uv add` ([#12209](astral-sh/uv#12209)) - Add support for `--global` default version in `uv python pin` ([#12115](astral-sh/uv#12115)) - Always reinstall local source trees passed to `uv pip install` ([#12176](astral-sh/uv#12176)) - Render token claims on publish permission error ([#12135](astral-sh/uv#12135)) - Add pip-compatible `--group` flag to `uv pip install` and `uv pip compile` ([#11686](astral-sh/uv#11686)) ##### Preview features - Avoid creating duplicate directory entries in built wheels ([#12206](astral-sh/uv#12206)) - Allow overriding module names for editable builds ([#12137](astral-sh/uv#12137)) ##### Performance - Avoid replicating core-metadata field on `File` struct ([#12159](astral-sh/uv#12159)) ##### Bug fixes - Add `src` to default cache keys ([#12062](astral-sh/uv#12062)) - Discard insufficient fork markers ([#10682](astral-sh/uv#10682)) - Ensure `python pin --global` creates parent directories if missing ([#12180](astral-sh/uv#12180)) - Fix GraalPy abi tag parsing and discovery ([#12154](astral-sh/uv#12154)) - Remove extraneous script packages in `uv sync --script` ([#12158](astral-sh/uv#12158)) - Remove redundant `activate.bat` output ([#12160](astral-sh/uv#12160)) - Avoid subsequent index hint when no versions are available on the first index ([#9332](astral-sh/uv#9332)) - Error on lockfiles with incoherent wheel versions ([#12235](astral-sh/uv#12235)) ##### Rust API - Update `BaseClientBuild` to accept custom proxies ([#12232](astral-sh/uv#12232)) ##### Documentation - Make testpypi index explicit in example snippet ([#12148](astral-sh/uv#12148)) - Reverse and format the archived changelogs ([#12099](astral-sh/uv#12099)) - Use consistent commas around i.e. and e.g. ([#12157](astral-sh/uv#12157)) - Fix typos in MRE docs ([#12198](astral-sh/uv#12198)) - Fix double space typo ([#12171](astral-sh/uv#12171)) ### [`v0.6.6`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#066) [Compare Source](astral-sh/uv@0.6.5...0.6.6) ##### Python - Add support for dynamic musl Python distributions on x86-64 Linux ([#12121](astral-sh/uv#12121)) - Allow the experimental JIT to be enabled at runtime on Python 3.13 and 3.14 on Linux - Upgrade the build toolchain to LLVM 20, improving performance See the [`python-build-standalone` release notes](https://github.com/astral-sh/python-build-standalone/releases/tag/20250311) for more details. ##### Enhancements - Add `--marker` flag to `uv add` ([#12012](astral-sh/uv#12012)) - Allow overriding module name for uv build backend ([#11884](astral-sh/uv#11884)) - Sync latest Python releases ([#12120](astral-sh/uv#12120)) - Use 'Upload' instead of 'Download' in publish reporter ([#12029](astral-sh/uv#12029)) - Add `[index].authenticate` allowing authentication to be required on an index ([#11896](astral-sh/uv#11896)) - Add support for Windows legacy scripts in `uv tool run` ([#12079](astral-sh/uv#12079)) - Propagate conflicting dependency groups when using `include-group` ([#12005](astral-sh/uv#12005)) - Show ambiguous requirements when `uv add` failed ([#12106](astral-sh/uv#12106)) ##### Performance - Cache workspace discovery ([#12096](astral-sh/uv#12096)) - Insert dependencies into fork state prior to fetching metadata ([#12057](astral-sh/uv#12057)) - Remove some allocations from `uv-auth` ([#12077](astral-sh/uv#12077)) ##### Bug fixes - Avoid considering `PATH` updated when the `export` is commented in the shellrc ([#12043](astral-sh/uv#12043)) - Fix `uv publish` retry on network failures ([#12041](astral-sh/uv#12041)) - Use a sized stream in `uv publish` to comply with WSGI PyPI server constraints ([#12111](astral-sh/uv#12111)) - Fix `uv python install --reinstall` when the version was not previously installed ([#12124](astral-sh/uv#12124)) ##### Preview features - Fix `uv_build` invocation ([#12058](astral-sh/uv#12058)) ##### Documentation - Quote versions string in `python-versions.md` ([#12112](astral-sh/uv#12112)) - Fix tool concept page headings ([#12053](astral-sh/uv#12053)) - Update the `[index].authenticate` docs ([#12102](astral-sh/uv#12102)) - Update versioning policy ([#11666](astral-sh/uv#11666)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

jtfmumm requested a review from BurntSushi March 6, 2025 14:00

jtfmumm temporarily deployed to release March 6, 2025 14:00 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:02 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:04 — with GitHub Actions Inactive

jtfmumm commented Mar 6, 2025

View reviewed changes

jtfmumm temporarily deployed to release March 6, 2025 14:10 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:11 — with GitHub Actions Inactive

jtfmumm force-pushed the jtfm/conflicting-groups branch from d196583 to 6db0af9 Compare March 6, 2025 14:12

jtfmumm temporarily deployed to release March 6, 2025 14:12 — with GitHub Actions Inactive

jtfmumm force-pushed the jtfm/conflicting-groups branch from 6db0af9 to 65f059f Compare March 6, 2025 14:13

jtfmumm temporarily deployed to release March 6, 2025 14:13 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:15 — with GitHub Actions Inactive

jtfmumm force-pushed the jtfm/conflicting-groups branch from c2aeec1 to 7f88996 Compare March 6, 2025 14:27

jtfmumm temporarily deployed to release March 6, 2025 14:27 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:36 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 14:49 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 6, 2025 15:28 — with GitHub Actions Inactive

BurntSushi reviewed Mar 6, 2025

View reviewed changes

jtfmumm temporarily deployed to release March 6, 2025 16:26 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 7, 2025 13:38 — with GitHub Actions Inactive

jtfmumm force-pushed the jtfm/conflicting-groups branch from 05c6bea to 5e28d63 Compare March 7, 2025 14:07

jtfmumm temporarily deployed to release March 7, 2025 14:07 — with GitHub Actions Inactive

jtfmumm force-pushed the jtfm/conflicting-groups branch from 5e28d63 to 23942fb Compare March 7, 2025 14:24

jtfmumm temporarily deployed to release March 7, 2025 14:24 — with GitHub Actions Inactive

jtfmumm temporarily deployed to release March 7, 2025 15:41 — with GitHub Actions Inactive

BurntSushi reviewed Mar 7, 2025

View reviewed changes

Cargo.lock Outdated Show resolved Hide resolved

charliermarsh approved these changes Mar 7, 2025

View reviewed changes

crates/uv-pypi-types/src/conflicts.rs Show resolved Hide resolved

Cargo.lock Outdated Show resolved Hide resolved

jtfmumm temporarily deployed to release March 7, 2025 16:57 — with GitHub Actions Inactive

jtfmumm added 4 commits March 7, 2025 17:59

Conflicting groups should handle conflicting inclusions automatically

e88eb1e

Remove Serialize derive from Conflicts

65f2382

Accept duplicate_torch_and_sympy_because_of_wrong_inferences snapshots

9200d12

..

32cac26

zanieb reviewed Mar 7, 2025

View reviewed changes

jtfmumm force-pushed the jtfm/conflicting-groups branch from 006ee40 to 32cac26 Compare March 7, 2025 21:31

jtfmumm temporarily deployed to release March 7, 2025 21:31 — with GitHub Actions Inactive

Checkout main Cargo.lock

5444d5a

jtfmumm temporarily deployed to release March 8, 2025 10:07 — with GitHub Actions Inactive

zanieb approved these changes Mar 8, 2025

View reviewed changes

jtfmumm merged commit b7968e7 into main Mar 8, 2025
98 checks passed

jtfmumm deleted the jtfm/conflicting-groups branch March 8, 2025 18:21

BrewTestBot mentioned this pull request Mar 12, 2025

uv 0.6.6 Homebrew/homebrew-core#210614

Merged

curtischong mentioned this pull request May 12, 2025

Fix installing conflicting optional dependencies via uv in pyproject.toml Radical-AI/torch-sim#188

Merged

Conflicting groups should handle conflicting inclusions automatically #12005

Conflicting groups should handle conflicting inclusions automatically #12005

Uh oh!

Conversation

jtfmumm commented Mar 6, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtfmumm Mar 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

BurntSushi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

charliermarsh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

charliermarsh Mar 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jtfmumm Mar 6, 2025 •

edited

Loading

charliermarsh Mar 8, 2025 •

edited

Loading