[ty] narrow tagged unions of TypedDict#22104
Merged
Merged
Conversation
oconnor663
requested review from
AlexWaygood,
carljm,
dcreager and
sharkdp
as code owners
December 20, 2025 03:07
Diagnostic diff on typing conformance testsNo changes detected when running ty on typing conformance tests ✅ |
|
Contributor
|
omg look at those pydantic diagnostics disappear |
Member
|
Wow! |
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-key |
0 | 3,124 | 0 |
invalid-assignment |
0 | 74 | 5 |
invalid-argument-type |
2 | 19 | 19 |
invalid-return-type |
2 | 0 | 13 |
possibly-missing-attribute |
0 | 2 | 0 |
type-assertion-failure |
0 | 1 | 0 |
| Total | 4 | 3,220 | 37 |
|
CodSpeed Performance ReportMerging #22104 will not alter performanceComparing Summary
Footnotes
|
oconnor663
commented
Dec 20, 2025
zaniebot
pushed a commit
to zaniebot/ruff
that referenced
this pull request
Dec 20, 2025
Adopt PR astral-sh#22104's approach for narrowing tagged TypedDict unions, with a performance optimization that uses direct filtering for simple unions instead of intersection-based narrowing. The key insight is that for unions containing only TypedDict variants (no intersections or other types), we can directly filter to matching variants in O(n) time, rather than building up intersection types with negations which requires O(n²) simplification. The optimization falls back to the intersection-based approach for complex cases like unions with intersections or non-TypedDict types.
AlexWaygood
reviewed
Dec 20, 2025
AlexWaygood
reviewed
Dec 20, 2025
AlexWaygood
left a comment
AlexWaygood
left a comment
Member
There was a problem hiding this comment.
Nice! It would be great to rebase on main and see what the codspeed report is now #22102 has landed
Comment on lines
+906
to
+914
| let known_equality_result = match (field.declared_ty, rhs_type) { | ||
| (Type::StringLiteral(lhs), Type::StringLiteral(rhs)) => Some(lhs == rhs), | ||
| (Type::IntLiteral(lhs), Type::IntLiteral(rhs)) => Some(lhs == rhs), | ||
| // It's unlikely that the user will mix int and string literals in the same | ||
| // union, but go ahead and handle it. | ||
| (Type::StringLiteral(_), Type::IntLiteral(_)) | ||
| | (Type::IntLiteral(_), Type::StringLiteral(_)) => Some(false), | ||
| _ => None, | ||
| }; |
Member
There was a problem hiding this comment.
another important case to handle here is enum-literal types.
However, there's some additional complications with enum literals, so I think it might be good to postpone enums to a followup. Namely, an enum literal can't be used as a "tag" if the enum class has a custom __eq__ method. That's true if the enum class inherits from IntEnum or StrEnum (we handle this incorrectly in some other situations right now).
Contributor
Author
|
Unfortunately merging |
oconnor663
force-pushed
the
typeddict_union_narrowing
branch
from
December 22, 2025 20:44
f1d30f5 to
ae2d96f
Compare
Contributor
Author
|
04ee4dc is a substantial refactoring. After staring at some profiles and adding logs and stuff, I'm pretty sure the problem is that I'm taking these unions of ~50 TypedDicts and intersecting them with ~49 negated TypedDicts in a lot of places, and that's slow. The new approach is to synthesize a TypedDict with a single field in both cases (the |
oconnor663
commented
Dec 23, 2025
| if u["tag"] == "foo": | ||
| # TODO: `dict & ~<TypedDict ...>` should simplify to `dict` here, but that's currently a | ||
| # false negative in `is_disjoint_impl`. | ||
| reveal_type(u) # revealed: Foo | (dict[Unknown, Unknown] & ~<TypedDict with items 'tag'>) |
Contributor
Author
There was a problem hiding this comment.
The display here changed from ~Bar to ~<TypedDict with items 'tag'>. Probably that's fine?
Member
There was a problem hiding this comment.
I think that's fine. I think the real fix to get rid of this is to implement disjointness between dict and TypedDict types... which theoretically isn't too hard, but I'm a bit worried it might lead to false positives due to missing pieces in our TypedDict/bidirectional-inference logic elsewhere. Best done as a standalone change, anyway.
oconnor663
force-pushed
the
typeddict_union_narrowing
branch
from
December 23, 2025 16:15
04ee4dc to
b31d29b
Compare
AlexWaygood
reviewed
Dec 23, 2025
AlexWaygood
reviewed
Dec 23, 2025
AlexWaygood
reviewed
Dec 23, 2025
AlexWaygood
reviewed
Dec 23, 2025
AlexWaygood
approved these changes
Dec 23, 2025
AlexWaygood
left a comment
AlexWaygood
left a comment
Member
There was a problem hiding this comment.
Awesome work. Thank you!!
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
oconnor663
enabled auto-merge (squash)
December 23, 2025 19:26
Contributor
|
Awesome! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Identify and narrow cases like this:
Fixes part of astral-sh/ty#1479.