Skip to content

Commit 10bda3d

Browse files
authored
[pyupgrade] Fix false positive for TypeVar with default on Python <3.13 (UP046,UP047) (#21045)
## Summary Type default for Type parameter was added in Python 3.13 (PEP 696). `typing_extensions.TypeVar` backports the default argument to earlier versions. `UP046` & `UP047` were getting triggered when `typing_extensions.TypeVar` with `default` argument was used on python version < 3.13 It shouldn't be triggered for python version < 3.13 This commit fixes the bug by adding a python version check before triggering them. Fixes #20929. ## Test Plan ### Manual testing 1 As the issue author pointed out in #20929 (comment), ran the following on `main` branch: > % cargo run -p ruff -- check ../efax/ --target-version py312 --no-cache <details><summary>Output</summary> ```zsh Compiling ruff_linter v0.14.1 (/Users/prakhar/ruff/crates/ruff_linter) Compiling ruff v0.14.1 (/Users/prakhar/ruff/crates/ruff) Compiling ruff_graph v0.1.0 (/Users/prakhar/ruff/crates/ruff_graph) Compiling ruff_workspace v0.0.0 (/Users/prakhar/ruff/crates/ruff_workspace) Compiling ruff_server v0.2.2 (/Users/prakhar/ruff/crates/ruff_server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 6.72s Running `target/debug/ruff check ../efax/ --target-version py312 --no-cache` UP046 Generic class `ExpectationParametrization` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/expectation_parametrization.py:17:48 | 17 | class ExpectationParametrization(Distribution, Generic[NP]): | ^^^^^^^^^^^ 18 | """The expectation parametrization of an exponential family distribution. | help: Use type parameters UP046 Generic class `ExpToNat` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/mixins/exp_to_nat/exp_to_nat.py:27:68 | 26 | @DataClass 27 | class ExpToNat(ExpectationParametrization[NP], SimpleDistribution, Generic[NP]): | ^^^^^^^^^^^ 28 | """This mixin implements the conversion from expectation to natural parameters. | help: Use type parameters UP046 Generic class `HasEntropyEP` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/mixins/has_entropy.py:25:20 | 23 | HasEntropy, 24 | JaxAbstractClass, 25 | Generic[NP]): | ^^^^^^^^^^^ 26 | @abstract_jit 27 | @AbstractMethod | help: Use type parameters UP046 Generic class `HasEntropyNP` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/mixins/has_entropy.py:64:20 | 62 | class HasEntropyNP(NaturalParametrization[EP], 63 | HasEntropy, 64 | Generic[EP]): | ^^^^^^^^^^^ 65 | @jit 66 | @Final | help: Use type parameters UP046 Generic class `NaturalParametrization` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/natural_parametrization.py:43:30 | 41 | class NaturalParametrization(Distribution, 42 | JaxAbstractClass, 43 | Generic[EP, Domain]): | ^^^^^^^^^^^^^^^^^^^ 44 | """The natural parametrization of an exponential family distribution. | help: Use type parameters UP046 Generic class `Structure` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/efax/_src/structure/structure.py:31:17 | 30 | @DataClass 31 | class Structure(Generic[P]): | ^^^^^^^^^^ 32 | """This class generalizes the notion of type for Distribution objects. | help: Use type parameters UP046 Generic class `DistributionInfo` uses `Generic` subclass instead of type parameters --> /Users/prakhar/efax/tests/distribution_info.py:20:24 | 20 | class DistributionInfo(Generic[NP, EP, Domain]): | ^^^^^^^^^^^^^^^^^^^^^^^ 21 | def __init__(self, dimensions: int = 1, safety: float = 0.0) -> None: 22 | super().__init__() | help: Use type parameters Found 7 errors. No fixes available (7 hidden fixes can be enabled with the `--unsafe-fixes` option). ``` </details> Running it after the changes: ```zsh ruff % cargo run -p ruff -- check ../efax/ --target-version py312 --no-cache Compiling ruff_linter v0.14.1 (/Users/prakhar/ruff/crates/ruff_linter) Compiling ruff v0.14.1 (/Users/prakhar/ruff/crates/ruff) Compiling ruff_graph v0.1.0 (/Users/prakhar/ruff/crates/ruff_graph) Compiling ruff_workspace v0.0.0 (/Users/prakhar/ruff/crates/ruff_workspace) Compiling ruff_server v0.2.2 (/Users/prakhar/ruff/crates/ruff_server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.86s Running `target/debug/ruff check ../efax/ --target-version py312 --no-cache` All checks passed! ``` --- ### Manual testing 2 Ran the check on the following script (mainly to verify `UP047`): ```py from __future__ import annotations from typing import Generic from typing_extensions import TypeVar T = TypeVar("T", default=int) def generic_function(var: T) -> T: return var Q = TypeVar("Q", default=str) class GenericClass(Generic[Q]): var: Q ``` On `main` branch: > ruff % cargo run -p ruff -- check ~/up046.py --target-version py312 --preview --no-cache <details><summary>Output</summary> ```zsh Compiling ruff_linter v0.14.1 (/Users/prakhar/ruff/crates/ruff_linter) Compiling ruff v0.14.1 (/Users/prakhar/ruff/crates/ruff) Compiling ruff_graph v0.1.0 (/Users/prakhar/ruff/crates/ruff_graph) Compiling ruff_workspace v0.0.0 (/Users/prakhar/ruff/crates/ruff_workspace) Compiling ruff_server v0.2.2 (/Users/prakhar/ruff/crates/ruff_server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.43s Running `target/debug/ruff check /Users/prakhar/up046.py --target-version py312 --preview --no-cache` UP047 Generic function `generic_function` should use type parameters --> /Users/prakhar/up046.py:10:5 | 10 | def generic_function(var: T) -> T: | ^^^^^^^^^^^^^^^^^^^^^^^^ 11 | return var | help: Use type parameters UP046 Generic class `GenericClass` uses `Generic` subclass instead of type parameters --> /Users/prakhar/up046.py:17:20 | 17 | class GenericClass(Generic[Q]): | ^^^^^^^^^^ 18 | var: Q | help: Use type parameters Found 2 errors. No fixes available (2 hidden fixes can be enabled with the `--unsafe-fixes` option). ``` </details> After the fix (this branch): ```zsh ruff % cargo run -p ruff -- check ~/up046.py --target-version py312 --preview --no-cache Compiling ruff_linter v0.14.1 (/Users/prakhar/ruff/crates/ruff_linter) Compiling ruff v0.14.1 (/Users/prakhar/ruff/crates/ruff) Compiling ruff_graph v0.1.0 (/Users/prakhar/ruff/crates/ruff_graph) Compiling ruff_workspace v0.0.0 (/Users/prakhar/ruff/crates/ruff_workspace) Compiling ruff_server v0.2.2 (/Users/prakhar/ruff/crates/ruff_server) Finished `dev` profile [unoptimized + debuginfo] target(s) in 7.40s Running `target/debug/ruff check /Users/prakhar/up046.py --target-version py312 --preview --no-cache` All checks passed! ``` Signed-off-by: Prakhar Pratyush <[email protected]>
1 parent e55bc94 commit 10bda3d

File tree

9 files changed

+67
-14
lines changed

9 files changed

+67
-14
lines changed
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
"""This is placed in a separate fixture as `TypeVar` needs to be imported
2+
from `typing_extensions` to support default arguments in Python version < 3.13.
3+
We verify that UP046 doesn't apply in this case.
4+
"""
5+
6+
from typing import Generic
7+
from typing_extensions import TypeVar
8+
9+
T = TypeVar("T", default=str)
10+
11+
12+
class DefaultTypeVar(Generic[T]):
13+
var: T
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
"""This is placed in a separate fixture as `TypeVar` needs to be imported
2+
from `typing_extensions` to support default arguments in Python version < 3.13.
3+
We verify that UP047 doesn't apply in this case.
4+
"""
5+
6+
from typing_extensions import TypeVar
7+
8+
T = TypeVar("T", default=int)
9+
10+
11+
def default_var(var: T) -> T:
12+
return var

crates/ruff_linter/src/rules/pyupgrade/mod.rs

Lines changed: 18 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -111,7 +111,7 @@ mod tests {
111111
#[test_case(Rule::NonPEP695TypeAlias, Path::new("UP040.pyi"))]
112112
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046_0.py"))]
113113
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046_1.py"))]
114-
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047.py"))]
114+
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047_0.py"))]
115115
#[test_case(Rule::PrivateTypeParameter, Path::new("UP049_0.py"))]
116116
#[test_case(Rule::PrivateTypeParameter, Path::new("UP049_1.py"))]
117117
#[test_case(Rule::UselessClassMetaclassType, Path::new("UP050.py"))]
@@ -125,6 +125,22 @@ mod tests {
125125
Ok(())
126126
}
127127

128+
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046_2.py"))]
129+
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047_1.py"))]
130+
fn rules_not_applied_default_typevar_backported(rule_code: Rule, path: &Path) -> Result<()> {
131+
let snapshot = path.to_string_lossy().to_string();
132+
let diagnostics = test_path(
133+
Path::new("pyupgrade").join(path).as_path(),
134+
&settings::LinterSettings {
135+
preview: PreviewMode::Enabled,
136+
unresolved_target_version: PythonVersion::PY312.into(),
137+
..settings::LinterSettings::for_rule(rule_code)
138+
},
139+
)?;
140+
assert_diagnostics!(snapshot, diagnostics);
141+
Ok(())
142+
}
143+
128144
#[test_case(Rule::SuperCallWithParameters, Path::new("UP008.py"))]
129145
#[test_case(Rule::TypingTextStrAlias, Path::new("UP019.py"))]
130146
fn rules_preview(rule_code: Rule, path: &Path) -> Result<()> {
@@ -144,7 +160,7 @@ mod tests {
144160
#[test_case(Rule::NonPEP695TypeAlias, Path::new("UP040.pyi"))]
145161
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046_0.py"))]
146162
#[test_case(Rule::NonPEP695GenericClass, Path::new("UP046_1.py"))]
147-
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047.py"))]
163+
#[test_case(Rule::NonPEP695GenericFunction, Path::new("UP047_0.py"))]
148164
fn type_var_default_preview(rule_code: Rule, path: &Path) -> Result<()> {
149165
let snapshot = format!("{}__preview_diff", path.to_string_lossy());
150166
assert_diagnostics_diff!(

crates/ruff_linter/src/rules/pyupgrade/rules/pep695/mod.rs

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ use std::fmt::Display;
66

77
use itertools::Itertools;
88
use ruff_python_ast::{
9-
self as ast, Arguments, Expr, ExprCall, ExprName, ExprSubscript, Identifier, Stmt, StmtAssign,
10-
TypeParam, TypeParamParamSpec, TypeParamTypeVar, TypeParamTypeVarTuple,
9+
self as ast, Arguments, Expr, ExprCall, ExprName, ExprSubscript, Identifier, PythonVersion,
10+
Stmt, StmtAssign, TypeParam, TypeParamParamSpec, TypeParamTypeVar, TypeParamTypeVarTuple,
1111
name::Name,
1212
visitor::{self, Visitor},
1313
};
@@ -369,15 +369,19 @@ fn in_nested_context(checker: &Checker) -> bool {
369369
}
370370

371371
/// Deduplicate `vars`, returning `None` if `vars` is empty or any duplicates are found.
372-
/// Also returns `None` if any `TypeVar` has a default value and preview mode is not enabled.
372+
/// Also returns `None` if any `TypeVar` has a default value and the target Python version
373+
/// is below 3.13 or preview mode is not enabled. Note that `typing_extensions` backports
374+
/// the default argument, but the rule should be skipped in that case.
373375
fn check_type_vars<'a>(vars: Vec<TypeVar<'a>>, checker: &Checker) -> Option<Vec<TypeVar<'a>>> {
374376
if vars.is_empty() {
375377
return None;
376378
}
377379

378-
// If any type variables have defaults and preview mode is not enabled, skip the rule
380+
// If any type variables have defaults, skip the rule unless
381+
// running with preview mode enabled and targeting Python 3.13+.
379382
if vars.iter().any(|tv| tv.default.is_some())
380-
&& !is_type_var_default_enabled(checker.settings())
383+
&& (checker.target_version() < PythonVersion::PY313
384+
|| !is_type_var_default_enabled(checker.settings()))
381385
{
382386
return None;
383387
}
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
source: crates/ruff_linter/src/rules/pyupgrade/mod.rs
3+
---
4+
Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
source: crates/ruff_linter/src/rules/pyupgrade/mod.rs
33
---
44
UP047 [*] Generic function `f` should use type parameters
5-
--> UP047.py:12:5
5+
--> UP047_0.py:12:5
66
|
77
12 | def f(t: T) -> T:
88
| ^^^^^^^
@@ -20,7 +20,7 @@ help: Use type parameters
2020
note: This is an unsafe fix and may change runtime behavior
2121

2222
UP047 [*] Generic function `g` should use type parameters
23-
--> UP047.py:16:5
23+
--> UP047_0.py:16:5
2424
|
2525
16 | def g(ts: tuple[*Ts]) -> tuple[*Ts]:
2626
| ^^^^^^^^^^^^^^^^^
@@ -38,7 +38,7 @@ help: Use type parameters
3838
note: This is an unsafe fix and may change runtime behavior
3939

4040
UP047 [*] Generic function `h` should use type parameters
41-
--> UP047.py:20:5
41+
--> UP047_0.py:20:5
4242
|
4343
20 | def h(
4444
| _____^
@@ -62,7 +62,7 @@ help: Use type parameters
6262
note: This is an unsafe fix and may change runtime behavior
6363

6464
UP047 [*] Generic function `i` should use type parameters
65-
--> UP047.py:29:5
65+
--> UP047_0.py:29:5
6666
|
6767
29 | def i(s: S) -> S:
6868
| ^^^^^^^
@@ -80,7 +80,7 @@ help: Use type parameters
8080
note: This is an unsafe fix and may change runtime behavior
8181

8282
UP047 [*] Generic function `broken_fix` should use type parameters
83-
--> UP047.py:39:5
83+
--> UP047_0.py:39:5
8484
|
8585
37 | # TypeVars with the new-style generic syntax and will be rejected by type
8686
38 | # checkers
@@ -100,7 +100,7 @@ help: Use type parameters
100100
note: This is an unsafe fix and may change runtime behavior
101101

102102
UP047 [*] Generic function `any_str_param` should use type parameters
103-
--> UP047.py:43:5
103+
--> UP047_0.py:43:5
104104
|
105105
43 | def any_str_param(s: AnyStr) -> AnyStr:
106106
| ^^^^^^^^^^^^^^^^^^^^^^^^
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Added: 1
1111

1212
--- Added ---
1313
UP047 [*] Generic function `default_var` should use type parameters
14-
--> UP047.py:51:5
14+
--> UP047_0.py:51:5
1515
|
1616
51 | def default_var(v: V) -> V:
1717
| ^^^^^^^^^^^^^^^^^
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
source: crates/ruff_linter/src/rules/pyupgrade/mod.rs
3+
---
4+

0 commit comments

Comments
 (0)