Add support for `__lazy_modules__` by ntBre · Pull Request #28459 · astral-sh/ruff

ntBre · 2026-09-09T15:25:50Z

Summary

For a bit of background, __lazy_modules__ is a backwards compatibility mechanism for lazy imports, introduced in PEP 810 alongside the lazy keyword itself. Instead of declaring an import lazy with the keyword, it can also be listed in a module-level constant named __lazy_modules__:

lazy import foo

# equivalently
__lazy_modules__ = ["foo"]
import foo

This can be used in projects that support older Python versions, as well as 3.15.

It also supports more dynamic approaches (that this PR doesn't), as noted in one of my inline comments, such as:

class LazyImporter:
    def __contains__(self, name): return True

__lazy_modules__ = LazyImporter()

As one example from the PEP for emulating wildcard lazy imports.

The PR is pretty big in total, but it's intended to be reviewed commit-by-commit. As usual, the very
first commit is empty for filling the PR summary, so the numbers below refer to the remaining
non-empty commits.

The first commit adds __lazy_modules__ tracking to the semantic model. Because the declaration
affects later imports and can also be overwritten, we just store whatever was assigned to it last
for later lookup when visiting other code.

The second commit expands TID254 to recognize __lazy_modules__ and includes a few helper methods
on the semantic model that are used by later rules as well.

The third commit adds __lazy_modules__ support to TID255, which also relies on a new
BindingFlags::LAZY variant. This marks bindings as originating from a lazy import, which is easier
than trying to resolve a sequence of potentially shadowed __lazy_modules__ bindings to check if an
import was lazy. For example, we set the LAZY flag on foo here, which would otherwise require
resolve multiple previous bindings:

__lazy_modules__ = ["foo"]
import foo

__lazy_modules__ = []

class C(foo.Something): ...

The last four commits make smaller changes to four other rules. As with TID254 and TID255, the
primary change is that we withhold a fix that involves __lazy_modules__ when it would change the
module's lazy status. Initially I had hoped to add fixes that modified __lazy_modules__ (adding to
it when we'd otherwise add lazy, removing from it when we'd remove lazy), but this turned out to
be quite complicated. Unlike the lazy keyword, __lazy_modules__ has nonlocal effects that could
change the behavior of any import from the same module in the file. Trying to resolve all of that
to be sure the fix is reasaonable was a real mess, and reverting Codex's attempt saved almost 700
lines for TID255 alone.

Similarly, in case this comes up in someone else's Codex review, my Codex initially tried to handle
even more edge cases with __lazy_modules__ definitions, such as definitions in a loop or
redefining it with * imports and things like that. I tried to rein it in toward something more
like our __all__ and from __future__ import annotations handling, which again seemed like the
better tradeoff to me.

Closes #25170.

Test Plan

New mdtests. The ecosystem results all look correct. They are cases where TID255 now fires in modules where __lazy_modules__ is defined.

Summary -- The PR is pretty big in total, but it's intended to be reviewed commit-by-commit. As usual, the very first commit is empty for filling the PR summary, so the numbers below refer to the remaining non-empty commits. The first commit adds `__lazy_modules__` tracking to the semantic model. Because the declaration affects later imports and can also be overwritten, we just store whatever was assigned to it last for later lookup when visiting other code. The second commit expands [TID254] to recognize `__lazy_modules__` and includes a few helper methods on the semantic model that are used by later rules as well. The third commit adds `__lazy_modules__` support to [TID255], which also relies on a new `BindingFlags::LAZY` variant. This marks bindings as originating from a lazy import, which is easier than trying to resolve a sequence of potentially shadowed `__lazy_modules__` bindings to check if an import was lazy. For example, we set the `LAZY` flag on `foo` here, which would otherwise require resolve multiple previous bindings: ```py __lazy_modules__ = ["foo"] import foo __lazy_modules__ = [] class C(foo.Something): ... ``` The last four commits make smaller changes to four other rules. As with TID254 and TID255, the primary change is that we withhold a fix that involves `__lazy_modules__` when it would change the module's lazy status. Initially I had hoped to add fixes that modified `__lazy_modules__` (adding to it when we'd otherwise add `lazy`, removing from it when we'd remove `lazy`), but this turned out to be quite complicated. Unlike the `lazy` keyword, `__lazy_modules__` has nonlocal effects that could change the behavior of _any_ import from the same module in the file. Trying to resolve all of that to be sure the fix is reasaonable was a real mess, and reverting Codex's attempt saved almost 700 lines for TID255 alone. Similarly, in case this comes up in someone else's Codex review, my Codex initially tried to handle even more edge cases with `__lazy_modules__` definitions, such as definitions in a loop or redefining it with `*` imports and things like that. I tried to rein it in toward something more like our `__all__` and `from __future__ import annotations` handling, which again seemed like the better tradeoff to me. Closes #25170. [TID254]: https://docs.astral.sh/ruff/rules/lazy-import-mismatch/ [TID255]: https://docs.astral.sh/ruff/rules/lazy-import-immediately-resolved/ Test Plan -- New mdtests

astral-sh-bot · 2026-09-09T15:36:04Z

`ruff-ecosystem` results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+18 -0 violations, +0 -0 fixes in 2 projects; 57 projects unchanged)

pypa/cibuildwheel (+6 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ cibuildwheel/logger.py:110:6: lazy-import-immediately-resolved Lazy import `functools` is resolved immediately
+ cibuildwheel/logger.py:116:6: lazy-import-immediately-resolved Lazy import `functools` is resolved immediately
+ cibuildwheel/logger.py:253:6: lazy-import-immediately-resolved Lazy import `contextlib` is resolved immediately
+ cibuildwheel/logger.py:378:6: lazy-import-immediately-resolved Lazy import `functools` is resolved immediately
+ cibuildwheel/logger.py:382:6: lazy-import-immediately-resolved Lazy import `functools` is resolved immediately
+ cibuildwheel/options.py:573:6: lazy-import-immediately-resolved Lazy import `contextlib` is resolved immediately

scikit-build/scikit-build-core (+12 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --no-fix --output-format concise --preview

+ src/scikit_build_core/file_api/model/common.py:21:13: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/common.py:22:12: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:18:30: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:19:27: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:20:36: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:21:25: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:27:11: lazy-import-immediately-resolved Lazy import `Optional` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:27:20: lazy-import-immediately-resolved Lazy import `Path` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:28:22: lazy-import-immediately-resolved Lazy import `Optional` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:29:9: lazy-import-immediately-resolved Lazy import `Optional` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:30:14: lazy-import-immediately-resolved Lazy import `Optional` is resolved immediately
+ src/scikit_build_core/file_api/model/toolchains.py:31:13: lazy-import-immediately-resolved Lazy import `Optional` is resolved immediately

Changes by rule (1 rules affected)

code	total	+ violation	- violation	+ fix	- fix
lazy-import-immediately-resolved	18	18	0	0	0

ntBre · 2026-09-09T15:32:44Z

+### Empty declarations on older Python versions
+
+An empty declaration enables lazy-import policies even when the target version predates the `lazy`
+keyword.
+
+```py
+__lazy_modules__ = []
+import json  # error: [lazy-import-mismatch]
+```


It's possible that we don't want to do this, but it made sense to me to consider a __lazy_modules__ declaration (plus enabling a lazy-import rule) as enough opt-in to flag these even for a configured Python version before 3.15.

Similarly, I added the "In the future..." note below in analogy to #18502.

ntBre · 2026-09-09T15:36:30Z

+### Unknown declarations
+
+An unsupported declaration value does not establish whether the import violates the policy, so no
+diagnostic is emitted.
+
+```py
+__lazy_modules__ = configured_modules()
+import json
+```


One of the relevant sections from the PEP is here:

It is worth noting that the implementation performs membership checks by calling __contains__ on the __lazy_modules__ object. Consequently, users requiring wildcard behavior may provide a custom object implementing __contains__ to return True for all queries or other desired patterns.

We treat this as an unknown laziness, different from certain laziness or certain eagerness.

ntBre · 2026-09-09T15:40:30Z

-
-    if selector.includes_all() {
-        report_all_matching_imports(checker, stmt, policy, selector);
+    if checker.lazy_import_context().is_some()


Just noting that this is a pretty awkwardly named API for this usage because its previous callers were for semantic syntax error reporting. If the context is Some, it means that we're in an invalid lazy import context:

ruff/crates/ruff_linter/src/checkers/ast/mod.rs

Lines 341 to 362 in 83c3487

pub(crate) fn lazy_import_context(&self) -> Option<LazyImportContext> {

match self.semantic.current_scope().kind {

// Possible, but invalid positions.

ScopeKind::Function(_) => return Some(LazyImportContext::Function),

ScopeKind::Class(_) => return Some(LazyImportContext::Class),

// Valid position.

ScopeKind::Module => {}

// Impossible positions because lambdas and comprehensions can't contain statements.

ScopeKind::Lambda(_)

| ScopeKind::Generator { .. }

| ScopeKind::Type

| ScopeKind::DunderClassCell => {}

}

for statement in self.semantic.current_statements().skip(1) {

if matches!(statement, Stmt::Try(_)) {

return Some(LazyImportContext::TryExceptBlocks);

}

}

None

}

I guess I could rename the Checker method at least.

ntBre · 2026-09-09T17:01:39Z

+
+    /// Test exact module membership in the current `__lazy_modules__` declaration.
+    /// Returns [`ImportLaziness::Unknown`] when the declaration is not a literal collection of strings.
+    pub fn module_laziness(&self, module: &str) -> ImportLaziness {


I wondered if we should try to cache this or compute it up front when storing self.lazy_modules, but I guess it only matters if you have defined __lazy_modules__ and probably also if it gets pretty long. If we preprocess, we'd likely end up allocating a Vec<&str> anyway.

MichaReiser

Great write up and the commits made reviewing very easy. Thank you. There's one additional edge case that needs handling (can be a follow up PR) and we should add a test for relative imports.

I think our semantic model already handles relative imports but let's add a test for it, similar to what's in the specification https://docs.python.org/3.15/reference/simple_stmts.html#lazy-imports

I also think we need to extend the rule to handle conditional lazy imports. I don't know how common they are and we can do this as a follow up if you prefer, but what I mean is

if False:
    __lazy_modules__ = ["json"]
import json
json.dumps({})  # PR reports an immediately resolved lazy import

I think what we need is to track whether a __lazy_modules__ entry is seen on every path or only some. If the module is only in __lazy_modules__ in some branches, then the module's state is Unknown (could be eager or lazy). If the module is in all __lazy_modules__ assignments, then it's lazy.

MichaReiser · 2026-09-10T06:50:08Z

+/// The rule also recognizes imports made lazy by a literal `__lazy_modules__`
+/// declaration, including when the target version is older than Python 3.15.
+/// This declaration allows a module to use lazy imports on Python 3.15 and later
+/// while retaining eager imports on older versions.
+///


Do we need to document known limitations around dynamic calls?

MichaReiser · 2026-09-10T06:51:57Z

+    pub fn module_laziness(&self, module: &str) -> ImportLaziness {
+        let Some(value) = self.lazy_modules else {
+            return ImportLaziness::Eager;
+        };
+        let (Expr::List(ast::ExprList { elts: elements, .. })
+        | Expr::Tuple(ast::ExprTuple { elts: elements, .. })
+        | Expr::Set(ast::ExprSet { elts: elements, .. })) = value
+        else {
+            return ImportLaziness::Unknown;
+        };
+        if !elements.iter().all(Expr::is_string_literal_expr) {
+            return ImportLaziness::Unknown;
+        }
+        if elements.iter().any(|element| {
+            element
+                .as_string_literal_expr()
+                .is_some_and(|literal| literal.value.to_str() == module)
+        }) {
+            ImportLaziness::Lazy
+        } else {
+            ImportLaziness::Eager
+        }
+    }


I'm inclined to parse out the lazy module names when we see the assignment instead of matching on Expr for every use. It's probably a bit more expensive but probably required anyway to support conditional assignments.

MichaReiser · 2026-09-10T06:53:38Z

+import json
+__lazy_modules__ = ["json"]


Is this an idea for another rule that warns about earlier imports that are later marked as lazy (same when using the lazy import syntax?)

Summary -- This is a follow-up to #28459 adding simple support for conditionally defining `__lazy_modules__`. We now track per-module laziness, which allows us to merge definitions like: ```py __lazy_modules__ = ["foo"] if condition: __lazy_modules__ = ["foo", "bar"] ``` into known laziness for `foo` and unknown laziness for `bar`. As in #28459, that unknown laziness is still helpful and distinct from known eagerness because it suppresses diagnostics. The one quirk with the `branch_id` tracking used here is that it doesn't handle exhaustive branches. For example, it would be nice if we noticed that `foo` is always lazy here: ```py if condition: __lazy_modules__ = ["foo", "bar"] else: __lazy_modules__ = ["foo", "baz"] ``` but it's instead merged with the implicit known eagerness of all imports before any `__lazy_modules__` assignment, producing `(foo, Unknown)`. So this is effectively equivalent to the version without an `else` for `foo`. I think this is a decent tradeoff that biases us against false positives, though, and doesn't involve any extra bookkeeping in `if` statement traversal. One of my three Codex reviewers also identified this as a false negative, where the assignment and use are in the same branch, but we've already marked the module as `Unknown`: ```py if condition: __lazy_modules__ = ["json"] import json json.dumps({}) ``` But again we're at least biased in the right direction, and we'd need more bookkeeping to identify this. Test Plan -- New mdtests for TID254 that exercise this behavior

ntBre · 2026-09-10T16:04:54Z

Thanks for the review! I addressed the simple stuff here and then opened #28491 for the conditional handling, which also didn't end up being too bad thanks to your suggestions on the approach.

Summary -- This is a follow-up to #28459 adding simple support for conditionally defining `__lazy_modules__`. We now track per-module laziness, which allows us to merge definitions like: ```py __lazy_modules__ = ["foo"] if condition: __lazy_modules__ = ["foo", "bar"] ``` into known laziness for `foo` and unknown laziness for `bar`. As in #28459, that unknown laziness is still helpful and distinct from known eagerness because it suppresses diagnostics. The one quirk with the `branch_id` tracking used here is that it doesn't handle exhaustive branches. For example, it would be nice if we noticed that `foo` is always lazy here: ```py if condition: __lazy_modules__ = ["foo", "bar"] else: __lazy_modules__ = ["foo", "baz"] ``` but it's instead merged with the implicit known eagerness of all imports before any `__lazy_modules__` assignment, producing `(foo, Unknown)`. So this is effectively equivalent to the version without an `else` for `foo`. I think this is a decent tradeoff that biases us against false positives, though, and doesn't involve any extra bookkeeping in `if` statement traversal. One of my three Codex reviewers also identified this as a false negative, where the assignment and use are in the same branch, but we've already marked the module as `Unknown`: ```py if condition: __lazy_modules__ = ["json"] import json json.dumps({}) ``` But again we're at least biased in the right direction, and we'd need more bookkeeping to identify this, so I was okay accepting that for now. Test Plan -- New mdtests for TID254 that exercise this behavior

…aliases * origin/main: (30 commits) [ty] Respect instance dictionary storage for slotted classes (#27749) Add a GitHub repository threat model for Ruff (#28395) [ty] Preserve wrapped signatures in nominal descriptor checks (#28466) [ty] Avoid rebinding extracted method calls (#28469) Recognize quoted types in `typing.TypeForm` (#28507) Support conditional assignment to `__lazy_modules__` (#28491) Add support for `__lazy_modules__` (#28459) [`pyupgrade`] Stop recommending deprecated `ByteString` aliases (`UP035`) (#28498) [ty] Anchor default exclude patterns at the project root (#28463) [ty] Add playground command to toggle inlay hints (#28517) [ty] Ignore divergent markers when detecting descriptors (#28514) [ty] Reuse rendered union elements when displaying types (#28494) [ty] Fix assignability of bounded typevars to intersection types (#28479) [ty] Retain package listings between resolution steps (#28278) Add accessors for comparison expressions (#28485) [ty] Refactor module resolution to use a `ModuleDirectory` abstraction. (#28418) [ty] Reduce retained AST memory by shrinking expressions (#28335) Bump version to 0.16.7 (#28496) Install rustfmt before linting releases (#28495) ensure prepare release changes pass prek (#28488) ... # Conflicts: # crates/ty_python_semantic/src/types/bool.rs

ntBre added 2 commits September 9, 2026 11:22

Track module-level __lazy_modules__ assignments

dafb8b8

ntBre added rule Implementing or modifying a lint rule python315 Related to Python 3.15 labels Sep 9, 2026

ntBre mentioned this pull request Sep 9, 2026

Python 3.15 Support #28206

Closed

27 of 32 tasks

Recognize module-declared lazy imports in TID254

4d57760

ntBre force-pushed the brent/lazy-modules branch from ee1c521 to 4edd863 Compare September 9, 2026 16:33

ntBre added 5 commits September 9, 2026 12:53

Remember effective import laziness for TID255

0b0d49a

Preserve module-declared laziness in PLR0402 fixes

e0bed85

Preserve module-declared laziness in UP023 fixes

0b4d67d

Preserve module-declared laziness in UP026 fixes

b586141

Preserve laziness in UP035 import replacements

e30f6a4

ntBre force-pushed the brent/lazy-modules branch from 4edd863 to e30f6a4 Compare September 9, 2026 16:53

ntBre commented Sep 9, 2026

View reviewed changes

ntBre marked this pull request as ready for review September 9, 2026 17:03

ntBre requested a review from MichaReiser September 9, 2026 17:08

MichaReiser approved these changes Sep 10, 2026

View reviewed changes

ntBre added 4 commits September 10, 2026 09:15

assert on messages

3b97118

document dynamic assignment limitation

d3e4552

add relative import tests

dd77ba8

eagerly process lazy modules

dd6cbf2

ntBre added this pull request to stack #28492 September 10, 2026 16:01

ntBre mentioned this pull request Sep 10, 2026

Support conditional assignment to __lazy_modules__ #28491

Merged

ntBre merged commit 67d03a5 into main Sep 11, 2026
50 checks passed

ntBre deleted the brent/lazy-modules branch September 11, 2026 13:55

BrewTestBot mentioned this pull request Sep 16, 2026

ruff 0.16.8 Homebrew/homebrew-core#311546

Merged

Navigation Menu

Repository navigation

Add support for `__lazy_modules__` - #28459

Add support for `__lazy_modules__`#28459
ntBre merged 12 commits into
mainfrom
brent/lazy-modules

ntBre commented Sep 9, 2026 •

edited

Loading

Uh oh!

astral-sh-bot Bot commented Sep 9, 2026

Uh oh!

ntBre Sep 9, 2026

Uh oh!

ntBre Sep 9, 2026

Uh oh!

ntBre Sep 9, 2026

Uh oh!

ntBre Sep 9, 2026

Uh oh!

MichaReiser left a comment

Uh oh!

Uh oh!

MichaReiser Sep 10, 2026

Uh oh!

MichaReiser Sep 10, 2026

Uh oh!

MichaReiser Sep 10, 2026

Uh oh!

ntBre Sep 10, 2026

Uh oh!

ntBre commented Sep 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	pub(crate) fn lazy_import_context(&self) -> Option<LazyImportContext> {
	match self.semantic.current_scope().kind {
	// Possible, but invalid positions.
	ScopeKind::Function(_) => return Some(LazyImportContext::Function),
	ScopeKind::Class(_) => return Some(LazyImportContext::Class),
	// Valid position.
	ScopeKind::Module => {}
	// Impossible positions because lambdas and comprehensions can't contain statements.
	ScopeKind::Lambda(_)
	\| ScopeKind::Generator { .. }
	\| ScopeKind::Type
	\| ScopeKind::DunderClassCell => {}
	}

	for statement in self.semantic.current_statements().skip(1) {
	if matches!(statement, Stmt::Try(_)) {
	return Some(LazyImportContext::TryExceptBlocks);
	}
	}

	None
	}

Conversation

ntBre commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

astral-sh-bot Bot commented Sep 9, 2026

ruff-ecosystem results

Linter (stable)

Linter (preview)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaReiser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ntBre commented Sep 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ntBre commented Sep 9, 2026 •

edited

Loading

`ruff-ecosystem` results