[ty] Add diagnostics for invalid `await` expressions #19711

theammir · 2025-08-03T13:18:40Z

Summary

This PR adds a new lint, invalid-await, for all sorts of reasons why an object may not be awaitable, as discussed in astral-sh/ty#919.
Precisely, __await__ is guarded against being missing, possibly unbound, or improperly defined (expects additional arguments or doesn't return an iterator).

Of course, diagnostics need to be fine-tuned. If __await__ cannot be called with no extra arguments, it indicates an error (or a quirk?) in the method signature, not at the call site. Without any doubt, such an object is not Awaitable, but I feel like talking about arguments for an implicit call is a bit leaky.
I didn't reference any actual diagnostic messages in the lint definition, because I want to hear feedback first.

Also, there's no mention of the actual required method signature for __await__ anywhere in the docs. The only reference I had is the typing stub. I basically ended up linking [Awaitable] to "must implement __await__", which is insufficient on its own.

Test Plan

The following code was tested:

import asyncio
import typing


class Awaitable:
    def __await__(self) -> typing.Generator[typing.Any, None, int]:
        yield None
        return 5


class NoDunderMethod:
    pass


class InvalidAwaitArgs:
    def __await__(self, value: int) -> int:
        return value


class InvalidAwaitReturn:
    def __await__(self) -> int:
        return 5


class InvalidAwaitReturnImplicit:
    def __await__(self):
        pass


async def main() -> None:
    result = await Awaitable()  # valid
    result = await NoDunderMethod()  # `__await__` is missing
    result = await InvalidAwaitReturn()  # `__await__` returns `int`, which is not a valid iterator 
    result = await InvalidAwaitArgs()  # `__await__` expects additional arguments and cannot be called implicitly
    result = await InvalidAwaitReturnImplicit()  # `__await__` returns `Unknown`, which is not a valid iterator


asyncio.run(main())

github-actions · 2025-08-03T13:28:39Z

Diagnostic diff on typing conformance tests

Changes were detected when running ty on typing conformance tests

--- old-output.txt	2025-08-14 21:31:01.838153846 +0000
+++ new-output.txt	2025-08-14 21:31:01.904154062 +0000
@@ -1,5 +1,5 @@
 WARN ty is pre-release software and not ready for production use. Expect to encounter bugs, missing features, and fatal errors.
-fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(1543f)): execute: too many cycle iterations`
+fatal[panic] Panicked at /home/runner/.cargo/git/checkouts/salsa-e6f3bb7c2a062968/918d35d/src/function/execute.rs:215:25 when checking `/home/runner/work/ruff/ruff/typing/conformance/tests/aliases_typealiastype.py`: `infer_definition_types(Id(6d14)): execute: too many cycle iterations`
 _directives_deprecated_library.py:15:31: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int`
 _directives_deprecated_library.py:30:26: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
 _directives_deprecated_library.py:36:41: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `Self@__add__`

github-actions · 2025-08-03T13:30:24Z

`mypy_primer` results

Changes were detected when running on open source projects

websockets (https://github.com/aaugustin/websockets)
- src/websockets/legacy/server.py:607:34: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `bytes`, found `Unknown | Literal[HTTPStatus.SERVICE_UNAVAILABLE, b"Server is shutting down.\n"] | list[Unknown]`
+ src/websockets/legacy/server.py:607:34: error[invalid-argument-type] Argument to bound method `__init__` is incorrect: Expected `bytes`, found `@Todo(Type::Intersection.call()) | Literal[HTTPStatus.SERVICE_UNAVAILABLE, b"Server is shutting down.\n"] | list[Unknown]`

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/client.py:771:33: warning[possibly-unbound-attribute] Attribute `get` on type `@Todo(Inference of subscript on special form) | under_cached_property[Unknown]` is possibly unbound
+ aiohttp/client.py:771:68: warning[possibly-unbound-attribute] Attribute `get` on type `@Todo(Inference of subscript on special form) | under_cached_property[Unknown]` is possibly unbound
- Found 207 diagnostics
+ Found 209 diagnostics

discord.py (https://github.com/Rapptz/discord.py)
+ discord/utils.py:727:26: error[invalid-await] `T@async_all | Awaitable[T@async_all]` is not awaitable
+ discord/webhook/async_.py:194:37: warning[possibly-unbound-attribute] Attribute `get` on type `Unknown | under_cached_property[Unknown]` is possibly unbound
+ discord/webhook/async_.py:208:36: warning[possibly-unbound-attribute] Attribute `get` on type `Unknown | under_cached_property[Unknown]` is possibly unbound
- Found 549 diagnostics
+ Found 552 diagnostics

prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-sqlalchemy/prefect_sqlalchemy/database.py:333:23: error[invalid-await] `None | CoroutineType[Any, Any, None]` is not awaitable
+ src/prefect/_internal/concurrency/services.py:471:11: error[invalid-await] `(tuple[set[Future[bool]], set[Future[bool]]] & ~tuple[Unknown, ...]) | (Coroutine[Any, Any, tuple[set[Future[bool]], set[Future[bool]]] | None] & ~tuple[Unknown, ...])` is not awaitable
- Found 2969 diagnostics
+ Found 2971 diagnostics

dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/contrib/internal/asyncio/patch.py:61:22: error[invalid-await] `Any | None` is not awaitable
- Found 6476 diagnostics
+ Found 6477 diagnostics

No memory usage changes detected ✅

theammir · 2025-08-03T13:46:13Z

Oops, I think Unknown is not a known Generator where return type is not explicitly specified.
Do we treat Unknown like Any?

sharkdp · 2025-08-04T07:36:55Z

Oops, I think Unknown is not a known Generator where return type is not explicitly specified.
Do we treat Unknown like Any?

Yes, exactly. All dynamic types (Any, Unknown, @Todo types) represented by the Type::Dynamic(…) variant in Rust should be treated as being awaitable. Accessing attributes or calling dunder methods on these type should return the same dynamic type. Looks like this is a pre-existing issue in generator_return_type that should hopefully be easy to fix, but let me know if you need help with that.

theammir · 2025-08-04T11:03:28Z

Do I just...

     fn generator_return_type(self, db: &'db dyn Db) -> Option<Type<'db>> {
         // -- snip --
         match self {
             Type::NominalInstance(instance) => {
                 instance.class.iter_mro(db).find_map(from_class_base)
             }
             Type::ProtocolInstance(instance) => {
                 if let Protocol::FromClass(class) = instance.inner {
                     class.iter_mro(db).find_map(from_class_base)
                 } else {
                     None
                 }
             }
+            ty @ Type::Dynamic(_) => Some(ty),
             _ => None,
         }
     }

It seems to work, but is there more to it?
UPD: now that I think about it, returning Type::unknown() makes way more sense than propagating a dynamic type. Maybe we can get something more specific?

sharkdp · 2025-08-04T12:05:33Z

It seems to work, but is there more to it?

That should be all there is to it 😄. Propagating the dynamic type (instead of returning Unknown) is fine. It's something that we also do elsewhere. The reason is that we want to "bubble up" dynamic types like our Todo types to the outermost layer such that they are user-visible.

theammir · 2025-08-04T16:10:44Z

Alright, now

class InvalidAwaitReturnImplicit:
    def __await__(self):
        pass

is awaitable as far as the type signature goes.

Also, union types don't work. I suppose I have to check every element for being a generator, and construct a union of possible return types if all of them are, in fact, awaitable.

sharkdp · 2025-08-04T18:01:37Z

Alright, now
class InvalidAwaitReturnImplicit:
    def __await__(self):
        pass
is awaitable as far as the type signature goes.

👍

Also, union types don't work. I suppose I have to check every element for being a generator, and construct a union of possible return types if all of them are, in fact, awaitable.

Ah, yes, you're right. Thank you for fixing these. I think you should be able to use UnionType::map to recursively apply generator_return_type to all elements of the union.

theammir · 2025-08-04T20:21:02Z

Now distinguishes between

class AwaitableUnion:  # Generator | Unknown, awaitable
    if datetime.today().weekday() == 6:

        def __await__(self) -> typing.Generator[typing.Any, None, None]:
            yield

    else:

        def __await__(self):
            pass


class UnawaitableUnion:  # Generator | int, non-awaitable
    if datetime.today().weekday() == 6:

        def __await__(self) -> typing.Generator[typing.Any, None, None]:
            yield

    else:

        def __await__(self) -> int:
            return 5

mypy_primer diff looks very promising to me.

sharkdp · 2025-08-04T21:23:19Z

mypy_primer diff looks very promising to me.

Never should be treated exactly the same like dynamic types. Accessing any argument on Never results in Never. So while it may seem strange, Never should be awaitable. This allows us to silence downstream errors if an expression is inferred as Never.

This can happen in unreachable code, for example:

a = returns_awaitable()

if sys.version_info >= (3, 12):
    # the type of `a` is `Never` here when checking on 3.11 or lower
    x = await a
else:
    ...

Now this is obviously a constructed example, but there are valid use cases.

carljm

This is looking pretty good, thank you! Can we add some mdtests (maybe in a new diagnostics/invalid_await.md) demonstrating cases where this diagnostic is emitted, and snapshotting their output?

carljm · 2025-08-05T03:07:27Z

crates/ty_python_semantic/src/types.rs

+        );
+        match self {
+            Self::Call(CallDunderError::CallError(..)) => {
+                // TODO: Indicate definition error, not usage error.


I think it's reasonable to emit a diagnostic at the callsite here. It might also be nice to emit a diagnostic where __await__ is defined with an incorrect signature, but I would see this as in addition, not instead of.

carljm · 2025-08-05T03:10:24Z

crates/ty_python_semantic/src/types.rs

+        match self {
+            Self::Call(CallDunderError::CallError(..)) => {
+                // TODO: Indicate definition error, not usage error.
+                diag.info(
+                    "`__await__` expects additional arguments and cannot be called implicitly",
+                );
+            }
+            Self::Call(CallDunderError::PossiblyUnbound(..)) => {
+                diag.info("`__await__` is possibly unbound");
+            }
+            Self::Call(CallDunderError::MethodNotAvailable) => diag.info("`__await__` is missing"),
+            Self::InvalidReturnType(return_type) => diag.info(format_args!(
+                "`__await__` returns `{return_type}`, which is not a valid iterator",
+                return_type = return_type.display(db)
+            )),
+        }
+    }


Is it possible that in these cases we can highlight the range of the __await__ method, or of the definition of the type that is lacking an __await__ method?

carljm · 2025-08-05T03:10:57Z

crates/ty_python_semantic/src/types/diagnostic.rs

+    ///     await InvalidAwait()  # error: TODO:
+    ///     await 42  # error: TODO:


why are there TODOs here?

Wanted to hear feedback before documenting actual error messages, I think.

I think it's sufficient to include the error code, but not the full message, here. So e.g.

Suggested change

/// await InvalidAwait() # error: TODO:

/// await 42 # error: TODO:

/// await InvalidAwait() # error: [invalid-await]

/// await 42 # error: [invalid-await]

theammir · 2025-08-08T21:09:59Z

Added some useful secondary annotations for the lint. Now report_diagnostic is a huge match-case, but from what I've seen, a lot of ty is, too.

Added mdtests as proposed. Some other tests that rely on await expressions have been broken, for instance:

ruff/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md

Lines 136 to 138 in 3a542a8

    
           # error: [invalid-syntax] "`await` statement outside of a function" 
        
           # error: [invalid-syntax] "`await` outside of an asynchronous function" 
        
           await 1

Would expecting error: [invalid-await] here as well break atomicity?

Even though providing the type definition of a non-awaitable object is useful, sometimes diagnostics peek into built-ins:

error[invalid-await]: `Literal[1]` is not awaitable
   --> test-project/main.py:83:11
    |
 81 |     await Awaitable()  # valid
 82 |     await AwaitableUnion()  # valid
 83 |     await 1  # invalid
    |           ^
 84 |
 85 |     await NonCallableAwait()  # invalid
    |
   ::: stdlib/builtins.pyi:337:7
    |
335 | _LiteralInteger = _PositiveInteger | _NegativeInteger | Literal[0]  # noqa: Y026  # TODO: Use TypeAlias once mypy bugs are fixed
336 |
337 | class int:
    |       --- type defined here
338 |     """int([x]) -> integer
339 |     int(x, base=10) -> integer
    |
info: `__await__` is missing
info: rule `invalid-await` is enabled by default

Also, while fooling around, I discovered that awaiting an instance of this type is perfectly valid, and it isn't considered possibly unbound:

class PossiblyUnbound:
    if datetime.today().weekday() == 0:

        def __await__(self):
            yield 1

    elif datetime.today().weekday() == 1:

        def __await__(self):
            yield 2

I wonder why.

carljm · 2025-08-13T13:45:40Z

Also, while fooling around, I discovered that awaiting an instance of this type is perfectly valid, and it isn't considered possibly unbound

Thanks for this comment! I explored it and found a bug, fixed in #19884

## Summary A [passing comment](#19711 (comment)) led me to explore why we didn't report a class attribute as possibly unbound if it was a method and defined in two different conditional branches. I found that the reason was because of our handling of "conflicting declarations" in `place_from_declarations`. It returned a `Result` which would be `Err` in case of conflicting declarations. But we only actually care about conflicting declarations when we are actually doing type inference on that scope and might emit a diagnostic about it. And in all cases (including that one), we want to otherwise proceed with the union of the declared types, as if there was no conflict. In several cases we were failing to handle the union of declared types in the same way as a normal declared type if there was a declared-types conflict. The `Result` return type made this mistake really easy to make, as we'd match on e.g. `Ok(Place::Type(...))` and do one thing, then match on `Err(...)` and do another, even though really both of those cases should be handled the same. This PR refactors `place_from_declarations` to instead return a struct which always represents the declared type we should use in the same way, as well as carrying the conflicting declared types, if any. This struct has a method to allow us to explicitly ignore the declared-types conflict (which is what we want in most cases), as well as a method to get the declared type and the conflict information, in the case where we want to emit a diagnostic on the conflict. ## Test Plan Existing CI; added a test showing that we now understand a multiply-conditionally-defined method as possibly-unbound. This does trigger issues on a couple new fuzzer seeds, but the issues are just new instances of an already-known (and rarely occurring) problem which I already plan to address in a future PR, so I think it's OK to land as-is. I happened to build this initially on top of #19711, which adds invalid-await diagnostics, so I also updated some invalid-syntax tests to not await on an invalid type, since the purpose of those tests is to check the syntactic location of the `await`, not the validity of the awaited type.

theammir · 2025-08-13T16:42:08Z

Thanks for this comment! I explored it and found a bug, fixed in #19884

Excellent! And thanks for fixing tests for me, I suppose I should merge them from main.

theammir · 2025-08-13T17:30:56Z

All mdtests are now passing.

carljm · 2025-08-14T20:52:39Z

I notice that mypy and pyright both implement these diagnostics by checking that anything you await is assignable to the Awaitable protocol, and then just emitting the usual diagnostics if that assignment fails. (Rather than trying to call __await__ and erroring if we can't make that call, or it returns the wrong type.)

It's not clear to me that using the protocol is a better approach to get good diagnostics here, but it's something we could consider in future. For now I think this PR is good.

carljm · 2025-08-14T21:05:28Z

@theammir Just for future reference (in case you make future PRs), you may want to note some things I'm fixing in this PR:

cargo dev generate-all (because this PR adds a new diagnostic and we need to generate some rules metadata)
Adding  to the mdtest so we get full diagnostic snapshots, then running cargo insta test -p ty_python_semantic followed by cargo insta review to approve those snapshots.
Allowing pre-commit to auto-format the mdtests.
Avoiding let-chains for now, because our minimum supported Rust version (tested in CI) is still 1.86. (Our policy is latest-2, and latest is 1.89, so we can bump that to 1.87, but that won't help, since let-chains were only stabilized in 1.88.)

theammir · 2025-08-14T21:18:02Z

It's not clear to me that using the protocol is a better approach to get good diagnostics here, but it's something we could consider in future. For now I think this PR is good.

My entire reasoning behind the design I went with was trying to build from the blocks available, and that one todo comment in __enter__-related functionality that says that we should check for assignability once we support protocols in the first place.

That said, pyright's "is not awaitable" diagnostics are thorough, but also deeply nested and somewhat convoluted:

Pyright: "InvalidAwaitArgs" is not awaitable
     "InvalidAwaitArgs" is incompatible with protocol "Awaitable[_T_co@Awaitable]"
       "__await__" is an incompatible type
         Type "(value: int) -> Generator[int, Any, None]" is not assignable to type "() -> Generator[Any, Any, _T_co@Awaitable]"
           Extra parameter "value" [reportGeneralTypeIssues]

All the info about specific extra parameters is there (if you are careful enough to find it on line 4, otherwise read even further), and there's more to the protocol error message than just reporting unassignability anyway.
Just thoughts.

Just for future reference (in case you make future PRs), you may want to note some things I'm fixing in this PR:

Thanks! It's been really fun so far, I'd stick around. :)

* main: [ty] Add diagnostics for invalid `await` expressions (#19711) [ty] Synthesize read-only properties for all declared members on `NamedTuple` classes (#19899) [ty] Remove use of `ClassBase::try_from_type` from `super()` machinery (#19902) [ty] Speedup project file discovery (#19913) [`pyflakes`] Add secondary annotation showing previous definition (`F811`) (#19900) Bump 0.12.9 (#19917) [ty] support `kw_only=True` for `dataclass()` and `field()` (#19677)

[ty] Add diagnostics for invalid await expressions

df7a189

theammir requested review from carljm, AlexWaygood, sharkdp and dcreager as code owners August 3, 2025 13:18

MichaReiser added the ty Multi-file analysis & type inference label Aug 3, 2025

fixup! [ty] Add diagnostics for invalid await expressions

b50e2a5

sharkdp self-assigned this Aug 4, 2025

unions

be896c4

AlexWaygood removed their request for review August 4, 2025 20:36

nevah

943b842

carljm reviewed Aug 5, 2025

View reviewed changes

theammir added 3 commits August 8, 2025 23:35

feat: secondary annotations for invalid-await diagnostics

93ebf5c

test: add diagnostics/invalid_await.md

70cc063

chore: remove TODOs

be4cc6e

carljm mentioned this pull request Aug 12, 2025

[ty] simplify return type of place_from_declarations #19884

Merged

theammir added 2 commits August 13, 2025 20:19

Merge remote-tracking branch 'upstream' into feat/await-diagnostics

53171aa

upstream fixes

d131f8a

snapshot diagnostics, tweak a message

a408fa1

cargo dev --generate-all

790a6a3

carljm requested a review from MichaReiser as a code owner August 14, 2025 21:05

don't use let-chains

44e37d3

carljm merged commit 957320c into astral-sh:main Aug 14, 2025
37 checks passed

		/// await InvalidAwait() # error: TODO:
		/// await 42 # error: TODO:

[ty] Add diagnostics for invalid await expressions #19711

[ty] Add diagnostics for invalid await expressions #19711

Uh oh!

Conversation

theammir commented Aug 3, 2025

Summary

Test Plan

Uh oh!

github-actions bot commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Diagnostic diff on typing conformance tests

Uh oh!

github-actions bot commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

theammir commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sharkdp commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theammir commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sharkdp commented Aug 4, 2025

Uh oh!

theammir commented Aug 4, 2025

Uh oh!

sharkdp commented Aug 4, 2025

Uh oh!

theammir commented Aug 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sharkdp commented Aug 4, 2025

Uh oh!

carljm left a comment

Choose a reason for hiding this comment

Uh oh!

carljm Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

carljm Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

carljm Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

theammir Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

carljm Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

theammir commented Aug 8, 2025

Uh oh!

carljm commented Aug 13, 2025

Uh oh!

theammir commented Aug 13, 2025

Uh oh!

theammir commented Aug 13, 2025

Uh oh!

carljm commented Aug 14, 2025

Uh oh!

carljm commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theammir commented Aug 14, 2025

Uh oh!

Uh oh!

Uh oh!

[ty] Add diagnostics for invalid `await` expressions #19711

[ty] Add diagnostics for invalid `await` expressions #19711

github-actions bot commented Aug 3, 2025 •

edited

Loading

github-actions bot commented Aug 3, 2025 •

edited

Loading

`mypy_primer` results

theammir commented Aug 3, 2025 •

edited

Loading

sharkdp commented Aug 4, 2025 •

edited

Loading

theammir commented Aug 4, 2025 •

edited

Loading

theammir commented Aug 4, 2025 •

edited

Loading

carljm commented Aug 14, 2025 •

edited

Loading