Add signature for dataclasses.replace

[ikonst]

Validate dataclassses.replace actual arguments to match the fields:

• Unlike __init__, the arguments are always named.
• All arguments are optional except for InitVars without a default value.

The tricks:

• We're looking up type of the first positional argument ("obj") through private API. See Adjusting the signature of a function on the basis of caller types in plugin #10216, Let function signature hook have context of caller #14845.
• We're preparing the signature of "replace" (for that specific dataclass) during the dataclass transformation and storing it in a "private" class attribute __mypy-replace (obviously not part of PEP-557 but contains a hyphen so should not conflict with any future valid identifier). Stashing the signature into the symbol table allows it to be passed across phases and cached across invocations. The stashed signature lacks the first argument, which we prepend at function signature hook time, since it depends on the type that replace is called on.

Based on #14526 but actually simpler.
Partially addresses #5152.

Remaining tasks

• handle generic dataclasses
• avoid data class transforms
• fine-grained mode tests

[ikonst]

I wish I could've deserialized metadata, or even accessed a list of DataclassAttributes that's already been deserialized...

Or maybe the right way to go is to replace dataclasses.replace with an OverloadedFuncDef that the transform (which has full context) will keep adding to.

[ikonst]

The OverloadedFuncDef appears to be a non-starter since by the time I can do anything to replace dataclasses.replace with a synthesized OverloadedFuncDef, the call expression has been resolved with the pointer to the generic FunctionDef.

[ikonst]

OK, solved it by creating a "secret" symbol with a signature at semantic analysis time, then using this signature in the function sig callback.

[ikonst]

@ilevkivskyi care to take a look?

[ikonst]

👋 @AlexWaygood

[AlexWaygood]

I'm just a triager over at mypy, so I'm afraid I don't have merge privileges! Here's a quick comment though:

[AlexWaygood]

This is pretty different to typeshed's stub for replace. Would it be worth adding a pythoneval test as well, to check that this machinery all works with the full typeshed stubs for the stdlib, outside the context of mypy's test suite?

[ikonst]

The stub has:

def replace(__obj: _DataclassT, **changes: Any) -> _DataclassT: ...

I can rename obj to __obj in this fixture. We can also change the stub to better reflect reality:

if sys.version_info >= (3, 9):
    def replace(obj: _DataclassT, /, **changes: Any) -> _DataclassT: ...
else:
   def replace(obj: _DataclassT, **changes: Any) -> _DataclassT: ...

Can we just use the Python 3.9 syntax even for Python 3.8 targets? Passing obj as kw is deprecated in Python 3.8 so being a checker/linter we might as well just disallow it, no?

Do you think it matters much? For the plugin to function, replace has to:

1. exist
2. have a first argument

The **changes are optional, the retval is overwritten, and FWIW it doesn't look for a keyword arg obj since it's deprecated since Python 3.8 and was never a good idea to begin with, so we might as well just not support it, right?

[AlexWaygood]

We can also change the stub to better reflect reality:

No need: prepending the parameter name with __ is the py37-compatible way of declaring that a parameter is positional-only, as is described in PEP 484: https://peps.python.org/pep-0484/#positional-only-arguments. So the obj parameter is already marked as positional-only on all Python versions in typeshed's stubs.

I don't think you need to change anything in your existing fixtures/tests for this PR, I'm just suggesting adding a single additional test in the pythoneval.test file. Unlike mypy's other tests, the tests in that file don't use fixtures — they run with typeshed's full stdlib stubs. So it's a good sanity check to make sure that the machinery you've added works with typeshed's stubs for this function as well as with mypy's fixtures.

[ikonst]

7b907cf

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

vision (https://github.com/pytorch/vision) got 1.42x slower (29.2s -> 41.3s)

[ikonst]

@ilevkivskyi can take another look?

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅

[ikonst]

@ilevkivskyi can take another look?

[ilevkivskyi]

@ikonst Please note there is now a merge conflict. You can also address couple remaining comments before I merge this.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

graphql-core (https://github.com/graphql-python/graphql-core): typechecking got 1.06x slower (281.6s -> 297.4s)
(Performance measurements are based on a single noisy sample)