Skip to content

Add informative notes to invariant function arguments #3411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Aug 19, 2017
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
4f6d386
Add notes to invariant argument error
quartox May 22, 2017
c12fc40
Fix type inference
quartox May 22, 2017
2b92465
Add invariant notes to tests
quartox May 22, 2017
05a8ee8
Remove stray comment
quartox May 22, 2017
1bf5a93
Split tests on ' # ' instead of '#'
quartox May 23, 2017
5a70c57
Move variance to new page
quartox May 23, 2017
b0536d5
Update doc url in invariant notes
quartox May 23, 2017
fa4093d
Revert to split on '#' for check tests
quartox May 23, 2017
2888c5f
Add variance to toc
quartox May 23, 2017
f8700c3
Move docs to correct files
quartox May 23, 2017
572160d
Switch order in toc
quartox May 23, 2017
900a927
Revert docs
quartox May 23, 2017
2fee5be
Improve test readability
quartox May 24, 2017
823e1c2
Merge branch 'master' into invariance_notes
quartox May 26, 2017
e22c4d4
Update documentation url
quartox May 26, 2017
add8811
Mapping is covariant in value only
quartox Aug 19, 2017
c31d6e4
Check type fullname
quartox Aug 19, 2017
b938394
Remove unused function
quartox Aug 19, 2017
fbdc381
Add Instance check and DRY
quartox Aug 19, 2017
c2810e3
Check for subtypes
quartox Aug 19, 2017
44a1846
Move Instance check to call site
quartox Aug 19, 2017
bffd7a4
Add more test cases
quartox Aug 19, 2017
7d18a99
Fix tests
quartox Aug 19, 2017
f17d6f6
Fix confusing names test
quartox Aug 19, 2017
35905de
Fix confusing names test and function length
quartox Aug 19, 2017
a9a02d4
Remove extra space between tests
quartox Aug 19, 2017
3599325
Remove extra space from end of line
quartox Aug 19, 2017
de2ede0
Add missing quotes
quartox Aug 19, 2017
bfe20dc
Check that arguments are Instance
quartox Aug 19, 2017
c225c59
Fix error message function name
quartox Aug 19, 2017
739e49f
Use is_same_type to check types
quartox Aug 19, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions mypy/messages.py
Original file line number Diff line number Diff line change
Expand Up @@ -533,6 +533,7 @@ def incompatible_argument(self, n: int, m: int, callee: CallableType, arg_type:
target = 'to {} '.format(name)

msg = ''
notes = [] # type: List[str]
if callee.name == '<list>':
name = callee.name[1:-1]
n -= 1
Expand Down Expand Up @@ -571,7 +572,11 @@ def incompatible_argument(self, n: int, m: int, callee: CallableType, arg_type:
arg_type_str = '**' + arg_type_str
msg = 'Argument {} {}has incompatible type {}; expected {}'.format(
n, target, arg_type_str, expected_type_str)
notes = invariance_notes(notes, arg_type, expected_type)
self.fail(msg, context)
if notes:
for note_msg in notes:
self.note(note_msg, context)

def invalid_index_type(self, index_type: Type, expected_type: Type, base_str: str,
context: Context) -> None:
Expand Down Expand Up @@ -992,6 +997,28 @@ def pretty_or(args: List[str]) -> str:
return ", ".join(quoted[:-1]) + ", or " + quoted[-1]


def invariance_notes(notes: List[str], arg_type: Type, expected_type: Type) -> List[str]:
"""Explain that the type is invariant and give notes for how to solve the issue."""
if isinstance(arg_type, Instance) and isinstance(expected_type, Instance):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I proposed to place this at the call site of this function not at the definition site, to avoid unnecessary calls.
(Also this will save you 4 spaces of indent.)

if (arg_type.type.fullname() == 'builtins.list' and
expected_type.type.fullname() == 'builtins.list'):
invariant_type = 'List'
covariant_suggestion = 'Consider using "Sequence" instead, which is covariant'
elif (arg_type.type.fullname() == 'builtins.dict' and
expected_type.type.fullname() == 'builtins.dict' and
arg_type.args[0].type == expected_type.args[0].type and
arg_type.args[1].type != expected_type.args[1].type):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is something important here missing (also for List): we should show the note only when using a covariant type will actually help, for example here:

x: List[int]
y: List[str]
x = y

we don't need to show any notes, since using Sequence will not help. So that instead arg_type.args[1].type != expected_type.args[1].type use if subtypes.is_subtype(arg_type.args[1], expected_type.args[1]).

invariant_type = 'Dict'
covariant_suggestion = ('Consider using "Mapping" instead, '
'which is covariant in the value')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"... covariant in the value type", if don't agree with a previous comment #3411 (comment), then please explain, just ignoring it is not a good option, I will not forget :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just missed it, thanks for catching.

Copy link
Contributor Author

@quartox quartox Aug 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am also not convinced that this message is clear to a novice user (which I expect is the intended audience for this type of note), but I can't think of a better concise message. I guess this assumes that the link is the most helpful part.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...at least value type suggests this is something related to types, not with the value itself (note that this error might appear in situations where list/dict literals are involved).

if invariant_type:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will crash, since invariant_type is not always defined.

notes.append(
'"{}" is invariant --- see '.format(invariant_type) +
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I proposed two dashes here instead of three.

'http://mypy.readthedocs.io/en/latest/common_issues.html#variance')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DRY, this should be

VARIANCE_LINK = '"{}" is blah-blah...'
...
notes.append(VARIANCE_LINK.format('List'))
...
notes.append(VARIANCE_LINK.format('Dict'))

Also please use two dashes -- not three.

notes.append(covariant_suggestion)
return notes


def make_inferred_type_note(context: Context, subtype: Type,
supertype: Type, supertype_str: str) -> str:
"""Explain that the user may have forgotten to type a variable.
Expand Down
8 changes: 6 additions & 2 deletions test-data/unit/check-inference.test
Original file line number Diff line number Diff line change
Expand Up @@ -684,7 +684,9 @@ g('a')() # E: List[str] not callable
# to backtrack later) and defaults to T = <nothing>. The result is an
# awkward error message. Either a better error message, or simply accepting the
# call, would be preferable here.
g(['a']) # E: Argument 1 to "g" has incompatible type List[str]; expected List[<nothing>]
g(['a']) # E: Argument 1 to "g" has incompatible type List[str]; expected List[<nothing>] \
# N: "List" is invariant --- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance \
# N: Consider using "Sequence" instead, which is covariant

h(g(['a']))

Expand All @@ -693,7 +695,9 @@ a = [1]
b = ['b']
i(a, a, b)
i(b, a, b)
i(a, b, b) # E: Argument 1 to "i" has incompatible type List[int]; expected List[str]
i(a, b, b) # E: Argument 1 to "i" has incompatible type List[int]; expected List[str] \
# N: "List" is invariant --- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance \
# N: Consider using "Sequence" instead, which is covariant
[builtins fixtures/list.pyi]

[case testCallableListJoinInference]
Expand Down
4 changes: 3 additions & 1 deletion test-data/unit/check-overloading.test
Original file line number Diff line number Diff line change
Expand Up @@ -610,7 +610,9 @@ n = 1
m = 1
n = 'x' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
m = 'x' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
f(list_object) # E: Argument 1 to "f" has incompatible type List[object]; expected List[int]
f(list_object) # E: Argument 1 to "f" has incompatible type List[object]; expected List[int] \
# N: "List" is invariant --- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance \
# N: Consider using "Sequence" instead, which is covariant
[builtins fixtures/list.pyi]

[case testOverlappingOverloadSignatures]
Expand Down
20 changes: 20 additions & 0 deletions test-data/unit/check-varargs.test
Original file line number Diff line number Diff line change
Expand Up @@ -592,3 +592,23 @@ class C:
def foo(self) -> None: pass
C().foo()
C().foo(1) # The decorator's return type says this should be okay


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One empty line between tests is enough (also below).

[case testInvariantDictArgNote]
from typing import Dict, Sequence
def f(a: Dict[str, Sequence[int]]) -> None: pass
a = {'a': [1, 2]}
f(a) # E: Argument 1 to "f" has incompatible type Dict[str, List[int]]; expected Dict[str, Sequence[int]] \
# N: "Dict" is invariant --- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance \
# N: Consider using "Mapping" instead, which is covariant in the value
[builtins fixtures/dict.pyi]


[case testInvariantListArgNote]
from typing import List, Union
def f(numbers: List[Union[int, float]]) -> None: pass
a = [1, 2]
f(a) # E: Argument 1 to "f" has incompatible type List[int]; expected List[Union[int, float]] \
# N: "List" is invariant --- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance \
# N: Consider using "Sequence" instead, which is covariant
[builtins fixtures/list.pyi]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests:

  • for tricky names (like DictReader)
  • for unrelated list types (like List[str] vs List[int]), see my comment above
  • for situations where covariance will help, and where it will not (like Dict[str, float] vs Dict[str, int] and vice-versa).

In general, it is a good idea to add tests for all situations that appeared in the discussion of a PR.