Tests behave differently without builtins fixtures

[pkch]

I was somewhat concerned that test builtins fixtures might not perfectly represent the production version of mypy, so I ran the tests without the fixtures.

The good news is that, as expected, fixtures make tests at least twice faster (pytest alone takes 212 sec without the fixtures, and 12 sec with).

The bad news is that 60 out of 2309 tests failed. Many of the failures were trivial (a slight change in the error message since the test was written, or not defining a type variables because it's already defined in the fixture). But a few were more serious.

For example this code (from check-class-namedtuple.test: testNewNamedTupleJoinTuple), instead of reporting the two revealed types, crashed mypy (under python 3.6):

class X(NamedTuple):
    x: int
    y: str

reveal_type([(3, 'b'), X(1, 'a')])  # E: Revealed type is 'builtins.list[Tuple[builtins.int, builtins.str]]'
reveal_type([X(1, 'a'), (3, 'b')])  # E: Revealed type is 'builtins.list[Tuple[builtins.int, builtins.str]]'

This code (from check-namedtuple.test: testNamedTupleMake) didn't report the second expected error message:

X = NamedTuple('X', [('x', int), ('y', str)])
reveal_type(X._make([5, 'a']))  # E: Revealed type is 'Tuple[builtins.int, builtins.str, fallback=__main__.X]'
X._make('a b')  # E: Argument 1 to X._make has incompatible type "str"; expected Iterable[Any]

Not sure what the best solution is. Maybe it's worth having a daily run of all tests without fixtures using some special commit.

In any case, I made a branch that allows to easily run tests without fixtures because I couldn't find a command line parameter that would do it. I'll push a PR for it, although it's probably more appropriate for testing and discussion, not for merging.

[pkch]

Edit: output updated now that I disabled [file builtins.py] in addition to [builtins ...].

And this is the output of pytest without fixtures:

FFFFFFFF..FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF.FF...FF.................s..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................s...............................................................................ss.........................s..........................................................................................................................................................ss.........s.................................................................s.....................................................................................................s.......................................................................................................................................................................................................................................................s.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................s............................................................................................................................................................................................................s.....................s..........................................s...................................s....................................................................
================================== FAILURES ===================================
_________________________ testGenericClassWith__new__ _________________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test:1739:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test, line 1739)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:10: error: Argument 1 to "set" of "C" has incompatible type "int"; expected "str" (diff)
Actual:
  main:5: error: Argument 1 to "__new__" of "object" has incompatible type C[T]; expected "object" (diff)
  main:10: error: Argument 1 to "set" of "C" has incompatible type "int"; expected "str" (diff)

Alignment of first line difference:
  E: main:10: error: Argument 1 to "set" of "C" has incompatible type "int"; ...
  A: main:5: error: Argument 1 to "__new__" of "object" has incompatible type...
          ^
______________________ testTypeEquivalentTypeAnyEdgeCase ______________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test:2248:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test, line 2248)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:10: error: Argument 1 to "__init__" of "type" has incompatible type "C"; expected "type" (diff)
Actual:
  main:10: error: No overload variant of "__init__" of "type" matches argument types [__main__.C, Any] (diff)

Alignment of first line difference:
  E: main:10: error: Argument 1 to "__init__" of "type" has incompatible type...
  A: main:10: error: No overload variant of "__init__" of "type" matches argu...
                     ^
______________________ testUnionMultipleNonCallableTypes ______________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test:55:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test, line 55)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:8: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:8: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
__________________________ testGetAttributeSignature __________________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test:1628:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test, line 1628)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Invalid signature "def (__main__.B, __main__.A) -> __main__....
  main:6: error: Invalid signature "def (__main__.C, builtins.str, builtins.str) -> __main__.C" (diff)
Actual:
  main:4: error: Invalid signature "def (__main__.B, __main__.A) -> __main__....
  main:4: error: Argument 1 of "__getattribute__" incompatible with supertype "object" (diff)
  main:6: error: Invalid signature "def (__main__.C, builtins.str, builtins.str) -> __main__.C" (diff)
  main:6: error: Signature of "__getattribute__" incompatible with supertype "object" (diff)

Alignment of first line difference:
  E: main:6: error: Invalid signature "def (__main__.C, builtins.str, builtin...
  A: main:4: error: Argument 1 of "__getattribute__" incompatible with supert...
          ^
________________________ testUnionMultipleReturnTypes _________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test:43:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test, line 43)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:6: error: Unsupported operand types for + ("Union[str, int]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[str, int]" and "int")
                                    ^
_______________________________ testCallableAnd _______________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test:148:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-callable.test, line 148)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:8: error: Unsupported left operand type for + (some union) (diff)
Actual:
  main:8: error: Unsupported operand types for + ("Union[int, Callable[[], str]]" and "int") (diff)

Alignment of first line difference:
  E: main:8: error: Unsupported left operand type for + (some union)...
  A: main:8: error: Unsupported operand types for + ("Union[int, Callable[[],...
                                ^
___________________________ testBuiltinClassMethod ____________________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test:782:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test, line 782)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Argument 1 to "from_bytes" of "int" has incompatible type "str"; expected "bytes" (diff)
Actual:
  main:3: error: Argument 1 to "from_bytes" of "int" has incompatible type "str"; expected Sequence[int] (diff)

Alignment of first line difference:
  E: ...patible type "str"; expected "bytes"
  A: ...patible type "str"; expected Sequence[int]
                                     ^
___________________________ testBuiltinStaticMethod ___________________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test:752:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-classes.test, line 752)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Argument 1 to "from_bytes" of "int" has incompatible type "str"; expected "bytes" (diff)
Actual:
  main:3: error: Argument 1 to "from_bytes" of "int" has incompatible type "str"; expected Sequence[int] (diff)

Alignment of first line difference:
  E: ...patible type "str"; expected "bytes"
  A: ...patible type "str"; expected Sequence[int]
                                     ^
________________________ testUnificationRedundantUnion ________________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-inference.test:1722:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-inference.test, line 1722)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:7: error: Argument 1 to "g" has incompatible type "Union[int, str, tuple]"; expected "Union[int, str]" (diff)
Actual:
  main:7: error: Argument 1 to "g" has incompatible type "Union[int, str, Tuple[Any, ...]]"; expected "Union[int, str]" (diff)

Alignment of first line difference:
  E: ...type "Union[int, str, tuple]"; expected "Union[int, str]"
  A: ...type "Union[int, str, Tuple[Any, ...]]"; expected "Union[int, str]"
                              ^
__________________________ testTypeApplicationCrash ___________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test:507:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test, line 507)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:1: error: The type "type" is not generic and not indexable (diff)
Actual:
  main:1: error: Value of type "type" is not indexable (diff)

Alignment of first line difference:
  E: main:1: error: The type "type" is not generic and not indexable
  A: main:1: error: Value of type "type" is not indexable
                    ^
__________________ testMultipleAssignmentWithListAndIndexing __________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test:1155:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test, line 1155)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:5: error: Incompatible types in assignment (expression has type "A", target has type "int") (diff)
Actual:
  main:5: error: No overload variant of "__setitem__" of "list" matches argument types [builtins.int, __main__.A*] (diff)

Alignment of first line difference:
  E: main:5: error: Incompatible types in assignment (expression has type "A"...
  A: main:5: error: No overload variant of "__setitem__" of "list" matches ar...
                    ^
_________________________ testGenericTypeAliasesUnion _________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test:748:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test, line 748)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:10: error: Unsupported left operand type for + (some union) (diff)
  main:16: error: Incompatible types in assignment (expression has type "str"...
  main:24: error: Revealed type is 'Union[builtins.int, __main__.Node[builtin...
  main:27: error: Incompatible types in assignment (expression has type "int"...
Actual:
  main:10: error: Unsupported operand types for + ("Union[int, Node[int]]" and "int") (diff)
  main:16: error: Incompatible types in assignment (expression has type "str"...
  main:24: error: Revealed type is 'Union[builtins.int, __main__.Node[builtin...
  main:27: error: Incompatible types in assignment (expression has type "int"...

Alignment of first line difference:
  E: main:10: error: Unsupported left operand type for + (some union)...
  A: main:10: error: Unsupported operand types for + ("Union[int, Node[int]]"...
                                 ^
______________________________ testTryExceptFlow ______________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-statements.test:562:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-statements.test, line 562)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:7: error: Unsupported left operand type for + ("int") (diff)
Actual:
  main:7: error: Unsupported operand types for + ("int" and "str") (diff)

Alignment of first line difference:
  E: main:7: error: Unsupported left operand type for + ("int")
  A: main:7: error: Unsupported operand types for + ("int" and "str")
                                ^
________________ testGenericTypeAliasesRuntimeExpressionsOther ________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test:911:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-generics.test, line 911)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:8: error: Unsupported left operand type for + ("Type alias to Callable") (diff)
  main:9: error: Revealed type is 'Any'         (diff)
  main:11: error: "Type alias to Tuple" not callable
  main:12: error: Revealed type is 'Any'
  main:14: error: "Type alias to Union" has no attribute "x"
  main:15: error: Revealed type is 'Any'
Actual:
  main:8: error: Unsupported operand types for + ("Type alias to Callable" and "int") (diff)
  main:9: error: Revealed type is 'builtins.int' (diff)
  main:11: error: "Type alias to Tuple" not callable
  main:12: error: Revealed type is 'Any'
  main:14: error: "Type alias to Union" has no attribute "x"
  main:15: error: Revealed type is 'Any'

Alignment of first line difference:
  E: main:8: error: Unsupported left operand type for + ("Type alias to Calla...
  A: main:8: error: Unsupported operand types for + ("Type alias to Callable"...
                                ^
_____________________ testConditionalTypeAliasPY3_python2 _____________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test:21:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test, line 21)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:9: error: "str" has no attribute "__div__" (diff)
Actual:
  main:9: error: Argument 1 to "__rdiv__" of "int" has incompatible type "str"; expected "int" (diff)

Alignment of first line difference:
  E: main:9: error: "str" has no attribute "__div__"...
  A: main:9: error: Argument 1 to "__rdiv__" of "int" has incompatible type "...
                    ^
______________________ testStubImportNonStubWhileSilent _______________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test:953:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test, line 953)
---------------------------- Captured stderr call -----------------------------
Expected:
  tmp/main.py:3: error: Unsupported left operand type for + ("int") (diff)
Actual:
  tmp/main.py:3: error: Unsupported operand types for + ("int" and "str") (diff)

Alignment of first line difference:
  E: ...y:3: error: Unsupported left operand type for + ("int")
  A: ...y:3: error: Unsupported operand types for + ("int" and "str")
                                ^
________________________ testConditionalAssignmentPY2 _________________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test:34:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test, line 34)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:7: error: Unsupported left operand type for / ("str") (diff)
Actual:
  main:7: error: Unsupported operand types for / ("str" and "int") (diff)

Alignment of first line difference:
  E: main:7: error: Unsupported left operand type for / ("str")
  A: main:7: error: Unsupported operand types for / ("str" and "int")
                                ^
______________________________ testSuperWithNew _______________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-super.test:81:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-super.test, line 81)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:8: error: Too many arguments for "__new__" of "A" (diff)
  main:9: error: Argument 1 to "B" has incompatible type "str"; expected "int" (diff)
Actual:
  main:3: error: Argument 1 to "__new__" of "object" has incompatible type "A"; expected "object" (diff)
  main:8: error: Too many arguments for "__new__" of "A" (diff)
  main:9: error: Argument 1 to "B" has incompatible type "str"; expected "int" (diff)

Alignment of first line difference:
  E: main:8: error: Too many arguments for "__new__" of "A"...
  A: main:3: error: Argument 1 to "__new__" of "object" has incompatible type...
          ^
__________________________ testTypeCheckNamedModule2 __________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test:677:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test, line 677)
---------------------------- Captured stderr call -----------------------------
Expected:
  tmp/m/a.py:1: error: Unsupported left operand type for + (None) (diff)
Actual:
  (empty)

__________________________ testTypeCheckNamedModule ___________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test:669:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test, line 669)
---------------------------- Captured stderr call -----------------------------
Expected:
  tmp/m/__init__.py:1: error: Unsupported left operand type for + (None) (diff)
Actual:
  (empty)

__________________________ testTypeCheckNamedModule5 __________________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test:700:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test, line 700)
---------------------------- Captured stderr call -----------------------------
Expected:
  tmp/m.py:1: error: Unsupported left operand type for + (None) (diff)
Actual:
  (empty)

__________________________ testTypeCheckNamedModule3 __________________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test:685:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-modules.test, line 685)
---------------------------- Captured stderr call -----------------------------
Expected:
  tmp/m/__init__.py:1: error: Unsupported left operand type for + (None) (diff)
Actual:
  (empty)

____________________________ testModifyRepeatedly _____________________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:734:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 734)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported operand types for + (likely involving Union) (diff)
  main:7: error: Unsupported operand types for + (likely involving Union)
  main:11: error: Unsupported operand types for + ("int" and "str")
  main:14: error: Unsupported operand types for + ("str" and "int")
  main:18: error: Unsupported operand types for + (likely involving Union) (diff)
  main:19: error: Unsupported operand types for + (likely involving Union)
Actual:
  main:6: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:7: error: Unsupported operand types for + (likely involving Union)
  main:11: error: Unsupported operand types for + ("int" and "str")
  main:14: error: Unsupported operand types for + ("str" and "int")
  main:18: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:19: error: Unsupported operand types for + (likely involving Union)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
________________________ testIsInstanceSubClassMember _________________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:486:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 486)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:21: error: Unsupported operand types for + (likely involving Union) (diff)
  main:22: error: Unsupported operand types for + (likely involving Union)
  main:27: error: Unsupported operand types for + ("int" and "str")
Actual:
  main:21: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:22: error: Unsupported operand types for + (likely involving Union)
  main:27: error: Unsupported operand types for + ("int" and "str")

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[int, str]" and "int")
                                     ^
_____________________________ testMypyConditional _____________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test:83:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unreachable-code.test, line 83)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Unsupported left operand type for + (None) (diff)
Actual:
  (empty)

______________________________ testUnionIndexing ______________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unions.test:89:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-unions.test, line 89)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:4: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
_______________________________ testModifyLoop ________________________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:757:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 757)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported operand types for + (likely involving Union) (diff)
  main:8: error: Unsupported operand types for + ("str" and "int")
  main:13: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:6: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:8: error: Unsupported operand types for + ("str" and "int")
  main:13: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
_____________________________ testIsInstanceBasic _____________________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:459:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 459)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported operand types for + ("str" and "int")
  main:9: error: Unsupported operand types for + ("int" and "str") (diff)
Actual:
  main:6: error: Unsupported operand types for + ("str" and "int")
  main:7: error: Unsupported operand types for + ("int" and "str") (diff)
  main:9: error: Unsupported operand types for + ("int" and "str") (diff)

Alignment of first line difference:
  E: main:9: error: Unsupported operand types for + ("int" and "str")
  A: main:7: error: Unsupported operand types for + ("int" and "str")
          ^
___________________________ testUnionListIsinstance ___________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:383:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 383)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:16: error: Unsupported operand types for + ("int" and "str")
  main:19: error: Unsupported operand types for + (likely involving Union) (diff)
  main:21: error: Value of type "int" is not indexable
  main:23: error: Value of type "Union[List[int], List[str], int]" is not ind...
  main:24: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:16: error: Unsupported operand types for + ("int" and "str")
  main:19: error: Unsupported operand types for + ("Union[List[int], List[str]]" and "int") (diff)
  main:21: error: Value of type "int" is not indexable
  main:23: error: Value of type "Union[List[int], List[str], int]" is not ind...
  main:24: error: Unsupported operand types for + ("Union[List[int], List[str], int]" and "int") (diff)

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[List[int], List[str]]" and "int")
                                     ^
__________________________ testUnionStrictDefnBasic ___________________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:429:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 429)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:9: error: Unsupported operand types for + (likely involving Union) (diff)
  main:11: error: Unsupported operand types for + ("str" and "int") (diff)
Actual:
  main:9: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
________________________ testIsInstanceWithUnknownType ________________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:1365:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 1365)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Unsupported operand types for + (likely involving Union) (diff)
  main:5: error: Revealed type is 'Union[builtins.int, builtins.str]'
  main:7: error: Revealed type is 'builtins.str'
Actual:
  main:4: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:5: error: Revealed type is 'Union[builtins.int, builtins.str]'
  main:7: error: Revealed type is 'builtins.str'

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
_____________________________ testModifyLoopLong ______________________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:904:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 904)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:9: error: Unsupported left operand type for + (some union) (diff)
  main:15: error: Unsupported operand types for + ("int" and "str")
  main:18: error: "str" has no attribute "a"
  main:42: error: Some element of union has no attribute "a"
Actual:
  main:9: error: Unsupported operand types for + ("Union[int, str, A]" and "int") (diff)
  main:15: error: Unsupported operand types for + ("int" and "str")
  main:18: error: "str" has no attribute "a"
  main:42: error: Some element of union has no attribute "a"

Alignment of first line difference:
  E: main:9: error: Unsupported left operand type for + (some union)...
  A: main:9: error: Unsupported operand types for + ("Union[int, str, A]" and...
                                ^
_______________________________ testModifyLoop2 _______________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:774:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 774)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported operand types for + (likely involving Union) (diff)
  main:8: error: Unsupported operand types for + ("str" and "int")
  main:15: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:6: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:8: error: Unsupported operand types for + ("str" and "int")
  main:15: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...ported operand types for + (likely involving Union)
  A: ...ported operand types for + ("Union[int, str]" and "int")
                                    ^
_______________________________ testModifyLoop3 _______________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:794:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 794)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:14: error: Unsupported operand types for + (likely involving Union) (diff)
  main:22: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:14: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:22: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[int, str]" and "int")
                                     ^
_____________________________ testNamedTupleMake ______________________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-namedtuple.test:284:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-namedtuple.test, line 284)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Revealed type is 'Tuple[builtins.int, builtins.str, fallback...
  main:5: error: Argument 1 to X._make has incompatible type "str"; expected Iterable[Any] (diff)
Actual:
  main:4: error: Revealed type is 'Tuple[builtins.int, builtins.str, fallback...

____________________________ testModifyNestedLoop _____________________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:875:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 875)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:16: error: Unsupported operand types for + (likely involving Union) (diff)
  main:26: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:16: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:26: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[int, str]" and "int")
                                     ^
____________________________ testModifyLoopWhile4 _____________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:819:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 819)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:19: error: Unsupported operand types for + (likely involving Union) (diff)
  main:24: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:19: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:24: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[int, str]" and "int")
                                     ^
_____________________________ testModifyLoopFor4 ______________________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test:847:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-isinstance.test, line 847)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:19: error: Unsupported operand types for + (likely involving Union) (diff)
  main:24: error: Unsupported operand types for + (likely involving Union) (diff)
Actual:
  main:19: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)
  main:24: error: Unsupported operand types for + ("Union[int, str]" and "int") (diff)

Alignment of first line difference:
  E: ...pported operand types for + (likely involving Union)
  A: ...pported operand types for + ("Union[int, str]" and "int")
                                     ^
___________________ testInferOptionalFromDefaultNoneComment ___________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test:134:
..\..\mypy\test\testcheck.py:105: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test, line 134)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Unsupported left operand type for + (some union) (diff)
Actual:
  main:3: error: Unsupported operand types for + ("Optional[int]" and "int") (diff)

Alignment of first line difference:
  E: main:3: error: Unsupported left operand type for + (some union)...
  A: main:3: error: Unsupported operand types for + ("Optional[int]" and "int...
                                ^
___________________________ testNamedTupleJoinTuple ___________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-namedtuple.test:329:
SystemExit: 1
---------------------------- Captured stdout call -----------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 7, in <module>
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1534, in serve
    SlaveGateway(io=io, id=id, _startcount=2).serve()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1047, in serve
    self._execpool.integrate_as_primary_thread()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 259, in integrate_as_primary_thread
    self._perform_spawn(reply)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 277, in _perform_spawn
    reply.run()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 213, in run
    self._result = func(*args, **kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1072, in executetask
    do_exec(co, loc)  # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 159, in <module>
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 127, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 98, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 133, in _main
    config.hook.pytest_runtestloop(session=session)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "<remote exec>", line 61, in pytest_runtestloop
  File "<remote exec>", line 77, in run_tests
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 66, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 79, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 133, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 151, in call_runtest_hook
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 163, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 151, in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 104, in pytest_runtest_call
    item.runtest()
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\data.py", line 481, in runtest
    self.parent.obj(update_data=update_data).run_case(self.obj)
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\testcheck.py", line 111, in run_case
    self.run_case_once(testcase)
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\testcheck.py", line 167, in run_case_once
    alt_lib_path=test_temp_dir)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 189, in build
    dispatch(sources, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1560, in dispatch
    process_graph(graph, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1802, in process_graph
    process_stale_scc(graph, scc, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1901, in process_stale_scc
    graph[id].type_check_first_pass()
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1475, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 174, in check_first_pass
    self.accept(d)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 240, in accept
    stmt.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 822, in accept
    return visitor.visit_expression_stmt(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 1793, in visit_expression_stmt
    self.expr_checker.accept(s.expr, allow_none_return=True)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2040, in accept
    typ = self.visit_call_expr(node, allow_none_return=True)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 176, in visit_call_expr
    return self.accept(e.analyzed, self.type_context[-1])
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2044, in accept
    typ = node.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 1509, in accept
    return visitor.visit_reveal_type_expr(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1538, in visit_reveal_type_expr
    revealed_type = self.accept(expr.expr, type_context=self.type_context[-1])
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2044, in accept
    typ = node.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 1552, in accept
    return visitor.visit_list_expr(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1621, in visit_list_expr
    return self.check_lst_expr(e.items, 'builtins.list', '<list>', e)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1647, in check_lst_expr
    context)[0]
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 377, in check_call
    callee, args, arg_kinds, formal_to_actual, context)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 609, in infer_function_type_arguments
    strict=self.chk.in_checked_function())  # type: List[Type]
  File "C:\Users\maxmo\Downloads\mypy\mypy\infer.py", line 35, in infer_function_type_arguments
    return solve_constraints(type_vars, constraints, strict)
  File "C:\Users\maxmo\Downloads\mypy\mypy\solve.py", line 46, in solve_constraints
    bottom = join_types(bottom, c.target)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 90, in join_types
    return t.accept(TypeJoinVisitor(s))
  File "C:\Users\maxmo\Downloads\mypy\mypy\types.py", line 852, in accept
    return visitor.visit_tuple_type(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 225, in visit_tuple_type
    fallback = join_instances(self.s.fallback, t.fallback)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 304, in join_instances
    return join_instances_via_supertype(s, t)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 320, in join_instances_via_supertype
    res = join_instances(mapped, s)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 293, in join_instances
    args.append(join_types(t.args[i], s.args[i]))
IndexError: list index out of range
---------------------------- Captured stderr call -----------------------------
main:4: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues
main:4: note: use --pdb to drop into pdb
_______________ testInferOptionalFromDefaultNoneWithFastParser ________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test:127:
..\..\mypy\test\testcheck.py:105: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test, line 127)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Unsupported left operand type for + (some union) (diff)
Actual:
  main:3: error: Unsupported operand types for + ("Optional[int]" and "int") (diff)

Alignment of first line difference:
  E: main:3: error: Unsupported left operand type for + (some union)...
  A: main:3: error: Unsupported operand types for + ("Optional[int]" and "int...
                                ^
___________________________ testTypeAliasInBuiltins ___________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-type-aliases.test:59:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-type-aliases.test, line 59)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Argument 1 to "f" has incompatible type "int"; expected "str" (diff)
Actual:
  main:3: error: Argument 1 to "f" has incompatible type "int"; expected "bytes" (diff)

Alignment of first line difference:
  E: ...type "int"; expected "str"
  A: ...type "int"; expected "bytes"
                              ^
____________ testInferOptionalFromDefaultNoneCommentWithFastParser ____________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test:141:
..\..\mypy\test\testcheck.py:105: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test, line 141)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Unsupported left operand type for + (some union) (diff)
Actual:
  main:4: error: Unsupported operand types for + ("Optional[int]" and "int") (diff)

Alignment of first line difference:
  E: main:4: error: Unsupported left operand type for + (some union)...
  A: main:4: error: Unsupported operand types for + ("Optional[int]" and "int...
                                ^
______________________ testInferOptionalFromDefaultNone _______________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test:121:
..\..\mypy\test\testcheck.py:105: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-optional.test, line 121)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:2: error: Unsupported left operand type for + (some union) (diff)
Actual:
  main:2: error: Unsupported operand types for + ("Optional[int]" and "int") (diff)

Alignment of first line difference:
  E: main:2: error: Unsupported left operand type for + (some union)...
  A: main:2: error: Unsupported operand types for + ("Optional[int]" and "int...
                                ^
________________________ testNoReturnImportFromTyping _________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-flags.test:196:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-flags.test, line 196)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:5: error: Return statement in function which does not return (diff)
  main:7: error: Return statement in function which does not return (diff)
  main:13: error: Incompatible types in assignment (expression has type "int", variable has type NoReturn) (diff)
Actual:
  main:1: error: Module 'typing' has no attribute 'NoReturn' (diff)

Alignment of first line difference:
  E: main:5: error: Return statement in function which does not return
  A: main:1: error: Module 'typing' has no attribute 'NoReturn'
          ^
______________________________ testAsyncForError ______________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-async-await.test:144:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-async-await.test, line 144)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: AsyncIterable expected
  main:4: error: List[int] has no attribute "__aiter__" (diff)
Actual:
  main:4: error: AsyncIterable expected
  main:4: error: List[int] has no attribute "__aiter__"; maybe "__iter__"? (diff)

Alignment of first line difference:
  E: ...o attribute "__aiter__"
  A: ...o attribute "__aiter__"; maybe "__iter__"?
                               ^
______________________________ testTupleMethods _______________________________
[gw6] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-tuples.test:589:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-tuples.test, line 589)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:7: error: Incompatible types in assignment (expression has type "int",...
  main:8: error: Incompatible types in assignment (expression has type "str",...
  main:9: error: Incompatible types in assignment (expression has type "bool", variable has type "int") (diff)
  main:10: error: "Tuple[int, str]" has no attribute "foo" (diff)
Actual:
  main:7: error: Incompatible types in assignment (expression has type "int",...
  main:8: error: Incompatible types in assignment (expression has type "str",...
  main:10: error: "Tuple[int, str]" has no attribute "foo" (diff)

Alignment of first line difference:
  E: main:9: error: Incompatible types in assignment (expression has type "bo...
  A: main:10: error: "Tuple[int, str]" has no attribute "foo"...
          ^
___________________________ testTupleErrorMessages ____________________________
[gw5] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-tuples.test:558:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-tuples.test, line 558)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:4: error: Unsupported left operand type for + ("Tuple[A, A]") (diff)
  main:5: error: Unsupported operand types for + ("A" and "Tuple[A, A]")
  main:6: error: Argument 1 to "f" has incompatible type "Tuple[A, A]"; expec...
  main:7: error: "Tuple[A, A]" has no attribute "foo"
Actual:
  main:4: error: Unsupported operand types for + ("tuple" and "A") (diff)
  main:5: error: Unsupported operand types for + ("A" and "Tuple[A, A]")
  main:6: error: Argument 1 to "f" has incompatible type "Tuple[A, A]"; expec...
  main:7: error: "Tuple[A, A]" has no attribute "foo"

Alignment of first line difference:
  E: main:4: error: Unsupported left operand type for + ("Tuple[A, A]")
  A: main:4: error: Unsupported operand types for + ("tuple" and "A")
                                ^
_________________________ testNewNamedTupleJoinTuple __________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-class-namedtuple.test:328:
SystemExit: 1
---------------------------- Captured stdout call -----------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<string>", line 7, in <module>
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1534, in serve
    SlaveGateway(io=io, id=id, _startcount=2).serve()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1047, in serve
    self._execpool.integrate_as_primary_thread()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 259, in integrate_as_primary_thread
    self._perform_spawn(reply)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 277, in _perform_spawn
    reply.run()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 213, in run
    self._result = func(*args, **kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\execnet\gateway_base.py", line 1072, in executetask
    do_exec(co, loc)  # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 159, in <module>
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 127, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 98, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\main.py", line 133, in _main
    config.hook.pytest_runtestloop(session=session)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "<remote exec>", line 61, in pytest_runtestloop
  File "<remote exec>", line 77, in run_tests
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 66, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 79, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 133, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 151, in call_runtest_hook
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 163, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 151, in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 613, in execute
    return _wrapped_call(hook_impl.function(*args), self.execute)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 248, in _wrapped_call
    call_outcome = _CallOutcome(func)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\vendored_packages\pluggy.py", line 614, in execute
    res = hook_impl.function(*args)
  File "c:\users\maxmo\appdata\local\programs\python\python36\lib\site-packages\_pytest\runner.py", line 104, in pytest_runtest_call
    item.runtest()
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\data.py", line 481, in runtest
    self.parent.obj(update_data=update_data).run_case(self.obj)
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\testcheck.py", line 111, in run_case
    self.run_case_once(testcase)
  File "C:\Users\maxmo\Downloads\mypy\mypy\test\testcheck.py", line 167, in run_case_once
    alt_lib_path=test_temp_dir)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 189, in build
    dispatch(sources, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1560, in dispatch
    process_graph(graph, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1802, in process_graph
    process_stale_scc(graph, scc, manager)
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1901, in process_stale_scc
    graph[id].type_check_first_pass()
  File "C:\Users\maxmo\Downloads\mypy\mypy\build.py", line 1475, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 174, in check_first_pass
    self.accept(d)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 240, in accept
    stmt.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 822, in accept
    return visitor.visit_expression_stmt(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checker.py", line 1793, in visit_expression_stmt
    self.expr_checker.accept(s.expr, allow_none_return=True)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2040, in accept
    typ = self.visit_call_expr(node, allow_none_return=True)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 176, in visit_call_expr
    return self.accept(e.analyzed, self.type_context[-1])
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2044, in accept
    typ = node.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 1509, in accept
    return visitor.visit_reveal_type_expr(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1538, in visit_reveal_type_expr
    revealed_type = self.accept(expr.expr, type_context=self.type_context[-1])
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 2044, in accept
    typ = node.accept(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\nodes.py", line 1552, in accept
    return visitor.visit_list_expr(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1621, in visit_list_expr
    return self.check_lst_expr(e.items, 'builtins.list', '<list>', e)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 1647, in check_lst_expr
    context)[0]
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 377, in check_call
    callee, args, arg_kinds, formal_to_actual, context)
  File "C:\Users\maxmo\Downloads\mypy\mypy\checkexpr.py", line 609, in infer_function_type_arguments
    strict=self.chk.in_checked_function())  # type: List[Type]
  File "C:\Users\maxmo\Downloads\mypy\mypy\infer.py", line 35, in infer_function_type_arguments
    return solve_constraints(type_vars, constraints, strict)
  File "C:\Users\maxmo\Downloads\mypy\mypy\solve.py", line 46, in solve_constraints
    bottom = join_types(bottom, c.target)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 90, in join_types
    return t.accept(TypeJoinVisitor(s))
  File "C:\Users\maxmo\Downloads\mypy\mypy\types.py", line 852, in accept
    return visitor.visit_tuple_type(self)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 225, in visit_tuple_type
    fallback = join_instances(self.s.fallback, t.fallback)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 304, in join_instances
    return join_instances_via_supertype(s, t)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 320, in join_instances_via_supertype
    res = join_instances(mapped, s)
  File "C:\Users\maxmo\Downloads\mypy\mypy\join.py", line 293, in join_instances
    args.append(join_types(t.args[i], s.args[i]))
IndexError: list index out of range
---------------------------- Captured stderr call -----------------------------
main:8: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues
main:8: note: use --pdb to drop into pdb
_______________________________ testCmp_python2 _______________________________
[gw7] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test:454:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test, line 454)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:3: error: Unsupported operand types for == ("A" and "A")
  main:4: error: Argument 1 to "__cmp__" of "A" has incompatible type "A"; expected "B" (diff)
  main:9: error: Argument 1 to "__cmp__" of "B" has incompatible type "C"; expected "B" (diff)
  main:12: error: Argument 1 to "__cmp__" of "C" has incompatible type "B"; expected "A" (diff)
  main:14: error: Unsupported operand types for == ("C" and "B") (diff)
Actual:
  main:3: error: Unsupported operand types for == ("A" and "A")
  main:9: error: Argument 1 to "__cmp__" of "B" has incompatible type "C"; expected "B" (diff)
  main:12: error: Argument 1 to "__cmp__" of "C" has incompatible type "B"; expected "A" (diff)
  main:20: error: Argument 1 of "__eq__" incompatible with supertype "object" (diff)
  main:34: error: Argument 1 of "__eq__" incompatible with supertype "object" (diff)

Alignment of first line difference:
  E: main:4: error: Argument 1 to "__cmp__" of "A" has incompatible type "A";...
  A: main:9: error: Argument 1 to "__cmp__" of "B" has incompatible type "C";...
          ^
_____________________ testSubtypingFunctionsImplicitNames _____________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test:90:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test, line 90)
---------------------------- Captured stderr call -----------------------------
Expected:
Actual:
  main:3: error: Name 'Any' is not defined      (diff)

____________________________ testRedefineFunction _____________________________
[gw4] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test:1569:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test, line 1569)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Incompatible types in assignment (expression has type Callable[[Arg('y', Any)], Any], variable has type Callable[[Arg('x', Any)], Any]) (diff)
  main:7: error: Incompatible types in assignment (expression has type Callable[[Any, Any], Any], variable has type Callable[[Any], Any]) (diff)
Actual:
  main:1: error: Name 'Any' is not defined      (diff)
  main:4: error: Name 'Any' is not defined      (diff)
  main:6: error: Incompatible types in assignment (expression has type Callable[[Arg('y', Any)], Any], variable has type Callable[[Arg('x', Any)], Any]) (diff)
  main:7: error: Incompatible types in assignment (expression has type Callable[[Any, Any], Any], variable has type Callable[[Any], Any]) (diff)

Alignment of first line difference:
  E: main:6: error: Incompatible types in assignment (expression has type Cal...
  A: main:1: error: Name 'Any' is not defined...
          ^
_____________________________ testQuickAndDirty3 ______________________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-incremental.test:1944:
..\..\mypy\test\testcheck.py:101: in run_case
    self.run_case_once(testcase, 2)
..\..\mypy\test\testcheck.py:197: in run_case_once
    res.manager.rechecked_modules)
..\..\mypy\test\testcheck.py:209: in check_module_equivalence
    'Set of {} modules does not match expected set'.format(name))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Set of rechecked modules does not match expected set
---------------------------- Captured stderr call -----------------------------
Expected:
  a                                             (diff)
  b                                             (diff)
  builtins                                      (diff)
Actual:
  _importlib_modulespec                         (diff)
  a                                             (diff)
  abc                                           (diff)
  b                                             (diff)
  builtins                                      (diff)
  mypy_extensions                               (diff)
  sys                                           (diff)
  types                                         (diff)
  typing                                        (diff)

_______________________ testBytesInterpolationBefore35 ________________________
[gw1] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test:1062:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test, line 1062)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:2: error: Unsupported left operand type for % ("bytes") (diff)
Actual:
  main:2: error: Unsupported operand types for % ("bytes" and "int") (diff)

Alignment of first line difference:
  E: main:2: error: Unsupported left operand type for % ("bytes")
  A: main:2: error: Unsupported operand types for % ("bytes" and "int")
                                ^
____________________________ testChainedCompResTyp ____________________________
[gw0] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test:562:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-expressions.test, line 562)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:5: error: Incompatible types in assignment (expression has type "B", v...
  main:7: error: Incompatible types in assignment (expression has type "P", v...
Actual:
  main:5: error: Incompatible types in assignment (expression has type "B", v...
  main:7: error: Incompatible types in assignment (expression has type "P", v...
  main:23: error: Argument 1 of "__eq__" incompatible with supertype "object" (diff)
  main:23: error: Return type of "__eq__" incompatible with supertype "object" (diff)

_________________________ testNewSyntaxWithRevealType _________________________
[gw2] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-newsyntax.test:36:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-newsyntax.test, line 36)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:8: error: Revealed type is 'builtins.dict[builtins.str*, builtins.int]' (diff)
Actual:
  main:4: error: Name 'T' is not defined        (diff)
  main:5: error: Name 'T' is not defined        (diff)
  main:8: error: Revealed type is 'builtins.dict[Any, builtins.int]' (diff)

Alignment of first line difference:
  E: main:8: error: Revealed type is 'builtins.dict[builtins.str*, builtins.i...
  A: main:4: error: Name 'T' is not defined...
          ^
_______________ testCallableWithArbitraryArgsInGenericFunction ________________
[gw3] win32 -- Python 3.6.0 c:\users\maxmo\appdata\local\programs\python\python36\python.exe
data: C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test:1428:
..\..\mypy\test\testcheck.py:111: in run_case
    self.run_case_once(testcase)
..\..\mypy\test\testcheck.py:188: in run_case_once
    assert_string_arrays_equal(output, a, msg.format(testcase.file, testcase.line))
..\..\mypy\test\helpers.py:85: in assert_string_arrays_equal
    raise AssertionFailure(msg)
E   mypy.myunit.AssertionFailure: Invalid type checker output (C:\Users\maxmo\Downloads\mypy\test-data\unit\check-functions.test, line 1428)
---------------------------- Captured stderr call -----------------------------
Expected:
  main:6: error: Unsupported left operand type for + ("str") (diff)
Actual:
  main:6: error: Unsupported operand types for + ("str" and "int") (diff)

Alignment of first line difference:
  E: main:6: error: Unsupported left operand type for + ("str")
  A: main:6: error: Unsupported operand types for + ("str" and "int")
                                ^
============ 57 failed, 2252 passed, 16 skipped in 252.23 seconds =============

[JukkaL]

Thanks for looking into this! Would you like to create issues just for the crashes, as we should definitely investigate them?

[ilevkivskyi]

I could immediately guess the cause for the crash in normalize_type_alias: lib-stub/typing.pyi has all of List, Set, Dict, but some fixtures for builtins don't have list, or set, or dict. Then there is a crash, since normalize_type_alias does not check whether list was defined, it just translates List to list etc. This is why the crash happens on unknown node.

[pkch]

@JukkaL I did; luckily, only one real crash (well 2 tests crash for the same reason; and 4 tests crash for the reason @ilevkivskyi said)

@ilevkivskyi Actually, I just didn't notice that [file builtins.py] is another way to effectively make a fixture. I've disabled it now, so no such crashes occur. I updated my comment with the test output log with the new output.

[pkch]

So there are 4 reasons tests may fail without builtins fixture:

1. A problem with mypy that was hidden by its custom stub; I created an issue for each such problem, and linked it here.
2. The test uses incorrect text for the error messages (but all error messages are on the same lines). For example, Mypy gives different message for Callable and other types (although perhaps Overloaded should be treated the same as Callable). And since custom stub has type defined just once, it's seen as Callable, unlike wiht the real builtins.pyi where it's defined several times and hence is Overloaded.
3. The test relies on its custom stub to provide it extra names (e.g., TypeVar('T') defined inside stub, instead of inside the test).
4. The test is incorrect (usually because it makes an incorrect assumption about what should / shouldn't type check) but the mistake is hidden by its custom stub.

My suggestions for each type of test:

1. Fix the issues linked to here (there are only 5 open ones)
2. Identify and ignore those cases. It's easy to identify automatically, we can just look for tests that fail with mypy.myunit.AssertionFailure: Invalid type checker output, but in Captured stderr call have the same line numbers of expected and actual error messages
3. Fix the test (usually trivial)
4. Fix the test (slightly less trivial, but usually still fast)

[pkch]

I forgot the rest of my proposal:

To make sure that without custom builtins stubs the tests would actually fail in cases 1), 3), 4), and pass in case 2) , we can:

• add an option to the unit tests to ignore differences between expected and actual messages as long as the line numbers perfectly match; this option would be on by default
• have default runtests.py still use custom builtins stubs (for faster local tests), but change travis.yml (not appveyor.yml) to disable custom builtins stubs (it adds ~3 min to the tests)

I can modify my #3112 to do that, if this approach seems desirable.

Alternatively, if we really want Travis CI to finish fast, we can create a new branch release, and whenever master is updated, automatically run slow tests and automatically push --force the tip from master into release if those tests pass.

[JukkaL]

Closing due to lack of activity.