Crash with forward reference in NamedTuple

[JelleZijlstra]

Found this testing my production codebase against mypy master:

(venv) jelle@devjelle:~/mypy$ cat testnt.py
from typing import NamedTuple

class A(NamedTuple):
    b: 'B'
    x: int

class B:
    pass
(venv) jelle@devjelle:~/mypy$ python -m mypy --show-traceback testnt.py 
testnt.py:3: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.510-dev-09e7b124b82dae1207d9eac6805b64a98290015c
Traceback (most recent call last):
  File "/home/jelle/ans/venv64/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/jelle/ans/venv64/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/__main__.py", line 5, in <module>
    main(None)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/main.py", line 46, in main
    res = type_check_only(sources, bin_dir, options)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/main.py", line 93, in type_check_only
    options=options)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/build.py", line 188, in build
    graph = dispatch(sources, manager)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/build.py", line 1595, in dispatch
    process_graph(graph, manager)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/build.py", line 1838, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/build.py", line 1931, in process_stale_scc
    graph[id].semantic_analysis()
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/build.py", line 1495, in semantic_analysis
    self.manager.semantic_analyzer.visit_file(self.tree, self.xpath, self.options)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 257, in visit_file
    self.accept(d)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 3279, in accept
    node.accept(self)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/nodes.py", line 749, in accept
    return visitor.visit_class_def(self)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 617, in visit_class_def
    with self.analyze_class_body(defn) as should_continue:
  File "/home/jelle/ans/venv64/lib/python3.6/contextlib.py", line 82, in __enter__
    return next(self.gen)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 629, in analyze_class_body
    if self.analyze_namedtuple_classdef(defn):
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 819, in analyze_namedtuple_classdef
    defn.name, items, types, default_items)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/semanal.py", line 2053, in build_namedtuple_typeinfo
    fallback = self.named_type('__builtins__.tuple', [join.join_type_list(types)])
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/join.py", line 388, in join_type_list
    joined = join_types(joined, t)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/join.py", line 88, in join_types
    return t.accept(TypeJoinVisitor(s))
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/types.py", line 411, in accept
    return visitor.visit_instance(self)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/join.py", line 141, in visit_instance
    return join_instances(t, self.s)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/join.py", line 295, in join_instances
    return join_instances_via_supertype(s, t)
  File "/home/jelle/mypy/venv/lib/python3.6/site-packages/mypy/join.py", line 314, in join_instances_via_supertype
    assert best is not None
AssertionError: 
testnt.py:3: note: use --pdb to drop into pdb

Haven't tried yet to minimize this further. Hopefully we can get this fixed before the release.

Edit: this also repros with TypedDict:

from mypy_extensions import TypedDict

X = TypedDict('X', {'b': 'B', 'c': int})

class B: pass

[JelleZijlstra]

Reverting #3305 fixes it (cc @ilevkivskyi).

[gvanrossum]

Thanks for testing! We should get to this one way or another before the release. @JukkaL.

[ilevkivskyi]

Edit: this also repros with TypedDict:

My PR surfaced the bug already present in TypedDict, we need to postpone the join in fallback instance argument until third pass, since MROs are not yet calculated. I will take a look at this now.

[ilevkivskyi]

OK, I have fixed these two crashes and also three crashes from #3308 (for both class and functional syntax, total nine crash scenarios). I will now add tests and make a PR.

The point is that "synthetic" types were sometimes ignored in the third pass, now they are correctly processed.

[JukkaL]

I'm working on a quick fix that just uses Any in the fallback for named tuples. The TypedDict issue is also present in the previous release, I think, so it's less urgent. (Also TypedDict is not even documented yet.)

[JukkaL]

Created a separate issue for the TypedDict crash so that these can be tracked independently (#3319).