AttributeError: 'UnionType' object has no attribute 'type' #3468

zero323 · 2017-05-26T21:31:13Z

This looks like a regression (?) between Mypy 0.501 and 0.510.

Failure is reported at this line: https://github.com/zero323/pyspark-stubs/blob/d25da4d051f878d358bdef6a0e08a6597afbee4f/third_party/3/pyspark/sql/functions.pyi#L24 but it can be reproduced with simplified input:

# Stubs for pyspark.sql.functions (Python 3.5)

from typing import Union
from pyspark.sql.column import Column

ColumnOrName = Union[pyspark.sql.column.Column, str]

def last(col: ColumnOrName, ignorenulls: bool = ...) -> Column: ...

Python version:

python --version
Python 3.5.3 :: Anaconda custom (64-bit)

Mypy version: 0.510, 0.511

Traceback:

third_party/3/pyspark/sql/column.pyi:24: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.511
Traceback (most recent call last):
  File "/home/user/anaconda3/bin/mypy", line 6, in <module>
    main(__file__)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/main.py", line 46, in main
    res = type_check_only(sources, bin_dir, options)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/main.py", line 93, in type_check_only
    options=options)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/build.py", line 188, in build
    graph = dispatch(sources, manager)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/build.py", line 1595, in dispatch
    process_graph(graph, manager)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/build.py", line 1838, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/build.py", line 1937, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/build.py", line 1510, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 177, in check_first_pass
    self.accept(d)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 264, in accept
    stmt.accept(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/nodes.py", line 750, in accept
    return visitor.visit_class_def(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 1080, in visit_class_def
    self.accept(defn.defs)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 264, in accept
    stmt.accept(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/nodes.py", line 811, in accept
    return visitor.visit_block(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 1173, in visit_block
    self.accept(s)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 264, in accept
    stmt.accept(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/nodes.py", line 565, in accept
    return visitor.visit_func_def(self)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 510, in visit_func_def
    self.check_func_item(defn, name=defn.name())
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 569, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 616, in check_func_def
    self.check_reverse_op_method(item, typ, name)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 806, in check_reverse_op_method
    defn)
  File "/home/user/anaconda3/lib/python3.5/site-packages/mypy/checker.py", line 876, in check_overlapping_op_methods
    forward_base.type.name(), forward_name, context)
AttributeError: 'UnionType' object has no attribute 'type'
third_party/3/pyspark/sql/column.pyi:24: note: use --pdb to drop into pdb

The text was updated successfully, but these errors were encountered:

gvanrossum · 2017-05-26T22:33:25Z

Can you come up with a repro that doesn't still depend on the pyspark stubs? I get an error on the import line.

zero323 · 2017-05-26T23:30:51Z

Let me try to dig deeper into this over the weekend. It doesn't appear in isolation, so it guess the problem can be somewhere completely else, and just manifests in an awkward way.

ilevkivskyi · 2017-05-28T09:52:01Z

Here is a simple repro for the crash:

class X:
    def __mod__(self, other: Union[X, str]) -> X: ...
    def __rmod__(self, other: Union[X, str]) -> X: ...

I think the problem is because of an unsafe cast here https://github.com/python/mypy/blob/master/mypy/checker.py#L805

gvanrossum · 2017-05-28T18:57:57Z

Ah, casts! They cause so much trouble and are so misunderstood. That one should have been an assert, but how to prevent crashing?

(PS Crash >= Bug so I removed the bug label.)

ilevkivskyi · 2017-05-30T08:48:18Z

but how to prevent crashing?

TBH I am not sure what to do here. It looks like this crash happens only if Any appears in the special method signature (like in str.__mod__). @JukkaL do you have any ideas?

JukkaL · 2017-05-30T11:02:13Z

I can have a look at this. The relevant code is really old and seems to have bit rotted (assuming that it ever worked).

elazarg · 2017-06-04T13:48:19Z

I think it is related to the issue I have described in #3227

wilfreddenton · 2017-11-28T19:15:02Z

I get the same error when I run mypy on fractions.pyi in the typeshed. Using mypy 0.550.

❯ mypy /Users/wilfred/python-envs/sailfish/lib/mypy/typeshed/stdlib/2and3/fractions.pyi --show-traceback
/Users/wilfred/python-envs/sailfish/lib/mypy/typeshed/stdlib/2and3/fractions.pyi:80: error: INTERNAL ERROR -- please report a bug at https://github.com/pyth
on/mypy/issues version: 0.550                                                                                                                              
Traceback (most recent call last):
  File "/Users/wilfred/python-envs/sailfish/bin/mypy", line 11, in <module>
    sys.exit(console_entry())
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/__main__.py", line 7, in console_entry
    main(None)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/main.py", line 66, in main
    res = type_check_only(sources, bin_dir, options)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/main.py", line 119, in type_check_only
    options=options)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/build.py", line 214, in build
    graph = dispatch(sources, manager)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/build.py", line 1935, in dispatch
    process_graph(graph, manager)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/build.py", line 2199, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/build.py", line 2327, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/build.py", line 1836, in type_check_first_pass
    self.type_checker.check_first_pass()
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 196, in check_first_pass
    self.accept(d)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/nodes.py", line 690, in accept
    return visitor.visit_class_def(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 1185, in visit_class_def
    self.accept(defn.defs)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/nodes.py", line 750, in accept
    return visitor.visit_block(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 1305, in visit_block
    self.accept(s)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 283, in accept
    stmt.accept(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/nodes.py", line 496, in accept
    return visitor.visit_func_def(self)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 531, in visit_func_def
    self.check_func_item(defn, name=defn.name())
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 591, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 641, in check_func_def
    self.check_reverse_op_method(item, typ, name)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 859, in check_reverse_op_method
    defn)
  File "/Users/wilfred/python-envs/sailfish/lib/python3.5/site-packages/mypy/checker.py", line 929, in check_overlapping_op_methods
    forward_base.type.name(), forward_name, context)
AttributeError: 'UnionType' object has no attribute 'type'
/Users/wilfred/python-envs/sailfish/lib/mypy/typeshed/stdlib/2and3/fractions.pyi:80: : note: use --pdb to drop into pdb

However if I run the same command but within my home directory I do not have this issue. I made sure to delete all instances of .mypy_cache.

elazarg · 2017-11-28T19:48:58Z

I can't reproduce this, but there's a slim chance that #4017 will avoid the crash.

elazarg · 2017-11-28T19:55:07Z

@ilevkivskyi in your repro (with the additional from typing import Union), #4017 gives

tmp.py:5: error: Signatures of "__rmod__" of "X" and "__mod__" of "Union[X, str]" are unsafely overlapping

I'm not sure if that's correct, but at least it's not a crash.

wilfreddenton · 2017-11-28T20:16:35Z

I just created a new env and then the errors went away. Not sure what why. I had deleted all instances of .mypy_cache and __pycache__ in the old env before running mypy.

ilevkivskyi · 2017-11-30T07:53:29Z

@elazarg I am not sure this is correct behaviour, IIUC this error is intended for cases where an argument is a superclass (because of some Python logic that binary methods are called on the most derived class), therefore it should probably only apply to Instances. But this code is very old, so @JukkaL knows much more about this.

ilevkivskyi · 2018-02-17T01:04:19Z

The invalid cast I mentioned moved to another line https://github.com/python/mypy/blob/master/mypy/checker.py#L925

elias6 · 2018-02-18T18:45:48Z

Link to the invalid cast that will not go out of date if the content of checker.py changes:

mypy/mypy/checker.py

Line 925 in 46ecea5

typ2, other_method, cast(Instance, arg_type),

elazarg · 2018-02-18T18:56:32Z

I'm not sure the specific invalid cast is the important thing - either #4017 solve it (in which case I believe it should be merged), or the problem is not the cast itself.

JukkaL · 2018-02-19T15:08:30Z

Increased priority as the crash has been reported multiple times.

* Remove old comments, clean bail-out logic * Use fallback to find Instance when possible * Remove cast; move formatting logic to messages.py * Remove unused method * Rename variables to match check_overlapping_op_methods terminology * Remove many unused imports Potential fix to #3468, or at least avoids the crash (since it removes the unsafe cast).

JukkaL · 2018-03-01T17:12:53Z

#4017 hopefully fixed this. Let's reopen if this continues to be a problem.

* Remove old comments, clean bail-out logic * Use fallback to find Instance when possible * Remove cast; move formatting logic to messages.py * Remove unused method * Rename variables to match check_overlapping_op_methods terminology * Remove many unused imports Potential fix to python#3468, or at least avoids the crash (since it removes the unsafe cast).

ilevkivskyi added bug mypy got something wrong crash priority-1-normal labels May 28, 2017

gvanrossum removed the bug mypy got something wrong label May 28, 2017

JukkaL self-assigned this May 30, 2017

elazarg mentioned this issue Nov 28, 2017

Cleanup check_reverse_op_method #4017

Merged

ilevkivskyi added the topic-union-types label Feb 17, 2018

ilevkivskyi mentioned this issue Feb 17, 2018

AttributeError: 'UnionType' object has no attribute 'type' when overloaded arithmetic operation has Union[float, Fraction] type #4581

Closed

JukkaL added priority-0-high and removed priority-1-normal labels Feb 19, 2018

JukkaL closed this as completed in #4017 Mar 1, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AttributeError: 'UnionType' object has no attribute 'type' #3468

AttributeError: 'UnionType' object has no attribute 'type' #3468

zero323 commented May 26, 2017

gvanrossum commented May 26, 2017

zero323 commented May 26, 2017 •

edited

Loading

ilevkivskyi commented May 28, 2017

gvanrossum commented May 28, 2017

ilevkivskyi commented May 30, 2017

JukkaL commented May 30, 2017

elazarg commented Jun 4, 2017

wilfreddenton commented Nov 28, 2017 •

edited

Loading

elazarg commented Nov 28, 2017

elazarg commented Nov 28, 2017 •

edited

Loading

wilfreddenton commented Nov 28, 2017

ilevkivskyi commented Nov 30, 2017

ilevkivskyi commented Feb 17, 2018

elias6 commented Feb 18, 2018 •

edited

Loading

elazarg commented Feb 18, 2018

JukkaL commented Feb 19, 2018

JukkaL commented Mar 1, 2018

AttributeError: 'UnionType' object has no attribute 'type' #3468

AttributeError: 'UnionType' object has no attribute 'type' #3468

Comments

zero323 commented May 26, 2017

gvanrossum commented May 26, 2017

zero323 commented May 26, 2017 • edited Loading

ilevkivskyi commented May 28, 2017

gvanrossum commented May 28, 2017

ilevkivskyi commented May 30, 2017

JukkaL commented May 30, 2017

elazarg commented Jun 4, 2017

wilfreddenton commented Nov 28, 2017 • edited Loading

elazarg commented Nov 28, 2017

elazarg commented Nov 28, 2017 • edited Loading

wilfreddenton commented Nov 28, 2017

ilevkivskyi commented Nov 30, 2017

ilevkivskyi commented Feb 17, 2018

elias6 commented Feb 18, 2018 • edited Loading

elazarg commented Feb 18, 2018

JukkaL commented Feb 19, 2018

JukkaL commented Mar 1, 2018

zero323 commented May 26, 2017 •

edited

Loading

wilfreddenton commented Nov 28, 2017 •

edited

Loading

elazarg commented Nov 28, 2017 •

edited

Loading

elias6 commented Feb 18, 2018 •

edited

Loading