More precise return types for open(), Path.open(), bz2.open(), etc. #3371
Conversation
|
Thank you, that is very useful! Without doing a full review yet, I think it would make sense to consolidate the |
|
Do you have any suggestion on where to put these constants? When consolidating the constants in I fixed that issue in 0335f3f by defining these constants in |
|
I'd probably put them into |
|
@srittau Actually what about |
|
Good idea! |
|
Moved to |
|
There's some merge conflicts and a CI failure, could you take a look? |
|
ilai-deutel/mypy@b5f8f01 fixes the |
|
Thanks! We'll need to make simultaneous merges in both repos. I can do that in a few days. |
Prepare for merging python/typeshed#3371 and the accompanying mypy PR.
Prepare for merging python/typeshed#3371 and the accompanying mypy PR.
There was a problem hiding this comment.
This is impressive work! I only checked the Python 3 builtins.open() stubs, they look correct to me.
I wonder if you considered making it return the actual types, instead of the IO protocol? Specifically, the open() documentation describes the return type as follows:
The type of file object returned by the open() function depends on the mode. When open() is used to open a file in a text mode ('w', 'r', 'wt', 'rt', etc.), it returns a subclass of io.TextIOBase (specifically io.TextIOWrapper). When used to open a file in a binary mode with buffering, the returned class is a subclass of io.BufferedIOBase. The exact class varies: in read binary mode, it returns an io.BufferedReader; in write binary and append binary modes, it returns an io.BufferedWriter, and in read/write mode, it returns an io.BufferedRandom. When buffering is disabled, the raw stream, a subclass of io.RawIOBase, io.FileIO, is returned.
By my count, encoding it fully would require 5(!) overloads, but for such an important function it might be worthwhile.
|
@bluetech For instance, import io
from click.testing import EchoingStdin
f = open('a', 'rb')
g = open('b', 'wb')
EchoingStdin(input=f, output=g)However, if See also #1229 Perhaps these concrete classes should inherit from |
|
@ilai-deutel Right.
I tried it. There are several issues, fixable except one: Anyway, this is probably already a separate discussion from this PR. Trying to switch to concrete types can be done after this PR is merged. |
|
Sorry for letting this slide! There's a merge conflict now. |
…shed into ilai-deutel-overloaded-open
python/typeshed#3371 causes this to produce a different error message. open() with no arguments doesn't seem like an important use case, so testing for it explicitly in pythoneval isn't all that useful.
|
I fixed the merge conflict locally (https://github.com/JelleZijlstra/typeshed/tree/ilai-deutel-overloaded-open) but it looks like I don't have permission to push to the PR branch. I'll probably just open a separate PR from this branch. I also submitted python/mypy#8908 to remove the failing test from mypy, since it doesn't seem useful. |
No worries!
You should now have write access to my branch, but feel free to open a separate PR with your existing branch if it's easier |
Didn't work unfortunately. |
|
Looks like it's a GitHub issue, I get a javascript error ( I pushed your merge commits to this branch |
python/typeshed#3371 causes this to produce a different error message. open() with no arguments doesn't seem like an important use case, so testing for it explicitly in pythoneval isn't all that useful.
| encoding: Optional[str] = ..., | ||
| errors: Optional[str] = ..., | ||
| newline: Optional[str] = ..., | ||
| ) -> Union[BZ2File, TextIO]: ... |
There was a problem hiding this comment.
Union return types are usually problematic, since they force the use to assert or cast the return value, even if the user knows the return type. In this particular case I actually like the returned Union, since the last overload is only used if mode is dynamic, so type checking at runtime makes sence.
|
CI is green now after I removed the mypy test, so merging this won't require any complications. |
Co-authored-by: Sebastian Rittau <[email protected]>
|
🎉 |
…ython#3371) Co-authored-by: Jelle Zijlstra <[email protected]> Co-authored-by: Sebastian Rittau <[email protected]>
This PR adds overloaded signatures for
{builtin,pathlib.Path,bz2,gwip,lzma}.open()functions, usingLiteralmodes to determine the return types.mypytests pass after updating theopenandPath.openplugins and making a few changes to the expected outputs: python/mypy#7794Fixes #2911
Related discussion: python/mypy#7643