Skip to content

add __fspath__ support to os.path #2053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2018

Conversation

JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented Apr 14, 2018

Fixes #1997, #2068.

This is tricky because we need to get the return values right (see #1960 for
prior attempts) and we often run into python/mypy#3644. I found that I
could express most signatures correctly using a series of overloads.

A few other changes in here:

I couldn't come up with a correct signature for os.path.commonpath, so I am falling back to using Any as its return type.

I used the following test program to make sure mypy accepted the signatures used here:

import os
from os.path import abspath, commonpath, relpath, split, join
from typing import Generic, AnyStr, Any

class PL(os.PathLike[AnyStr]):
    def __init__(self, path: AnyStr) -> None:
        self.path: AnyStr = path
    def __fspath__(self) -> AnyStr:
        return self.path

bad_path: PL[Any] = PL('')

reveal_type(abspath('hello'))  # str
reveal_type(abspath(b'hello'))  # bytes
reveal_type(abspath(PL('hello')))  # str
reveal_type(abspath(PL(b'hello')))  # bytes

reveal_type(commonpath(['hello', PL('hello')]))  # Any (couldn't get it work correctly)
reveal_type(commonpath([b'hello', PL(b'hello')]))  # Any

reveal_type(relpath('hello'))  # str
reveal_type(relpath('hello', PL('hello')))  # str
reveal_type(relpath(b'hello', b'hello'))  # bytes
reveal_type(relpath(PL('hello')))  # str
reveal_type(relpath(PL(b'hello'), b'hello'))  # bytes
reveal_type(relpath(bad_path, 'hello'))  # str
reveal_type(relpath(bad_path, bad_path))  # Any

reveal_type(split('hello'))  # (str, str)
reveal_type(split(b'hello'))  # (bytes, bytes)
reveal_type(split(PL('hello')))  # (str, str)
reveal_type(split(PL(b'hello')))  # (bytes, bytes)

reveal_type(join('hello', PL('hello')))  # str
reveal_type(join(PL('hello'), 'hello'))  # str
reveal_type(join(b'hello', PL(b'hello')))  # bytes
reveal_type(join(b'hello', PL('hello')))  # error

I made the same changes to the Python 2 stub in order to keep the two consistent.

Fixes python#1997.

This is tricky because we need to get the return values right (see python#1960 for
prior attempts) and we often run into python/mypy#3644. I found that I
could express most signatures correctly using a series of overloads.

A few other changes in here:
- Added splitunc, which according to https://docs.python.org/3/library/os.path.html
  should exist in both Unix and Windows.
- Made the second argument to os.path.curdir Optional to match the implementation.
- Fixed os.path.split, whose previous Path-aware signature triggered python/mypy#3644.

I used the following test program to make sure mypy accepted the signatures used here:
@overload
def relpath(path: _StrPath, start: Optional[_StrPath] = ...) -> Text: ... # type: ignore
@overload
def relpath(path: _BytesPath, start: _BytesPath) -> bytes: ...
Copy link
Contributor

@FichteFoll FichteFoll Apr 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

relpath isn't guarded by a 3.6 version check and there is no AnyStr fallback.

Also, the bytes variant is missing Optional.

(Applies to the 2 version as well.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is intentional. _StrPath is just Text below 3.6, so this implementation is correct there.

If start is None, the code uses os.curdir, which is Text, so the path also has to be Text. Although come to think of it, we may need something more complicated in Python 2, where you can mix bytes and Text paths.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, the current code already works in Python 2 (modulo a mypy bug where some overloads silently fall back to Any).

Copy link
Member

@gvanrossum gvanrossum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, merge whenever you want to.

@JelleZijlstra JelleZijlstra merged commit 55be42f into python:master May 15, 2018
@JelleZijlstra JelleZijlstra deleted the ospathfspath branch May 15, 2018 14:31
@JelleZijlstra
Copy link
Member Author

Thanks for the review!

gwk pushed a commit to gwk/typeshed that referenced this pull request May 29, 2018
Fixes python#1997, python#2068.

This is tricky because we need to get the return values right (see python#1960 for
prior attempts) and we often run into python/mypy#3644. I found that I
could express most signatures correctly using a series of overloads.

A few other changes in here:
- Added splitunc, which according to https://docs.python.org/3/library/os.path.html
  should exist in both Unix and Windows.
- Made the second argument to os.path.curdir Optional to match the implementation.
- Fixed os.path.split, whose previous Path-aware signature triggered python/mypy#3644.
yedpodtrzitko pushed a commit to yedpodtrzitko/typeshed that referenced this pull request Jan 23, 2019
Fixes python#1997, python#2068.

This is tricky because we need to get the return values right (see python#1960 for
prior attempts) and we often run into python/mypy#3644. I found that I
could express most signatures correctly using a series of overloads.

A few other changes in here:
- Added splitunc, which according to https://docs.python.org/3/library/os.path.html
  should exist in both Unix and Windows.
- Made the second argument to os.path.curdir Optional to match the implementation.
- Fixed os.path.split, whose previous Path-aware signature triggered python/mypy#3644.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants