-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
add universaldetector #3734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add universaldetector #3734
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
from .universaldetector import UniversalDetector as UniversalDetector | ||
|
||
def __getattr__(name: str) -> Any: ... # incomplete |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,19 +1,39 @@ | ||
from typing import Dict, Union, AnyStr, Pattern | ||
from typing import Dict, Union, AnyStr, Pattern, Optional | ||
from logging import Logger | ||
|
||
class UniversalDetector: | ||
if sys.version_info >= (3, 8): | ||
from typing import TypedDict | ||
|
||
class FinalResultType(TypedDict): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Since this is a typestub-only type, it should start with an underscore. |
||
encoding: str | ||
confidence: float | ||
language: str | ||
|
||
class IntermediateResultType(TypedDict): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This too. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And I had a doubt. If I have a set of files all of which use a specific There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You can just import it from another file. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Just like python modules? Cool. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @JelleZijlstra Are there any conventions to be followed for the There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name should probably be prefixed with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Got another doubt. Can you help me out? I came across a line like this:
What should be the type of There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @JelleZijlstra I think I found another way to keep the common class definition in one place. Just defined it inside the |
||
encoding: Optional[str] | ||
confidence: float | ||
language: Optional[str] | ||
|
||
class UniversalDetector(object): | ||
MINIMUM_THRESHOLD: float | ||
HIGH_BYTE_DETECTOR: Pattern[bytes] | ||
ESC_DETECTOR: Pattern[bytes] | ||
WIN_BYTE_DETECTOR: Pattern[bytes] | ||
ISO_WIN_MAP: Dict[str, str] | ||
|
||
result: Dict[str, Union[str, float]] | ||
if sys.version_info >= (3, 8): | ||
result: IntermediateResultType | ||
else: | ||
result: Dict[str, Union[str, float]] | ||
done: bool | ||
lang_filter: int | ||
logger: Logger | ||
|
||
def __init__(self, lang_filter: int) -> None: ... | ||
def reset(self) -> None: ... | ||
def feed(self, byte_str: bytes) -> None: ... | ||
def close(self) -> Dict[str, Union[str, float]]: ... | ||
|
||
if sys.version_info >= (3, 8): | ||
def close(self) -> FinalResultType: ... | ||
else: | ||
def close(self) -> Dict[str, Union[str, float]]: ... |
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While
TypedDict
is only available intyping
starting in Python 3.8, you can just import it fromtyping_extensions
for earlier Python versions:Type checkers support this. This way you would avoid the version checks below and make the typed dicts work with all Python versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even easier, you can also just do
from typing_extensions import TypedDict
so you don't need the version check.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for that both of you! So
typing_extensions
has the required info anyway?Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@JelleZijlstra Stubs of some other modules seems to be doing the version check as @srittau mentioned. As in
from stdlib/3/dbm/dumb.pyi
Does this method have any advantage?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No advantage, except that whenever 3.7 support ends, a simple grep for "version_info" will flag this. But we should just grep for typing_extensions when that happens.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean when
typing_extensions
stop supporting 3.7?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, when typeshed stops supporting 3.7 in a few years, we can just do
from typing import Final
in typeshed.