Skip to content

PyREPL doesn't correctly color double braces in f-strings / t-strings #134158

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
loic-simon opened this issue May 17, 2025 · 1 comment
Closed
Labels
3.14 bugs and security fixes 3.15 new features, bugs and security fixes stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error

Comments

@loic-simon
Copy link
Contributor

loic-simon commented May 17, 2025

Bug report

Bug description:

Syntax highlighting in PyREPL (added in #131507) does not correctly color the double-braces used for escaping braces in f-strings or t-strings:

Image

After investigation, this is caused by Python tokenizer producing only 1 brace in this case:

>>> from tokenize import tokenize
>>> from io import BytesIO
>>> from pprint import pp
>>> 
>>> pp(list(tokenize(BytesIO(b'f"a{{b"').readline)))
[TokenInfo(type=68 (ENCODING), string='utf-8', start=(0, 0), end=(0, 0), line=''),
TokenInfo(type=59 (FSTRING_START), string='f"', start=(1, 0), end=(1, 2), line='f"a{{b"'),
TokenInfo(type=60 (FSTRING_MIDDLE), string='a{', start=(1, 2), end=(1, 4), line='f"a{{b"'),
TokenInfo(type=60 (FSTRING_MIDDLE), string='b', start=(1, 5), end=(1, 6), line='f"a{{b"'),
TokenInfo(type=61 (FSTRING_END), string='"', start=(1, 6), end=(1, 7), line='f"a{{b"'),
TokenInfo(type=4 (NEWLINE), string='', start=(1, 7), end=(1, 8), line='f"a{{b"'),
TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]

So PyREPL think there is only one brace in the original string, and doesn't color the second one.

I have a fix for this issue, PR on the way!

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Windows, macOS

Linked PRs

@loic-simon loic-simon added the type-bug An unexpected behavior, bug, or error label May 17, 2025
@picnixz picnixz added stdlib Python modules in the Lib dir 3.14 bugs and security fixes topic-repl Related to the interactive shell 3.15 new features, bugs and security fixes labels May 17, 2025
ambv added a commit that referenced this issue May 19, 2025
…4159)

Co-authored-by: Loïc Simon <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 19, 2025
…ythongh-134159)

(cherry picked from commit 71ea6a6)

Co-authored-by: Loïc Simon <[email protected]>
Co-authored-by: Loïc Simon <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
hugovk pushed a commit that referenced this issue May 19, 2025
…gh-134159) (#134227)

Co-authored-by: Loïc Simon <[email protected]>
Co-authored-by: Loïc Simon <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
Co-authored-by: Łukasz Langa <[email protected]>
@hugovk
Copy link
Member

hugovk commented May 19, 2025

Thank you for the report and the fix!

@hugovk hugovk closed this as completed May 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes 3.15 new features, bugs and security fixes stdlib Python modules in the Lib dir topic-repl Related to the interactive shell type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants