[mohmedmm]

Fixes #1443.

strip_ipython_syntax previously stripped lines starting with ! or % indiscriminately, breaking multiline strings whose interior lines contain magic-like commands or LaTeX comments. We track multiline string line intervals via standard library okenize so string contents are left untouched while real cell magics and shell escapes are still neutralized.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses an issue where the strip_ipython_syntax utility would incorrectly strip lines starting with '!' or '%' inside multiline strings. By leveraging Python's tokenize library, the utility now accurately identifies the boundaries of multiline strings and preserves their contents, while maintaining the existing functionality of neutralizing actual IPython cell magics and shell escapes.

Highlights

• Preservation of multiline strings: Updated the IPython syntax stripping logic to use the tokenize module to identify multiline strings, ensuring that lines within these strings are not incorrectly modified even if they contain magic-like characters.
• Improved robustness: Added handling for unterminated multiline strings during tokenization to prevent false positives or crashes.
• New test suite: Added tools/test_nblint_syntax.py to verify the fix and ensure standard IPython magics and shell escapes are still correctly neutralized.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request updates the syntax linter to avoid stripping IPython magics and shell escapes when they appear inside multiline strings, using Python's tokenize module. It also introduces a comprehensive suite of unit tests. The feedback suggests improving the robustness of the tokenization logic by handling Python 3.12+ f-string tokens, refining the TokenError exception handling to prevent false positives on mismatched parentheses, and adding a test case specifically for multiline f-strings.

[BichengWang]

Reviewed 00a0ed0c2af816fc771bfaf677edb98610ed75f9. The ordinary-string and f-string regressions both fail against the base and pass here; all five syntax tests pass on Python 3.13, 3.14, and 3.15. The amended head also addresses the earlier f-string tokenization and TokenError feedback. I ran the 18 existing link-checker tests and compared syntax validation across all 154 notebooks against the base, with no new failures. Validation was offline; I did not execute notebook kernels or call the API. No blocking findings.

[kkorpal]

Thanks @mohmedmm, the fix is right. Verified: the #1443 reproducer passes, both regression tests fail on main and pass here, magics in the same cell as a multiline string are still neutralized, and a full nblint run over all notebooks is unchanged (93 warnings, 0 failures, same as main). Thanks @BichengWang for the cross-check on 3.13–3.15 matches what I see.

One thing before merging: #1434 (also touching syntax.py) creates the same new file tools/test_nblint_syntax.py, so the two PRs conflict (add/add). Since #1434 is further along and wires that file into CI, could you rebase onto it once it lands and add your tests to the existing file? Everything else can stay as is.

[mohmedmm]

Sounds good, I will rebase and merge the tests into ools/test_nblint_syntax.py as soon as #1434 lands.

[Giom-V]

Thanks for the fix @mohmedmm! I removed the unit test file from this PR so we don't have to maintain a separate unit test suite for internal helper scripts. Merging!