Disallow access to Parsed output, use the API instead
#11741
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dhruvmanila
force-pushed
the
dhruv/checker-comment-ranges
branch
from
adece38 to
73bd3ed
Compare
3 tasks
|
zanieb
approved these changes
Jun 4, 2024
MichaReiser
approved these changes
Jun 5, 2024
dhruvmanila
force-pushed
the
dhruv/parsed-tokens
branch
from
1d129f0 to
f8260cb
Compare
dhruvmanila
added a commit
that referenced
this pull request
Jun 5, 2024
## Summary This PR fixes a bug where the checker would require the tokens for an invalid offset w.r.t. the source code. Taking the source code from the linked issue as an example: ```py relese_version :"0.0is 64" ``` Now, this isn't really a valid type annotation but that's what this PR is fixing. Regardless of whether it's valid or not, Ruff shouldn't panic. The checker would visit the parsed type annotation (`0.0is 64`) and try to detect any violations. Certain rule logic requests the tokens for the same but it would fail because the lexer would only have the `String` token considering original source code. This worked before because the lexer was invoked again for each rule logic. The solution is to store the parsed type annotation on the checker if it's in a typing context and use the tokens from that instead if it's available. This is enforced by creating a new API on the checker to get the tokens. But, this means that there are two ways to get the tokens via the checker API. I want to restrict this in a follow-up PR (#11741) to only expose `tokens` and `comment_ranges` as methods and restrict access to the parsed source code. fixes: #11736 ## Test Plan - [x] Add a test case for `F632` rule and update the snapshot - [x] Check all affected rules - [x] No ecosystem changes
dhruvmanila
force-pushed
the
dhruv/checker-comment-ranges
branch
from
73bd3ed to
86f4019
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR is a follow-up to #11740 to restrict access to the
Parsedoutput by replacing theparsedAPI function with a more specific one. Currently, that iscomment_rangesbut the linked PR exposes atokensmethod.The main motivation is so that there's no way to get an incorrect information from the checker. And, it also encapsulates the source of the comment ranges and the tokens itself. This way it would become easier to just update the checker if the source for these information changes in the future.
Test Plan
cargo insta test