Skip to content

[Parser] Do not eagerly lex numbers #6544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 26, 2024
Merged

[Parser] Do not eagerly lex numbers #6544

merged 1 commit into from
Apr 26, 2024

Conversation

tlively
Copy link
Member

@tlively tlively commented Apr 25, 2024

Lex integers and floats on demand to avoid wasted work. Remove Token
completely now that all kinds of tokens are lexed on demand.

Copy link
Member Author

tlively commented Apr 25, 2024

@tlively
Copy link
Member Author

tlively commented Apr 25, 2024

This stack of PRs brings us from being ~8% slower than the old parser to being ~30% faster 🎉

}
return getIndex();
}
size_t getPos() const { return index; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe worth renaming index to pos or vice versa?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we actually also have a getIndex method that does the same thing. Now that they are the same, it would be good to clean this up. I'll take a look at it as a follow-up.

@tlively tlively force-pushed the parser-no-tok-string branch from 7c48a77 to 2cfb8a1 Compare April 26, 2024 00:03
@tlively tlively force-pushed the parser-no-tok-nums branch from 6bf227b to 1c0f12e Compare April 26, 2024 00:03
@tlively tlively force-pushed the parser-no-tok-string branch from 2cfb8a1 to fb535c3 Compare April 26, 2024 01:34
@tlively tlively force-pushed the parser-no-tok-nums branch 2 times, most recently from dc21b29 to 17ec230 Compare April 26, 2024 02:55
@tlively tlively changed the title [Parser] Do not eagerly parse numbers [Parser] Do not eagerly lex numbers Apr 26, 2024
Copy link
Member Author

tlively commented Apr 26, 2024

Merge activity

  • Apr 25, 11:47 PM EDT: @tlively started a stack merge that includes this pull request via Graphite.
  • Apr 26, 1:23 AM EDT: Graphite rebased this pull request as part of a merge.
  • Apr 26, 1:56 AM EDT: @tlively merged this pull request with Graphite.

@tlively tlively force-pushed the parser-no-tok-string branch from d6a6223 to 15d01ea Compare April 26, 2024 04:51
Base automatically changed from parser-no-tok-string to main April 26, 2024 05:22
Lex integers and floats on demand to avoid wasted work. Remove `Token`
completely now that all kinds of tokens are lexed on demand.
@tlively tlively force-pushed the parser-no-tok-nums branch from 17ec230 to f07ff79 Compare April 26, 2024 05:22
@tlively tlively merged commit abd5143 into main Apr 26, 2024
13 checks passed
@tlively tlively deleted the parser-no-tok-nums branch April 26, 2024 05:56
@gkdn gkdn mentioned this pull request Aug 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants