[WIP] Scanner Refactoring & Optimization #58728
Conversation
typescript-bot
added
the
For Uncommitted Bug
|
This PR doesn't have any linked issues. Please open an issue that references this PR. From there we can discuss and prioritise. |
| @@ -1127,7 +1116,7 @@ export function createScanner(languageVersion: ScriptTarget, skipTrivia: boolean | |||
| * `pos` is outside the bounds set for `text`, `CharacterCodes.EOF` is returned instead. | |||
| */ | |||
| function codePointChecked(pos: number) { | |||
| return pos >= 0 && pos < end ? codePointUnchecked(pos) : CharacterCodes.EOF; | |||
| return pos < end ? codePointUnchecked(pos) : CharacterCodes.EOF; | |||
There was a problem hiding this comment.
I carefully examined all expressions involving pos-- and pos -= number and determined that the pos >= 0 check is unnecessary.
|
Is this the same as #58362? |
| @@ -2559,7 +2550,7 @@ export function createScanner(languageVersion: ScriptTarget, skipTrivia: boolean | |||
| let regExpFlags = RegularExpressionFlags.None; | |||
| while (true) { | |||
| const ch = codePointChecked(pos); | |||
| if (ch === CharacterCodes.EOF || !isIdentifierPart(ch, languageVersion)) { | |||
| if (!isIdentifierPart(ch, languageVersion)) { | |||
There was a problem hiding this comment.
This check is unnecessary since isIdentifierPart(CharacterCodes.EOF, languageVersion) must return false (because CharacterCodes.EOF <= CharacterCodes.maxAsciiCharacter).
| @@ -2997,7 +2988,7 @@ export function createScanner(languageVersion: ScriptTarget, skipTrivia: boolean | |||
| } | |||
|
|
|||
| function isClassContentExit(ch: number) { | |||
| return ch === CharacterCodes.closeBracket || ch === CharacterCodes.EOF || pos >= end; | |||
| return ch === CharacterCodes.closeBracket || ch === CharacterCodes.EOF; | |||
There was a problem hiding this comment.
This is unnecessary as all isClassContentExit calls are directly after charCodeChecked.
Ah, I overlooked this. Let me omit all bound-checking related changes and leave the rest then. |
graphemecluster
changed the title
|
I guess I need to get into the habit of searching for relevant PRs, not just issues, in a search engine, before open up one. |
|
Fine, let me close this first since the branch cannot be renamed. |
This involves only the scanner, and includes refactoring like preferringcharCodeCheckedandcodePointCheckedover bound checks pluscharCodeUncheckedandcodePointUnchecked, avoiding the use of string literals and string index property accessing, and preferringString#substringoverString#slicedue to performance.Part of the above is already done in #58362.
Originally I would like to eliminate all regexes inside the scanner, but they don’t seem to affect performance at all, so I ended up abandoning the related changes.
This PR shouldn’t affect anything other than performance.