SI-8879 fix quadratic reading time in StreamReader #33
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
StreamReader.nextEol
used to loop all the way to Eol every time anelement was read. That's very costly when lines are long.
Furthermore, it used to call
PagedSeq.length
, forcingPagedSeq
toload the whole input in memory, even when a single character was read.
nextEol
is now saved as part of the state ofStreamReader
, and is passedto child readers when created (as long as we do not read past the end of
the line). Thus it computed only once per line, whatever the length.
With the example in the ticket (SI-8879), we get:
isDefinedAt
is used instead oflength
so that pages beyond thetested index do not need to be read. The test only tests this part.