Skip to content

Conversation

@Byron
Copy link
Member

@Byron Byron commented Jan 8, 2022

Tasks

  • basic test case for most simple of index files/figure out how to get test cases
  • stop using FileBuffer due to lack of maintenance and shortcomings in windows
  • parse header
  • parse EOIE (and refactor to get rid of struct)
  • parse TREE extension
  • parse entries V2
  • load extensions threaaded/unthreaded machinery
  • parse entries V4
  • parse IEOT extension
  • actual multi-threaded tests (like in git-pack crate)
  • use multi-threading as intended

Related to #293

Byron added 30 commits January 8, 2022 12:51
Note that in-code we must make sufficiently clear where a particular
fixture is coming from, or we name it after the test and file right
away.
It deals with comparing items from the work tree and the index,
and is generally what makes use of exclude specificiations.
It should be easy enough to learn from git tests to generate whichever
kind of index we need.
This is now sufficiently well implemented in the standard library.
It's sufficiently well supported using the standard library now.
For now the data structure is just 'as-written' and we see what
needs to change there as we have to maintain it.
Byron added 27 commits January 10, 2022 14:58
…even though it doesn't work yet as the flags don't pass an assertion.
Now it works more, but for some reason we don't see the trailer
checksum.
It seems extensions consume too much.
This leads to the first seemingly correct parsing of simple index files.
It's here only so that we can share the code across crates, for now
without any feature toggles.
Unfortunately we are a little more inefficient there as we have to copy
the shared portion into a buffer before we can use these bytes to extend
the backing storage with. Fair enough, it's most definitely not
measurable.
Now for actually using it, that needs some work.
This allows more delicate threading control like is required for the
index.
This iterator makes possible identifies results using a sequence id
and returns only consecutive items.

Use it to collect unordered results produced by threads.
It's advantage to collecting yourself and sorting is the potential
for a smaller memory footprint of in-flight results, one doesn't
have to collect them all for ordering, necessarily.
@Byron Byron merged commit 995994a into main Jan 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants