Improve next_cache_entry cache traversal performance

[vdye]

Changes

• Update next_cache_entry to accept a "cached" starting position for index search
  • Because next_cache_entry searches the src_index, which shouldn't be modified in-place over the course of unpack_trees, the first search position for the next next_cache_entry call can be last_position + 1

This might also help with git blame performance, if it's using unpack_trees (CC: @ldennington).

Performance

This change ultimately only had an effect on sparse index checkouts, as demonstrated by the performance test results:

Test                                              ms/vfs-2.33.0     HEAD                  
------------------------------------------------------------------------------------------
2000.2: git reset -- does-not-exist (full-v3)     0.79(0.38+0.30)   0.91(0.43+0.34) +15.2%
2000.3: git reset -- does-not-exist (full-v4)     0.80(0.38+0.29)   0.85(0.40+0.35) +6.2% 
2000.4: git reset -- does-not-exist (sparse-v3)   0.76(0.43+0.69)   0.44(0.08+0.67) -42.1%
2000.5: git reset -- does-not-exist (sparse-v4)   0.71(0.40+0.65)   0.41(0.09+0.65) -42.3%

The reason appears to be because, in full indexes, mark_ce_used will advance cache_bottom such that, in next_cache_entry, already-checked entries are skipped. In sparse indexes, however, the presence of sparse directory entries will block the cache_bottom from advancing, so already-checked cache entries are not skipped in next_cache_entry. Since the cache tree still needs cache_bottom to not advance (per 17a1bb5), the transient hint position lets next_cache_entry shortcut already-checked entries even for a sparse index.

[derrickstolee]

Changes

• Update next_cache_entry to accept a "cached" starting position for index search

  • Because next_cache_entry searches the src_index, which shouldn't be modified in-place over the course of unpack_trees, the first search position for the next next_cache_entry call can be last_position + 1

I think this is a very promising direction, especially in the short term.

Test                                              ms/vfs-2.33.0     HEAD                  
------------------------------------------------------------------------------------------
2000.2: git reset -- does-not-exist (full-v3)     0.79(0.38+0.30)   0.91(0.43+0.34) +15.2%
2000.3: git reset -- does-not-exist (full-v4)     0.80(0.38+0.29)   0.85(0.40+0.35) +6.2% 
2000.4: git reset -- does-not-exist (sparse-v3)   0.76(0.43+0.69)   0.44(0.08+0.67) -42.1%
2000.5: git reset -- does-not-exist (sparse-v4)   0.71(0.40+0.65)   0.41(0.09+0.65) -42.3%

This is great!

The reason appears to be because, in full indexes, mark_ce_used will advance cache_bottom such that, in next_cache_entry, already-checked entries are skipped. In sparse indexes, however, the presence of sparse directory entries will block the cache_bottom from advancing, so already-checked cache entries are not skipped in next_cache_entry. Since the cache tree still needs cache_bottom to not advance (per 17a1bb5), the transient hint position lets next_cache_entry shortcut already-checked entries even for a sparse index.

This is a good point to indicate that cache_bottom is not advancing the same way with sparse directory entries. We might want to dig into that more, since maybe there is something that causes other problems in the future. It's probable that the things I did around cache_bottom could be done better.

But let's make a note to explore cache_bottom later (I'm thinking October timeframe, still with time to respond before the 2.34 release), but this is an excellent short-term step.

[derrickstolee]

I was worried that the changes here could be short-term and want to be removed in the future, but it seems like adding this "hint memory" to next_cache_entry() is a good thing to do, anyway. It avoids quadratic growth in the algorithm!

Just a couple nits.

[derrickstolee]

Now that I look closely at your implementation, this old code had quadratic complexity! That helps me understand why having ~70,000 index entries still took 5 seconds to iterate.

I suppose I was expecting hint to be initialized with some information other than -1, but this seems to be a good idea regardless of any issues with the sparse index.

[ldennington]

Sorry for the retroactive review comments - if it would be better to discuss on a call lmk!

I see index->cache_nr frequently but am having some trouble understanding it/finding docs on what it means (I think you mentioned "cache number," but I'm still not sure why that's significant.

[derrickstolee]

cache_nr is the number of entries in the cache array.

In the Git codebase, you'll see a lot of X_nr and X_alloc representing that there are X_nr elements in an array currently allocated to have X_alloc entries.

[ldennington]

Thanks!

[ldennington]

I know cache_bottom is critical to this change, but a little unsure what it means?

[derrickstolee]

As we scan the index, we are also drilling down the trees, so the cache-tree extension (if available) is giving us some idea of the range of index entries corresponding to a given tree. This can be computed dynamically, too. The cache_bottom value is storing the expected lower-end of this range. As we recurse into subtrees, the cache_bottom is stored in a stack variable somewhere and then the o->cache_bottom value is rewritten as the recursion ends.

In general: the cache_bottom thing is necessary to make these index scans work properly, but it is complicated because the index is a flat list of files. It's just one more thing that is added on top to try and make sense of that difference with the trees in the object database.