Skip to content

fix: resolve hash mismatch and --indexer-path ignored in shallow index#85

Open
Sangoku wants to merge 1 commit into
johnhuang316:masterfrom
Sangoku:fix/hash-mismatch-and-indexer-path-shallow-index
Open

fix: resolve hash mismatch and --indexer-path ignored in shallow index#85
Sangoku wants to merge 1 commit into
johnhuang316:masterfrom
Sangoku:fix/hash-mismatch-and-indexer-path-shallow-index

Conversation

@Sangoku

@Sangoku Sangoku commented Mar 11, 2026

Copy link
Copy Markdown

Two bugs caused the shallow index to be rebuilt on every server startup:

  1. Hash length mismatch: shallow_index_manager.py and sqlite_index_manager.py used a 12-char truncated MD5 hash (hexdigest()[:12]) for the storage directory name, while project_settings.py used the full 32-char MD5 hash. This meant the index was written to e.g. /tmp/code_indexer/32697df368a6/ but looked for at /tmp/code_indexer/32697df368a617d19aac2f121d60624f/, so load_index() always failed and the index was always rebuilt from scratch.

    Fix: remove [:12] truncation from both files so all components use the same full 32-char MD5 hash.

  2. --indexer-path flag ignored by shallow index: shallow_index_manager.py hardcoded tempfile.gettempdir() and ignored ProjectSettings.custom_index_root, so the --indexer-path CLI argument had no effect on the shallow index storage location (only the deep SQLite index respected it).

    Fix: check ProjectSettings.custom_index_root first, falling back to tempfile.gettempdir() when not set.

Together these fixes ensure the shallow index is written to and loaded from the correct persistent location, eliminating unnecessary rebuilds on startup.

Two bugs caused the shallow index to be rebuilt on every server startup:

1. Hash length mismatch: shallow_index_manager.py and sqlite_index_manager.py
   used a 12-char truncated MD5 hash (hexdigest()[:12]) for the storage
   directory name, while project_settings.py used the full 32-char MD5 hash.
   This meant the index was written to e.g. /tmp/code_indexer/32697df368a6/
   but looked for at /tmp/code_indexer/32697df368a617d19aac2f121d60624f/,
   so load_index() always failed and the index was always rebuilt from scratch.

   Fix: remove [:12] truncation from both files so all components use the
   same full 32-char MD5 hash.

2. --indexer-path flag ignored by shallow index: shallow_index_manager.py
   hardcoded tempfile.gettempdir() and ignored ProjectSettings.custom_index_root,
   so the --indexer-path CLI argument had no effect on the shallow index storage
   location (only the deep SQLite index respected it).

   Fix: check ProjectSettings.custom_index_root first, falling back to
   tempfile.gettempdir() when not set.

Together these fixes ensure the shallow index is written to and loaded from
the correct persistent location, eliminating unnecessary rebuilds on startup.

@johnhuang316 johnhuang316 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix — I agree there is a real inconsistency in how index paths are derived today.

That said, I could not confirm the main claim that the 12-char vs 32-char hash mismatch is what causes the shallow index to rebuild on every startup. From the current startup flow, shallow load/build seems to stay entirely within , and both paths there use the same 12-char hash.

What I could confirm is that is not fully wired into the actual index managers yet. This PR updates the shallow side, but still appears to use directly.

Could you please:

  1. narrow the PR description to the behavior that is definitely fixed,
  2. update the deep/SQLite path handling as well if full support is the goal, and
  3. add regression tests for the real shallow/deep index storage paths and load behavior?

@johnhuang316 johnhuang316 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix — I agree there is a real inconsistency in how index paths are derived today.

That said, I could not confirm the main claim that the 12-char vs 32-char hash mismatch is what causes the shallow index to rebuild on every startup. From the current startup flow, shallow load/build seems to stay entirely within ShallowIndexManager, and both paths there use the same 12-char hash.

What I could confirm is that --indexer-path is not fully wired into the actual index managers yet. This PR updates the shallow side, but SQLiteIndexManager still appears to use tempfile.gettempdir() directly.

Could you please:

  1. narrow the PR description to the behavior that is definitely fixed,
  2. update the deep/SQLite path handling as well if full --indexer-path support is the goal, and
  3. add regression tests for the real shallow/deep index storage paths and load behavior?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants