Persistent finalised-state: version-graph migrations + config-driven major selection#1347
Open
idky137 wants to merge 6 commits into
Open
Persistent finalised-state: version-graph migrations + config-driven major selection#1347idky137 wants to merge 6 commits into
idky137 wants to merge 6 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Replaces the finalised state's rigid (major, minor, patch) migration ladder with a version-graph planner, and rebuilds startup so it honours a configured DB major even when several exist on disk. The headline win: adding a migration is now one impl + one line in a registry list — no dispatch match to edit — and startup can never strand a database that crashed mid-migration.
This is the last plumbing DbV2 needs: once this lands, DbV2 is "a backend + one registry line".
Note since the issue was written
The issue describes major migrations via a "shadow DB". That mechanism was already removed and replaced by the ephemeral passthrough (reads served from the validator during migration). We built on the current primitives instead. A nice consequence: the indexer is paused during migration, so the old DB is static — the promotion swap is atomic and needs no write freeze.
Key decisions
All of this is pinned as a spec in docs/adr/0002-persistent-finalised-state-migrations.md — read that first.
The safety guarantee
Startup classifies each on-disk major as Authoritative or IncompleteBuild. A half-built new major (marked mid-build) is never served; everything else — including a DB that crashed mid-patch or mid-minor — is always reopened. So this change cannot brick your only database. There's a test asserting exactly that.
Adding a migration (the payoff)
…then add Migration1_2_1To2_0_0 to migrations! { … }. That's it. (Patch is even shorter; only minor needs a migrate() body. Full recipes in the ADR.)
Testing
Cargo nextest run — 190 tests green, clippy clean. Covers the planner, classification, selection probes, retention Keep/Delete, major build/promote/resume, and the minor fail-fast guard, alongside the unchanged existing migration suite.
Major switching is implemented and tested via a #[cfg(test)] stand-in. Two tests (full restart-selection, reopen-as-major-2) wait for real DbV2, because the stand-in reuses DbV1 which enforces major == 1 — noted in spawn_major. The retention config is wired to the chain-index layer only, to be surfaced in zainod when DbV2 ships.