Skip to content

Persistent finalised-state: version-graph migrations + config-driven major selection#1347

Open
idky137 wants to merge 6 commits into
devfrom
update_migration_for_db_options
Open

Persistent finalised-state: version-graph migrations + config-driven major selection#1347
idky137 wants to merge 6 commits into
devfrom
update_migration_for_db_options

Conversation

@idky137

@idky137 idky137 commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

What this does

Replaces the finalised state's rigid (major, minor, patch) migration ladder with a version-graph planner, and rebuilds startup so it honours a configured DB major even when several exist on disk. The headline win: adding a migration is now one impl + one line in a registry list — no dispatch match to edit — and startup can never strand a database that crashed mid-migration.

This is the last plumbing DbV2 needs: once this lands, DbV2 is "a backend + one registry line".

Note since the issue was written

The issue describes major migrations via a "shadow DB". That mechanism was already removed and replaced by the ephemeral passthrough (reads served from the validator during migration). We built on the current primitives instead. A nice consequence: the indexer is paused during migration, so the old DB is static — the promotion swap is atomic and needs no write freeze.

Key decisions

  • Version graph, not a migration-ID DAG — Zaino has a single authoritative DbVersion per DB, so a graph covers every case without a persisted applied-set (issue's Option A).
  • A migrations! { … } macro generates the dispatch enum + planner edges from one list, keeping the trait's default impls and static dispatch (no dyn) — this is what kills the boilerplate.
  • Three self-plumbing types: patch/major carry zero migration code by default; only minor (an in-place rebuild) needs a body. Rule of thumb: code-only → patch, change within a major → minor, change of major → major.
  • Retention config OldDbRetention { Keep, Delete } (default Keep) controls whether an old major's files survive a switch.

All of this is pinned as a spec in docs/adr/0002-persistent-finalised-state-migrations.md — read that first.

The safety guarantee

Startup classifies each on-disk major as Authoritative or IncompleteBuild. A half-built new major (marked mid-build) is never served; everything else — including a DB that crashed mid-patch or mid-minor — is always reopened. So this change cannot brick your only database. There's a test asserting exactly that.

Adding a migration (the payoff)

  // major: switch majors — default builds from the validator. No body needed.
  struct Migration1_2_1To2_0_0;
  impl<T: BlockchainSource> Migration<T> for Migration1_2_1To2_0_0 {
      const CURRENT_VERSION: DbVersion = DbVersion { major: 1, minor: 2, patch: 1 };
      const TO_VERSION:      DbVersion = DbVersion { major: 2, minor: 0, patch: 0 };
      fn migration_type(&self) -> MigrationType { MigrationType::Major }
  }

…then add Migration1_2_1To2_0_0 to migrations! { … }. That's it. (Patch is even shorter; only minor needs a migrate() body. Full recipes in the ADR.)

Testing

Cargo nextest run — 190 tests green, clippy clean. Covers the planner, classification, selection probes, retention Keep/Delete, major build/promote/resume, and the minor fail-fast guard, alongside the unchanged existing migration suite.

Major switching is implemented and tested via a #[cfg(test)] stand-in. Two tests (full restart-selection, reopen-as-major-2) wait for real DbV2, because the stand-in reuses DbV1 which enforces major == 1 — noted in spawn_major. The retention config is wired to the chain-index layer only, to be surfaced in zainod when DbV2 ships.

@idky137 idky137 changed the title Update migration for db options Persistent finalised-state: version-graph migrations + config-driven major selection Jul 1, 2026
@zancas zancas added the Low Priority Low priority issues that are not currently on our roadmap label Jul 1, 2026
@zancas zancas self-assigned this Jul 1, 2026
@oolu4236
oolu4236 requested a review from nachog00 July 1, 2026 15:27
@idky137 idky137 removed the Low Priority Low priority issues that are not currently on our roadmap label Jul 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor ZainoDB migrations from linear chain to version-graph planning

2 participants