Add dbtool fold: emit self-contained baseline from registered migrations#481
Open
christianparpart wants to merge 1 commit intomasterfrom
Open
Add dbtool fold: emit self-contained baseline from registered migrations#481christianparpart wants to merge 1 commit intomasterfrom
christianparpart wants to merge 1 commit intomasterfrom
Conversation
1c305bc to
d8bf4e2
Compare
1bf3c43 to
35bcc7e
Compare
35cbced to
83bd900
Compare
Yaraslaut
approved these changes
Apr 30, 2026
Member
Yaraslaut
left a comment
There was a problem hiding this comment.
Thanks, left few small comments, mostly nitpicks
83bd900 to
7a78773
Compare
dbtool fold --output FILE emits a self-contained baseline (.cpp plugin
or .sql script) that reproduces the post-migration state from an empty
DB. .sql output requires --dialect (sqlite, postgres, mssql, mysql);
.cpp output is dialect-agnostic. Runs without any DB connection - loads
plugins, walks migrations in memory, writes a file.
Built on a new pure plan-walk primitive
MigrationManager::FoldRegisteredMigrations(formatter, upToInclusive)
that folds every registered migration into a per-table view of the
final shape plus a chronological list of data steps, indexes, and
releases.
The fold module (src/Lightweight/MigrationFold/{Folder,CppEmitter,
SqlEmitter}.{hpp,cpp}) emits via the existing ToSql() formatter path so
each dialect's CREATE TABLE / CREATE INDEX / INSERT codegen stays the
single source of truth. The .cpp emitter wraps the body in
LIGHTWEIGHT_SQL_MIGRATION; the .sql emitter additionally emits CREATE
TABLE schema_migrations and a stamping INSERT for every folded
timestamp so the post-fold DB looks identical to a real apply-all run.
Also pulls in CodeGen/SplitFileWriter shared codegen helper used by the
.cpp emitter to bin-pack large baselines across multiple files.
Tests: fold unit tests cover create/altercolumn/drop-table cleanup,
data-step chronological order, --up-to truncation, RawSql passthrough,
column rename FK propagation, release-range filtering, ResolveUpTo
parsing. SqlEmitter/CppEmitter round-trip tests verify the emitted
artifacts match the expected shape. SplitFileWriter tests cover bin-
packing, single-chunk, zero-budget, and oversize-block boundaries.
All [Fold] and [SplitFileWriter] tests pass against sqlite3,
mssql2022, and postgres. Full SqlMigration suite (44 cases / 210
assertions) green on all three.
Signed-off-by: Christian Parpart <christian@parpart.family>
7a78773 to
173d234
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a new offline
dbtool foldsubcommand that walks all registered migrations and emits a single self-contained baseline — either a.cppmigration plugin or a.sqlscript — that reproduces the post-migration schema andschema_migrationsrows from an empty database. Useful for collapsing a long migration history into a fast-to-apply starting point, or for shipping a snapshot baseline alongside a release.The command is purely offline: it never opens a DB connection, never queries a live schema. It loads plugins, walks each migration's
Up()plan in timestamp order, folds the cumulative effect into a per-table view + chronological data steps, and emits via the existingToSql()formatter path so each dialect's CREATE TABLE / INSERT codegen stays the single source of truth.Changes
MigrationManager::FoldRegisteredMigrations(formatter, upToInclusive)primitive — pure plan-walk, returnsPlanFoldingResult(per-table state, creation order, indexes, chronological data steps, in-range releases). Used by the new module and available to any future caller.Lightweight::MigrationFoldlibrary module undersrc/Lightweight/MigrationFold/:Folder— thin facade plusResolveUpTo()which accepts an empty string (latest registered release), a numeric timestamp, or a release version string.SqlEmitter— emits a flat dialect-specific.sqlscript, including aCREATE TABLE schema_migrationsand a stampingINSERTper folded timestamp so a freshly-loaded DB looks identical to a real apply-all run.CppEmitter— emits a.cppbaseline plugin wrapped inLIGHTWEIGHT_SQL_MIGRATION, with optional--emit-cmakeand--max-lines-per-filefor splitting very large baselines across multiple files.CodeGen/SplitFileWriterhelper that bin-packs blocks within a per-file line budget; used byCppEmitterand intentionally factored out for reuse.dbtool fold --output FILE [--up-to X] [--dialect D] [--emit-cmake] [--plugin-name N] [--max-lines-per-file N]— output format is picked from the file extension..sqlrequires--dialect(sqlite, postgres, mssql, mysql);.cppis dialect-agnostic. Dispatched beforeSetupConnectionStringsince fold never touches a DB; uses a connection-lessGetMigrationManagerOfflinevariant.--up-totruncation, RawSql passthrough, column rename FK propagation, release-range filtering,ResolveUpToparsing) + 4SplitFileWritercases + 2 emitter round-trip cases. Green against sqlite3, mssql2022, and postgres.