feat(test): multi-target migration test harness (WS4)#496
Draft
saevarb wants to merge 16 commits into
Draft
Conversation
Add transient project workspace with spec/plan stubs seeded from the Linear project (PN May WS4) milestones M0-M3 and open questions.
Add a generic TestTargetAdapter<TContract, TSchemaIR, TDriver, TPolicy> plus an applyMigration helper in @prisma-next/test-utils, and concrete adapters for SQLite, Postgres, and MongoDB to validate the abstraction holds within the SQL family and across families. Existing sqlite migration tests are untouched.
Local cache for fetched reference repositories (e.g. prisma/prisma).
Smoke tests proving the generic harness works at runtime on all three targets. SQL family fan-out (sqlite + postgres) is two parallel describe blocks rather than a shared loop because applyMigration is invariant in its driver/contract type parameters; a heterogeneous-target array is a union TS will not distribute. A future SQL-fan-out helper can paper over that.
Add describeAcrossTargets in @prisma-next/test-utils/migration-fanout: it takes a map of target cases sharing a contract type and schema-IR type, generates one describe block per target (so failures attribute), and exposes a closure-typed runMigration that hides the driver behind unknown. Solves the invariant-TDriver problem the spike exposed when iterating a heterogeneous-target array. Wire the real verifyMongoSchema in the mongo test target (was a stub). Refactor the cross-sql spike test to use the new fan-out helper as a working example.
Reduce harness.ts to a thin compatibility shim that re-exports
applyMigration from @prisma-next/test-utils/migration-harness with
sqliteTestTarget pre-bound. The 35 existing migration tests across 5
files don't change — they keep importing { applyMigration, int, text,
pack, integerColumn } from './harness' as before.
Cuts ~150 lines of target wiring (planner/runner/control-stack/family
descriptor setup) that is now centralised in the test target adapter.
Promote the spec from drive-create-spec stubs to the actual scope: the three-layer harness (generic core, fan-out helper, concrete adapters), what M1/M2/M3 deliver, what stays target-specific, and the open questions around adapter homes and the Workers dimension. Update plan to reflect that M1 is mostly landed: generic core, real adapters, fan-out, migration of existing tests. Adapter homes and Workers dimension are pending but not blocking M3.
… to postgres Add test/e2e/framework/test/migration-targets/sql-fanout.ts: a SQL-family fan-out helper that wraps applyMigration with per-target builders (column types + a target-bound defineContract) and a structurally-typed driver that translates ?-placeholders to $N for postgres. Convert the four portable sqlite migration test files to describeSqlMigration: 27 scenarios now run against both sqlite and postgres. Sqlite-only behaviors (rowid auto-increment, datetime() now() canonicalization) stay as plain describe blocks. fk-preservation.test.ts is untouched - its assertions target SQLite recreate-table internals which have no postgres equivalent. Run on both targets surfaces 3 postgres-only failures, all default-mismatch: 1. integer @default(N) gets returned as bare N from introspection; parsePostgresDefault does not normalise back to literal(N) 2. changing a column default is not picked up by the postgres widening planner - the runner reports success but the stored default is still the origin value, so verifySqlSchema flags drift
Spike validates the contract-typed SQL DSL works without emitting
contract.d.ts:
- defineContract({ models }) inferred type carries literal model and
column keys end-to-end.
- validateContract<typeof contract>(contract, emptyCodecLookup) preserves
literals - no .d.ts artifact required.
- sql({ context }) produces a db with literal-keyed table proxies; column
selection is type-checked.
- The same machinery flows when the contract is built inside a function
(which is how migration tests build them inside it() callbacks).
- Negative type checks live in unreached lambdas so the runtime DSL's
column guard doesn't fire on the @ts-expect-error path.
Sqlite contracts are type-only here because createTestContext from
sql-runtime/test/utils is hardcoded to a postgres test target; runtime
construction needs a sqlite test target descriptor (follow-up).
Also commits issue-triage.md capturing the 3 postgres-only failures the
SQL fan-out surfaced (parsePostgresDefault round-trip + widening planner
missing change-default).
Replace the positional seed + assertions callbacks with a single options
object containing { origin?, destination, policy?, before?, after }. The
fan-out helper now stands up a contract-typed sql-builder runtime per
phase: before is typed against origin, after against destination.
await runMigration({
origin: ...,
destination: ...,
before: async ({ db, runtime, driver }) => { ... }, // typed via origin
after: async ({ db, runtime, driver, schema }) => { ... }, // typed via destination
});
Connection sharing:
- sqlite: each runtime opens its own DatabaseSync on the same file the
control driver uses. SQLite handles concurrent access via file locking.
- postgres: the test target opens a single pg.Client and constructs the
control driver directly around it; the wrapped control driver close()
is a no-op so cleanup owns lifecycle. The runtime driver receives a
Proxy-wrapped pg.Client whose end() is a no-op so runtime.close()
cannot tear down the shared connection mid-test.
Migration tests are unchanged behaviourally - all 4 portable test files
(additive, destructive, widening, default-drift) run on sqlite and
postgres. The 3 known postgres-only default_mismatch failures still
surface (parsePostgresDefault round-trip + widening planner missing
change-default); 57/60 pass.
Tests still call driver.query directly for INSERTs and SELECTs - the
contract-typed db / runtime are wired in but not yet exercised by test
bodies. That happens next.
Delete typed-dsl.spike.test-d.ts: the typing it proved is now exercised
in real tests via the harness.
The before/after callbacks now expose a contract-typed db (Db<TContract>)
that compiles down to a real runtime DSL on both targets. Two tests in
destructive.test.ts (drops-column-preserves-data and changes-type-
preserves-data) now use db.User.insert({...}) and db.User.select(...)
instead of raw driver.query, exercising the full inference + execution
path: defineContract (preserves Models literal) -> runMigration infers
TOrigin/TDestination -> Db<T> resolves column keys + JS types -> sqlite
/postgres runtime stack executes the plan.
Type-machinery fixes:
1. defineContract Models constraint uses exported ContractModelBuilder
(structurally a private ModelLike) instead of trying to approximate.
2. SqlFanoutContext.defineContract is declared as a generic METHOD
(not property) so `const Models` is preserved through ctx access.
3. SqlFanoutContext.runMigration uses `const TOrigin, const TDestination`
so the contract literal types flow into Db<T> in the callbacks.
4. SqlFanoutContext.int / text / integerColumn / textColumn typed
against sqlite canonical column descriptors so the resulting
ScalarFieldBuilder has a literal codecId (without this, the JS scalar
lookup at db.X.insert({...}) collapsed to `never`). Postgres runtime
values cast to the canonical type — JS scalar types coincide.
5. sqliteDefineContractTyping has an explicit ReturnType<typeof
baseDefineContract<...>> annotation to avoid TS6133 "inferred type
exceeds max serialization length" when typeof captures the function.
Results: 57/60 tests pass (3 known postgres-only default_mismatch
failures already triaged in projects/multi-target-test-harness/issue-
triage.md, unchanged by this commit).
Replace the sqlite-canonical-with-casts approach with proper union types.
The previous version typed the interface against sqlitePack and cast
postgres values via 'as unknown as CanonicalIntColumn' etc. — sqlite was
privileged at the type level even though both targets are equally
supported, and the casts hid a type-level lie (postgres tests pretended
to be sqlite tests for typing purposes).
Now the interface uses SupportedSqlPack / SupportedIntColumn /
SupportedTextColumn — unions over every supported SQL target. Each case
provides its own concrete pack and column descriptors at runtime; the
union types are upper bounds for the static interface. No target is
privileged. Adding a new SQL target (e.g. mysql) extends the unions.
The defineContract impls cast 'as unknown as DefineSqlContract' because
TypeScript treats ContractDefinition as nominally invariant in Target
— each impl returns a contract with its specific Target, structurally
a subtype of the union Target the interface advertises. The cast is
soundness-preserving (specific → supertype) but TS doesn't infer it.
Verified end-to-end:
- 57/60 migration tests pass (same 3 known postgres-only default_mismatch
failures, unchanged).
- Negative type checks confirmed: db.X.select('nonexistent') and
db.X.insert({ id: 'wrong-type' }) are real compile errors on both
targets (verified via temporary @ts-expect-error test, since removed).
…get registry
Spike a v2 SQL fanout that fixes the column-hardcoding problem in the
production sql-fanout: SqlFanoutContext.int/text/integerColumn/textColumn
forced every new column type to extend the abstraction.
Two structural changes:
1. sqlTargetRegistry is the source of truth — one object with the per-
target config (pack, testTarget, buildRuntime). SqlTargetName is
derived from keyof typeof sqlTargetRegistry, so adding mysql is a
one-entry change that propagates to every column map via the type
checker.
2. describeSqlMigration is generic over a caller-supplied column map.
A commonSqlCols module constant carries the baseline (int, text);
tests that need more columns pass them as a middle arg:
describeSqlMigration(name, body) // common only
describeSqlMigration(name, extras, body) // common ∪ extras
ctx.cols[K] is typed as the per-target union for that column.
Spike test file (destructive-v2.spike.test.ts) exercises both paths:
- Scenario 1: drops-column-preserves-data using only common cols.
- Scenario 2: a contract using a `datetime` column passed inline via
the extras arg — proves new column types do not require changes to
the fanout file.
4/4 spike tests pass (2 scenarios x 2 targets). Production tests
unchanged: 57/60 still pass with the same 3 known postgres-only
default_mismatch failures.
If this proves acceptable, the migration plan is: replace sql-fanout.ts
with this shape, drop column destructuring in the four existing test
files, delete these spike files.
…ed columns
Replace the column-hardcoded SqlFanoutContext (with int/text/integerColumn/
textColumn baked into the interface) with a parameterized shape:
- sqlTargetRegistry is the source of truth for per-target config (pack,
testTarget, buildRuntime). SqlTargetName is derived from
keyof typeof sqlTargetRegistry; adding a target propagates to every
caller-supplied column map via the type checker.
- commonSqlCols is a module-level constant carrying the shared baseline
(int, text). Tests that need more columns pass them as an extras arg.
- describeSqlMigration has two overloads:
describeSqlMigration(name, body) // common cols only
describeSqlMigration(name, extras, body) // common ∪ extras
ctx.cols is typed as { [name]: union-over-targets } so the DSL has
literal codecIds.
Test files updated mechanically: drop the destructured int/text/
integerColumn/textColumn from the body callback; replace direct usage
(int.id(), text.optional() etc.) with field.column(cols.int).id() etc.
The two SQLite-specific test sections (rowid auto-increment in additive
and now()-canonicalization in widening) continue to use the local
harness.ts helpers since they only run on sqlite.
Behavior is unchanged: 57/60 tests pass on both targets, same 3 known
postgres-only default_mismatch failures already triaged in
projects/multi-target-test-harness/issue-triage.md.
The spike files (sql-fanout-v2.spike.ts, destructive-v2.spike.test.ts)
are deleted now that the design is promoted.
Captures M1/M3 status, the two surfaced Postgres bugs (TML-2481/2482), the Mongo fromContract fix, describeMongoMigration design rationale, and immediate next steps for M3.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TestTargetAdaptercore (@prisma-next/test-utils/migration-harness), an L1 SQL fan-out helper (describeSqlMigration), and a parallel Mongo fan-out helper (describeMongoMigration)fromContractplumbing bug in the Mongo adapter that silently broke multi-step migrationsTracked failures (Postgres only, not regressions)
literalValuesEqual('0', 0)is false)setDefaultsilently skipped by the runner's idempotency probe becausecolumnDefaultExistsCheckis value-blindTest plan
pnpm --filter @prisma-next/e2e-tests test test/sqlite/migrations— 57/60 pass (3 Postgres failures are pre-existing bugs above)pnpm --filter @prisma-next/integration-tests test test/mongo— 10/10 pass (8 existing + 2 new spike tests)Notes
See
projects/multi-target-test-harness/handoff.mdfor full context on M1 status, the two bugs, and M3 next steps.