Skip to content

ci(migrations): Cache the migrated test databases between runs - #128394

Merged
joshuarli merged 3 commits into
masterfrom
ci/cache-migration-test-db
Oct 21, 2026
Merged

joshuarli merged 3 commits into
masterfrom
ci/cache-migration-test-db

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

backend / backend migration tests spends about 237s on master CI building test_region, test_control and test_secondary. The first test's setup runs every migration from scratch.

With this change:

  • On master, after the tests pass, the job saves a pg_dump of the three test databases plus a manifest of the migration files it was built from. The cache key hashes src/**/migrations/*.py and src/sentry/new_migrations/**.
  • Every run restores the exact key, or the newest master dump through restore-keys, before running pytest. The job already uses --reuse-db, so pytest-django keeps the restored databases and applies only the migrations added since the dump.
  • If the manifest lists a migration file this checkout doesn't have, the script skips the restore and pytest builds the databases from scratch as it does today. That happens when a PR branch is behind the master commit that saved the dump.
  • .github/file-filters.yml: changes to the new script now trigger the migration tests.

Local measurements (single migration test, --migrations --reuse-db)

  • From scratch: 306s DB setup
  • After restoring a dump (~6s): ~40s DB setup; the test passes
  • I also checked that a manifest with an unknown migration skips the restore, and that a missing cache falls through to a normal build.

Not verified yet

  • No cache exists until this lands, so this PR's own run takes the "no saved databases" path. The first master run saves the dump, and the run after that is the first real restore in CI.
  • I didn't check the case where the dump is behind and pytest applies the newer migrations on top. That path is plain pytest-django keepdb plus migrate.

via @joshuarli.

--

View Junior Session [Sentry]

The migration tests job spends about 4 minutes building the test databases by running every migration from scratch. Master now saves a pg_dump of them keyed on the migration files, and later runs restore the newest dump so pytest only applies the migrations added since. A dump that records migrations missing from the checkout, such as a master dump restored on a PR branch that is behind, is dropped and the databases are built from scratch.

Co-Authored-By: Joshua Li <14209781+joshuarli@users.noreply.github.com>
@sentry-junior
sentry-junior Bot requested a review from a team as a code owner October 21, 2026 18:16
@sentry-junior
sentry-junior Bot requested a review from a team as a code owner October 21, 2026 18:16
@github-actions github-actions Bot added Scope: Frontend Automatically applied to PRs that change frontend components Scope: Backend Automatically applied to PRs that change backend components labels Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🚨 Warning: This pull request contains Frontend and Backend changes!

It's discouraged to make changes to Sentry's Frontend and Backend in a single pull request. The Frontend and Backend are not atomically deployed. If the changes are interdependent of each other, they must be separated into two pull requests and be made forward or backwards compatible, such that the Backend or Frontend can be safely deployed independently.

Have questions? Please ask in the #discuss-dev-infra channel.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/workflows/backend.yml Outdated
sentry-junior Bot and others added 2 commits October 21, 2026 18:24
Check a saved manifest of migration files instead of running Django through sentry exec and parsing psql output, and name the workflow's repeated cache path and save condition once.

Co-Authored-By: Joshua Li <14209781+joshuarli@users.noreply.github.com>
new_migrations/** also matched __pycache__ bytecode, which changes every run and kept the exact cache key from ever hitting.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 4a148db. Configure here.

Comment thread .github/workflows/backend.yml
@joshuarli
joshuarli enabled auto-merge (squash) October 21, 2026 18:31
@joshuarli
joshuarli merged commit 9f7d5ea into master Oct 21, 2026
87 checks passed
@joshuarli
joshuarli deleted the ci/cache-migration-test-db branch October 21, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant