Devstack Migration Snapshot & Rollback

On this page

Overview

cargo xtask migrate snapshot and cargo xtask migrate rollback are developer-ergonomics tooling for the Docker devstack: snapshot every Canopy database into .devstack/snapshots/<timestamp>/ before applying an experimental migration, then roll back if it breaks.

This is dev/CI tooling only. Production rollback uses pg_basebackup plus WAL point-in-time recovery — see Database Backup & Restore.

Concern Tool

Production point-in-time recovery

pg_basebackup + WAL archive (operational-infrastructure plan Step 3)

Local dev — snapshot before applying a new migration

cargo xtask migrate snapshot

Local dev — undo a broken migration

cargo xtask migrate rollback

Down migrations (forward fix-up vs. rollback)

Forward-only is the modern pattern; emergency manual rollback uses this tool, not down-migration files (tracked as a follow-up issue).

When to use

  • About to merge a migration MR that touches a table you don’t own — snapshot first, run cargo xtask dev refresh to apply, smoke-test, roll back if the migration breaks something unexpected.

  • Reviewing a migration MR locally — snapshot, check out the branch, apply migrations via cargo xtask dev refresh, then roll back to your clean state when done.

  • Reproducing a determined-to-be-broken state in CI for a bug — capture a snapshot from an affected developer’s box, ship it as an attachment, restore on your own devstack with cargo xtask migrate rollback --id <ts>.

Commands

Command Effect

cargo xtask migrate snapshot

Runs pg_dump --format=custom --no-owner --no-privileges against each of the 19 Canopy databases via docker exec against the appropriate postgres-<program>-1 (or shared canopy-postgres-1) container. Writes archives + a manifest.json (snapshot id, git HEAD SHA, shared-db flag) to .devstack/snapshots/<timestamp>/. Skips databases that don’t exist (some stub services don’t create their DB).

cargo xtask migrate list

Prints all available snapshots in chronological order with their size, entry count, and git HEAD SHA at snapshot time.

cargo xtask migrate rollback

Restores from the most recent snapshot. Each archive is piped into pg_restore --clean --if-exists --no-owner --no-privileges --exit-on-error so existing tables are dropped and recreated. Aborts on first error rather than continuing through partial state.

cargo xtask migrate rollback --id 20260430T174056Z

Restore from a specific snapshot by id.

cargo xtask migrate rollback --db canopy_snap [--db canopy_tanf]

Restore only the named database(s). Pass --db multiple times to restore a subset. The filter is validated against the manifest BEFORE any restore runs — a typo aborts cleanly instead of partially restoring. Combine with --id <ts> to scope to an older snapshot.

Output paths

.devstack/snapshots/
  20260430T174056Z/
    manifest.json
    canopy_appeals.dump
    canopy_applications.dump
    ...
    canopy_wic.dump

The .devstack/ directory is gitignored — snapshots live on the developer’s machine only and never get checked in.

Manifest schema

{
  "snapshot_id": "20260430T174056Z",
  "git_sha": "ca0f836f44a773493a5fb6161f336724a612efdb",
  "shared_db": false,
  "entries": [
    {
      "database": "canopy_snap",
      "container": "canopy-postgres-snap-1",
      "archive": "20260430T174056Z/canopy_snap.dump",
      "size_bytes": 371320
    }
  ]
}

Limitations

  • Devstack only. The container names are hardcoded to canopy-postgres-*-1. Production databases live behind a different naming scheme (cloud-managed Postgres, host:port endpoints with TLS) — pg_basebackup is the right tool there.

  • Logical dump, not physical. Restored databases get the same data but may differ in toast/WAL state. Fine for dev; not equivalent to a physical replica.

  • No incremental snapshots. Each snapshot is a full dump of every DB. The 19 Canopy DBs total ~15 MB after seed data; a snapshot takes roughly 5-10 seconds and is cheap to take.

  • Per-database rollback uses the --db flag (issue #344, shipped 2026-05-01); no need to drop into raw pg_restore for the common case any more.

Implementation notes

Source: xtask/src/cmd/migrate.rs. Operational-infrastructure plan Step 4. Down-migration templates per critical table tracked as a separate follow-up issue (see plan footer).

Edit this page · default