T2-1 CONTRACT — Drop legacy address + household-member tables (#890)

On this page

Epic &56 / Track 2, the CONTRACT half of T2-1 (#890; Relates to #683). T2-1 was an expand-contract migration (ADR-016): A1 (#683) versioned addresses into address_versions and A2 (#683) versioned household membership into household_member_versions (the EXPAND), each keeping the legacy flat table frozen + an idempotent backfill_*_versions_v1() function so the backfill-transformation test stayed runnable. This is the CONTRACT step: drop the two backfill functions + the two legacy tables, fix the remaining writer-side truncate couplings, regenerate the demo dataset, and flip every doc from "frozen … drop deferred" to "dropped". Mirrors the merged T1-4 CONTRACT (20260618000000_drop_legacy_fact_tables.sql).

Scope boundary

The drop + its writer-side + doc fallout, in canopy-persons + the two seed truncate paths (canopy-seed render_persons and xtask seed --reset) + the demo dataset. The issue body names only addresses, but the title + the A2 migration confirm household_members + its backfill carry identical debt — both pairs drop.

OUT of scope:

  • The surviving *_versions corpus and its reads — untouched.

  • A DB-level append-only trigger on the version/determination rows (pre-existing gap; the append-only trigger lives only on determination_snapshots).

  • The stale "Placeholder until MR-b lands" doc comments on DemoSubcommand (xtask/src/cmd/demo.rs) — a separate doc nit, filed as a follow-up.

  • #894 / #895 supersession follow-ups; T2-2/T2-6/T2-7/T2-8 spec-stage issues.

Status

Step Description Status

(plan)

This execution plan + nav entry.

Done (2026-06-22) — the plan commit.

migration

20260625000000_drop_legacy_address_household_member_tables.sql — DROP FUNCTION ×2 then DROP TABLE ×2 (function-before-table, IF EXISTS, no CASCADE — no inbound FK).

Done (2026-06-22) — the implementation commit.

seed truncate ×2

canopy-seed render_persons truncate list (drop the 2 legacy tables) + fix the false truncate() IF EXISTS doc comment; xtask seed.rs RESET_TABLES canopy_persons → the 5 version tables + persons/households (fixes the latent T1-4 atomicity bug).

Done (2026-06-22) — the implementation commit.

tests

Remove the two backfill_*_populates_corpus_from_legacy tests; add a positive to_regclass/to_regprocedure drop-assertion test; update the test module doc + the two stale store/models.rs "frozen, no Rust reads it" production comments.

Done (2026-06-22) — the implementation commit.

docs

data-models/canopy-persons.adoc: catalog rows → DROPPED, prose, Cross-service FK section, Indexes list, full ERD correction (remove all phantom legacy entities incl. T1-4’s + add the 5 version tables), header counts, migration-files list; CHANGELOG (edit 4 contradictory bullets + add a Removed entry); master plan line 238.

Done (2026-06-22) — the implementation commit.

demo dataset

cargo xtask demo regenerate (generator IS render_persons, fixed above) → clean devstack/demo-dataset/*.sql; grep-clean + check-drift.

Done (2026-06-22) — the implementation commit.

Context

A1/A2 kept the legacy addresses / household_members tables frozen so each backfill_*_versions_v1() function (and its transformation test) stayed runnable on the ephemeral schema. No Rust code reads or writes either legacy table now. The version corpus (address_versions / household_member_versions) is the sole store, the seed writes it directly, and EphemeralSchema::new_for_persons applies every migration — so leaving the drop deferred only carries dead schema + a false "frozen pending drop" claim across the docs. This is the standard expand-contract CONTRACT step (T1-4 / #672 did the same for income/assets/expenses via 20260618000000_drop_legacy_fact_tables.sql).

Two writer-side truncate couplings still name the legacy tables: canopy-seed’s render_persons truncate list, and xtask seed --reset’s `RESET_TABLES (whose canopy_persons entry still lists the T1-4 income/assets/expenses too — so --reset already atomically fails its combined TRUNCATE and silently no-ops the persons truncate; this MR fixes that latent bug). The committed demo dataset is stale from before A1/A2; its generator is sql::render_persons, so regenerating after the truncate fix produces clean output.

Implementation

Single MR, Relates to #683 + Closes #890, two commits (the docs: plan commit first, then the chore(persons): implementation). Each commit build-green; per-commit the pre-commit token gate + a fresh J1–J8 subagent over the staged diff, reported as text.

  1. Migration — the DROP FUNCTION ×2 + DROP TABLE ×2 (function-before-table).

  2. Seed truncate #1 — tools/canopy-seed/src/sql.rs render_persons list − the 2 legacy tables; fix the truncate() doc comment.

  3. Seed truncate #2 — xtask/src/cmd/seed.rs RESET_TABLES canopy_persons → the 5 version tables + persons/households.

  4. Tests — remove the 2 backfill tests; add the drop-assertion test; keep use sqlx::Row;; update the test module doc + the 2 store/models.rs production comments.

  5. Docs — data-models/canopy-persons.adoc (rows/prose/FK/indexes/ERD-full/counts/migration-list); CHANGELOG (4 bullets + Removed); master plan line 238.

  6. Demo — cargo xtask demo regenerate + grep-clean + check-drift.

Verification

cargo build -p canopy-persons -p canopy-seed; cargo build -p xtask; set -a; source .ports.env; set +a; cargo nextest run -p canopy-persons -p canopy-seed (persons on the shared CANOPY_PORT_POSTGRES_5432/canopy_persons). cargo clippy -p canopy-persons -p canopy-seed --all-targets — -D warnings. cargo xtask quality-budgets (flat)
cargo xtask coverage (the drop is a legitimate loss — the removed tests covered only the dropped SQL functions). cargo xtask check-docs + docs plan-lint. After cargo xtask dev refresh: the 2 tables + 2 functions gone, the 5 version tables remain; cargo xtask demo regenerate + check-drift clean; cargo xtask seed --reset actually clears persons/households. Full pre-push battery (cargo xtask validate --skip-docker + e2e + cargo doc + k6 + lfs).

As-built

  • The migration + both seed-truncate fixes landed as planned; cargo xtask dev refresh re-migrated + reseeded clean (the persons seed load proves the render_persons truncate fix — it no longer names the dropped tables).

  • The xtask seed --reset RESET_TABLES fix also corrected a latent T1-4 bug: its combined TRUNCATE listed the already-dropped income/assets/expenses, so it aborted atomically and silently no-op’d the persons truncate.

  • The two backfill-transformation tests were replaced by one positive to_regclass/to_regprocedure drop-assertion test (legacy_address_and_member_tables_and_backfills_are_dropped).

  • demo-dataset regeneration was broader than scoped: cargo xtask demo regenerate (the generator is sql::render_persons) rewrote 9 service datasets / ~3,300 lines — almost entirely pre-existing drift accumulated across A1/A2/T1-10/T2-4 (the committed dataset had never been regenerated and no CI job gates demo check-drift). On the user’s call the full regeneration is committed here, turning the drift gate green.

  • Follow-up filed: harden the pre-push battery (cargo xtask validate) to run demo check-drift plus the other locally-runnable CI-only gates (compliance audit-data-tenancy, the policy audit/audit --source federal/drift/action-coverage/input-coverage + scenarios audit suite, secrets-yaml-lint, cargo-doctest) — the GitLab-native SAST/secret/dependency scanners
    the DinD validate-in-network + slow coverage stay CI-only.

Follow-ups

  • xtask/src/cmd/demo.rs DemoSubcommand doc comments still say "Placeholder until MR-b lands" though regenerate/check-drift are implemented — a doc nit to file separately.

Edit this page · default