T2-1 A1 — Address valid-time versioning (canopy-persons) (#683)

On this page

Epic &56 / Track 2, T2-1 (#683), slice A1. T1-3/T1-4 (#671/#672) gave canopy-persons an append-only, valid-time-versioned fact corpus for income/assets/expenses (ADR-027 §3). Addresses were left out — still a flat, mutable, event-less, un-versioned table. ADR-027 §3 names address as a time-varying fact that must be valid-time-versioned; A1 fulfils that, mirroring the proven income machinery. T2-1 is sliced: A1 — addresses (this MR), A2 — household_members (next), Half B — determination supersession (canopy-snap
canopy-eligibility, ADR-028 §57, the closing MR). A1 ships Relates to #683 (not Closes).

Scope boundary

A1 is the valid-time versioning of addresses in canopy-persons: a new address_versions table + testable backfill, an authored claim/close pipeline, attributed (street-redacted) events, and as-of reads — a near-mechanical mirror of income_versions. OUT of scope (each a follow-up or another slice): household_members versioning (A2); determination supersession (Half B, closes #683); dropping the legacy addresses table + backfill function (the CONTRACT step — kept this MR for backfill testability, a tracked follow-up mirroring T1-4’s dedicated drop migration); addresses in the T1-6 typed change-history (the per-resource history endpoint’s resource enum + worker UI — the generic audit capture DOES land here); a worker-portal address editor (none exists today); an address_type-aware "primary residential" selection (needs a canonical type enum — A1 only guarantees a deterministic first row).

Status

Step Description Status

(plan)

This execution plan + nav entry.

Done (2026-06-21) — 603af68.

migration + contracts + store

address_versions table + backfill_address_versions_v1(); AddressFactValue/AddressClaimRequest (+provenance on Address, remove CreateAddress); AddressEventValue (coarse) + AddressBeforeWindow/AddressClaimedEvent/AddressClosedEvent; paths CLAIM/CLOSE_ADDRESS; address_versions.rs store (mirror income + list_all_current_by_persons + deterministic order); delete store/addresses.rs + AddressRow.

Done (2026-06-21) — the A1 implementation commit.

api + events + downstream reads

claim_address/close_address_claim/persist_and_publish_address_claim; convert list_addresses (mirror list_income); remove add_address; map_fact_read_err pub(crate); routes/openapi (17→19); publish_address_claimed/publish_address_closed; repoint batch.rs + export.rs (all-current read; FOIA dedup; portability multi-row).

Done (2026-06-21) — the A1 implementation commit.

canopy-security + seed + consumers

address.claimed/address.closed parse_event_type arms + full parse_event tests; repoint the 4 seed address inserts to address_versions + reset/TRUNCATE list + integration check; test-lib client (claim_address/close_address); persons_test + batch_expansion_test + the insta snapshot; roundtrip.rs arbs.

Done (2026-06-21) — the A1 implementation commit.

tests

fact_versions/writes/reads/event_emission address coverage; backfill test (incl. active=false → closed); export-projection (multi-row portability, deduped FOIA); PII value-leak assertion; contracts roundtrip.

Done (2026-06-21) — the A1 implementation commit.

docs

data-models + api canopy-persons; CHANGELOG (Added/Changed/Removed); master plan A1 sub-note; this plan → Done + As-built; stale-doc sweep; api-docs --update.

Done (2026-06-21) — the A1 implementation commit.

Context

The income/asset/expense valid-time corpus (T1-3/T1-4) is the proven pattern: per-fact _versions tables (version_id/fact_id/person_id/valid_from/valid_to/superseded_at + provenance columns), a non-overlap GiST EXCLUDE over (fact_id, daterange) where current-accepted, the snapshot-and-supersede + remnant re-tile correction algorithm, attributed .claimed/income.closed outbox events, and an as-of read filtered to determination_statuses(). The addresses table predates all of it: flat, mutable, active soft-delete, no provenance, no events, no as-of read. A1 brings addresses to parity.

The shared substrate (store/fact_versions.rs: compute_remnants, lock_fact, author_columns, reconstruct_provenance, determination_statuses, FactReadError) is reused unchanged; store/income_versions.rs is the line-by-line template. Addresses differ only in (a) the value columns, (b) no numeric value (no negative guard, no value CHECK), and (c) a street-redacted event value (PII decision below).

Decisions

Decision Resolution

EXPAND, don’t drop

T1-3 created the version tables + a backfill_fact_versions_v1() function and kept the legacy tables; T1-4 separately dropped both. The ephemeral test harness runs every migration to head, so a backfill-transformation test needs the legacy addresses table present. So A1 creates address_versions + a testable backfill_address_versions_v1(), keeps the legacy addresses table frozen (no Rust reads it; the seed writes the version corpus), and flips all reads/writes to the corpus. The destructive DROP is a tracked CONTRACT follow-up.

Schema

One forward-only migration (ADR-016) sorting after 20260618000000 (e.g. 20260623000000_create_address_versions.sql), SPDX first line. Mirrors 20260604000000_create_fact_version_tables.sql: shared columns + value columns (address_type/line_1/line_2/city/state/zip/county_fips), address_versions_valid_range CHECK (valid_to IS NULL OR valid_to > valid_from), the non-deferrable address_versions_no_overlap EXCLUDE USING gist (fact_id WITH =, daterange(valid_from, valid_to, '[)') WITH &&) WHERE (superseded_at IS NULL AND claim_status LIKE 'accepted%'). No value CHECK (no numeric column). CREATE EXTENSION IF NOT EXISTS btree_gist WITH SCHEMA public;. Indexes on (fact_id) + (person_id) WHERE superseded_at IS NULL.

Identity

fact_id = the legacy address id (AddressId), stable across corrections (ADR-025); version_id per write. Person-scoped (ownership/as-of keyed by person_id) — identical to income. A person may hold several address facts (residential + mailing); each is its own fact_id; the per-fact_id EXCLUDE never conflicts across distinct ids.

Backfill

Idempotent backfill_address_versions_v1() (SQL), SELECT-called once. Per legacy row: fact_id=a.id, valid_from=a.effective_date, recorded_at=a.created_at, author system, claim_source='self_attestation', claim_status='accepted_unverified' (no verified column ⇒ conservative, mirroring the expense backfill), origin='backfill:v1', proposed_value=NULL, value columns copied. Both active and (defensively) inactive rows: valid_to = CASE WHEN a.active THEN a.end_date ELSE COALESCE(a.end_date, (a.updated_at AT TIME ZONE 'UTC')::date) END (an active=false row → closed at removal; none should exist today as addresses have no soft-delete write path, but no data is silently dropped). WHERE (<computed valid_to> IS NULL OR <computed valid_to> > a.effective_date) AND NOT EXISTS (… v.fact_id = a.id).

Address-event PII (street-redacted)

The address.claimed/address.closed events carry a coarse AddressEventValue — address_type/city/state/zip/county_fips, not line_1/line_2. Rationale: the canopy-mq restricted-field guard is top-level-keys-only and cannot catch a nested street; the FOIA export already redacts line_1. So the street never enters the outbox / RabbitMQ / audit ledger. The full AddressFactValue (with street) is the claim-request + stored type; the store keeps the full value for remnant re-tiling; only the event projection (AddressEventValue::from(&AddressFactValue)) drops the street. Consistent with ADR-004 (event bus carries non-restricted metadata) + ADR-027 §8 (the full value persists in canopy-persons, kept out of the immutable event history).

Contracts

addresses.rs: add AddressFactValue (full, Validate, ToSchema) + AddressClaimRequest (mirrors IncomeClaimRequest, deny_unknown_fields); add provenance: Option<Provenance> (additive) to Address; remove CreateAddress. events.rs: AddressEventValue (coarse, no ToSchema) + From<&AddressFactValue>; AddressBeforeWindow; AddressClaimedEvent (`from_claim`); `AddressClosedEvent` (new, author: None). paths.rs: add CLAIM_ADDRESS/CLOSE_ADDRESS_CLAIM, remove ADD_ADDRESS. lib.rs: do not re-export AddressClaimRequest (parity with income).

Store

address_versions.rs mirrors income_versions.rs: AddressSnap (full value, for remnant reinsert), the coarse projection, append_address_version → AddressAppendOutcome, close_address_version, into_address (provenance-bearing), list_by_persons_as_of/list_by_person_as_of (as-of, determination_statuses()), and list_all_current_by_persons (export — all live windows, no date filter). Deterministic ordering ORDER BY person_id, valid_from DESC, recorded_at DESC, fact_id (diverges from income’s tie-non-deterministic order, because canopy-web consumes .first()/addrs[0]). Delete store/addresses.rs; remove AddressRow + its From impl.

API

Add claim_address (mirrors claim_income minus the negative-amount guard) + close_address_claim + persist_and_publish_address_claim. Convert list_addresses to mirror list_income exactly (Path-only, resolve_as_of(None)); as-of-today is the correct "current address" semantic — a future-effective/closed address stops showing (acceptable behavior change). Remove add_address. Make map_fact_read_err pub(crate) (for export.rs). Update routes() (the /addresses URL now serves GET only via LIST_ADDRESSES + the two new claim routes), #[openapi], and the path-count assertion 17→19.

Export

Explicit rule: "all current windows" (list_all_current_by_persons, no date filter) — strictly ⊇ the prior active=true set, so no completeness regression (rejected: as-of-today would drop future-effective addresses from a legal export). Consequence: a gapped fact emits >1 row → the portability CSV intentionally emits one row per live window (id repeats), and the FOIA projection (coarse, date-less) must dedup identical rows. Both adjusted + tested.

Events / audit

publish_address_claimed/publish_address_closed (routing keys address.claimed/address.closed). canopy-security parse_event_type gains the two arms (canonical claim/close action vs the dot-split fallback); the generic fact_id + nested-author extraction in parse_event already applies to addresses (full attributed audit capture lands here).

Seed

Repoint the four tools/canopy-seed address inserts to address_versions (mirror the income_versions seed row; origin='seed'); add address_versions to the reset/TRUNCATE list + the seed integration check. Required for the demo + case-detail display.

Quality budgets

No new serde_json::Value in src (reuses the existing proposed_value JSONB; the new types are fully typed) ⇒ B3a flat. Removing CreateAddress/store/addresses.rs/AddressRow/add_address is a clean code-removal cluster: ratchet a budget DOWN if it moves; surface-and-decide any rise (ADR-030).

OpenAPI

persons.json via api-docs --update: + AddressClaimRequest/AddressFactValue, + provenance on Address, + the two claim paths, − CreateAddress + the POST on /addresses.

Implementation

Single MR, Relates to #683 (the split work does not close the issue — Half B closes it). The (plan) commit is first (this .adoc + nav). Each commit build-green; per-commit the pre-commit token gate + a fresh J1–J8 subagent over the staged diff, reported as text.

  1. Migration — migrations/<ts>_create_address_versions.sql: extension → CREATE TABLE address_versions → 2 indexes → backfill_address_versions_v1() → SELECT it once. Keep the legacy addresses table (comment: frozen, dropped in the CONTRACT follow-up).

  2. Contracts — addresses.rs/events.rs/paths.rs per the Decisions.

  3. Store — store/address_versions.rs (mirror income + list_all_current_by_persons + deterministic order); register in store/mod.rs; delete store/addresses.rs; remove AddressRow + re-export in store/models.rs.

  4. Events — publish_address_claimed + publish_address_closed.

  5. API — claim/close/persist + list_addresses conversion + remove add_address + map_fact_read_err pub(crate) + routes()/#[openapi]/17→19.

  6. Downstream reads — batch.rs → list_by_persons_as_of; export.rs → list_all_current_by_persons + import flip + super::map_fact_read_err + sample_address() provenance: None.

  7. canopy-security — two parse_event_type arms + full parse_event tests.

  8. Seed — repoint the four address inserts; add address_versions to the reset list + integration check.

  9. Consumers — test-lib client (claim_address/close_address); persons_test + batch_expansion_test + the insta snapshot redaction; roundtrip.rs arbs (remove arb_create_address/create_address_roundtrip!; add address_fact_value/address_claim_request roundtrips; arb_address gains provenance).

  10. Tests — the income test set mirrored for addresses + the backfill test (incl. active=false → closed) + export-projection tests (multi-row portability, deduped FOIA) + the PII value-leak assertion (no street literal anywhere in the serialized payload).

  11. Docs — data-models/api canopy-persons; CHANGELOG (Added/Changed/Removed); master plan A1 sub-note; this plan → Done + As-built; the stale-doc sweep (ClaimResponse doc, contracts lib.rs summary, require_fact_ownership doc); api-docs --update.

Verification

cargo build -p canopy-persons -p canopy-contracts-persons -p canopy-security -p canopy-test-lib -p canopy-seed; set -a; source .ports.env; set +a; cargo nextest run -p canopy-persons -p canopy-contracts-persons -p canopy-security (devstack-gated). cargo xtask quality-budgets (expect flat; ratchet DOWN if the removal lowers one). cargo xtask api-docs --update + commit the persons.json delta + confirm the path-count assertion. cargo xtask docs plan-lint
cargo xtask check-docs. After cargo xtask dev refresh: confirm seeded addresses appear via GET /v1/persons/{id}/addresses and the canopy-web case-detail address display still renders. Full pre-push battery (cargo xtask validate + e2e + cargo doc). Load-bearing assertions: the overlap EXCLUDE rejects a second current-accepted version; a correction re-tiles remnants; a close drops the fact from as-of-after reads but not as-of-before; the backfill produces the expected provenance; the address event payload contains no street value anywhere; the FOIA export dedups multi-window facts.

As-built notes

Built as planned (plan commit 603af68 + one implementation commit), Relates to #683. The plan held; the only deviations are mechanical, recorded here so the plan↔code diff stays zero.

  • list_by_person_as_of is a direct single-person query, not the income delegate pattern. The plan said "mirror income_versions`"; income’s `list_by_person_as_of delegates to the multi-person read + .remove(&id) .unwrap_or_default(). That unwrap_or_default would have pushed the LOCKED B5 budget 309→310. Per the offset-don’t-raise discipline the new debt was minimized: the single-person read is written as a direct person_id = $1 query that .collect()`s into the `Result<Vec<Address>, _> — no HashMap, no unwrap_or_default. B5 stays flat at 309 (and the single-person path is marginally cheaper). All other budgets flat (B3a 757 — zero new serde_json::Value; the typed value reuses the existing proposed_value JSONB).

  • AddressEventValue is a separate coarse type, not a field subset of AddressFactValue. To keep the street out of the event while keeping it in storage, the event payload uses a distinct AddressEventValue (address_type/city/state/zip/county_fips) with an impl From<&AddressFactValue> that drops line_1/line_2. The store’s AddressSnap keeps the full value (remnant re-tiling preserves the street); only address_before/from_claim project to the coarse event value. The redaction is asserted at BOTH layers: the contracts unit test + the outbox-payload-as-text integration test assert the street literal appears nowhere in the serialized event.

  • Export uses "all current windows" (list_all_current_by_persons), FOIA deduped. The bulk export reads every non-superseded accepted version (no date filter) — strictly ⊇ the prior active = true set, so a legal export loses no address; the FOIA projection dedups identical coarse rows (street + dates dropped), portability keeps one row per live window.

  • Legacy addresses table kept frozen (EXPAND only). The destructive CONTRACT-phase drop + backfill_address_versions_v1() removal is a filed follow-up (kept so the backfill-transformation test runs on the ephemeral schema, which applies every migration to head).

  • Cold-start flake note. The post-dev refresh warm battery is green; the unrelated fact_change_history_test trio cold-flaked once on broker poll-timeout immediately after the container recreate and passed on the warm re-run (the documented devstack cold-start pattern) — not a regression (A1 does not touch that endpoint).

  • Folded-in seed fix (determination_snapshots TRUNCATE guard). The forced reseed (this MR edits tools/canopy-seed/src, in the SEED_DIRS hash) surfaced a pre-existing, unrelated infra bug: the canopy-seed render_{snap,caps,wic} TRUNCATE … _determinations … CASCADE is blocked by the append-only determination_snapshots trigger (T1-10/T2-4) because the seed didn’t open the canopy.snapshot_maintenance window, and the loader swallowed the abort (WARN-and-continue), leaving snap/caps/wic stale on every reseed. Diagnosed by five independent contextless reviewers (consensus: the *generator is correct; the seed render + loader were not), proven by a byte-identical caps/wic seed diff (my change is addresses-only) + a live reproduction. Fixed in-MR (user decision, given it blocked the push + is a repo-wide reseed bug): SET LOCAL canopy.snapshot_maintenance = 'on' in the three program renders + a fail-loud loader. The two pre-existing caps/wic case-detail e2e specs (which depend on the freshly-seeded program data) now pass.

Follow-ups

File each as a separate GitLab issue and /relate #683:

  • chore: CONTRACT — drop the legacy addresses table + backfill_address_versions_v1() once A1 has baked (forward-only migration, mirroring T1-4’s drop migration).

  • feat: address change-history — extend the T1-6 fact-history resource enum (+ worker UI) to addresses, if product needs the per-resource address timeline (the generic audit capture already lands in A1).

  • feat: primary-address selection — a canonical address_type enum + an address_type-aware "primary residential address" read, so canopy-web’s case-detail shows the residential (not an order-arbitrary) address. A1 only guarantees a deterministic first row.

Edit this page · default