T2-1 A2 — Household-member valid-time versioning (canopy-persons) (#683)

On this page

Epic &56 / Track 2, T2-1 (#683), slice A2. T1-3/T1-4 (#671/#672) gave canopy-persons an append-only, valid-time-versioned fact corpus for income/assets/expenses (ADR-027 §3), and A1 (#683, merged 50ad5f65) extended it to addresses. household_members were left out — still a flat, mutable, active-soft-delete table with bare-JSON household.member_added/ member_removed events (no attribution), no provenance, no as-of read. ADR-027 §3 names household membership a time-varying fact that must be valid-time-versioned; A2 fulfils that. T2-1 is sliced: A1 — addresses (done), A2 — household_members (this MR), Half B — determination supersession (canopy-snap + canopy-eligibility, ADR-028 §57, the closing MR). A2 ships Relates to #683 (not Closes). User-confirmed direction: full claim/close mirror (uniform authored-fact contract) + no pre-1.0 backward compat — strip legacy.

Scope boundary

A2 is the valid-time versioning of household_members in canopy-persons: a new household_member_versions table + testable backfill, an authored claim/close pipeline, attributed household.member_claimed/member_closed events, an as-of household read, and — because membership feeds Person.household_id — an as-of-aware person projection so the denormalised household_id stays consistent with the as-of household reads at every as_of. Membership is household-scoped (not person-scoped like income/address): the ownership
correction key is household_id, and the non-overlap EXCLUDE is per-(household_id, person_id) (a person may be in two different households at once, never the same one twice).

OUT of scope (each a follow-up or another slice): determination supersession (Half B, closes #683); dropping the legacy household_members table + backfill function (the CONTRACT step — kept this MR for backfill testability, a tracked follow-up mirroring A1’s #890); versioning the households entity itself (name/effective window — not a per-person time-varying fact); membership provenance in MemberFull (the /full bundle keeps the relationship string; provenance is on HouseholdMember via GET /households/{id}); members in the T1-6 typed change-history (resource enum + worker UI — the generic audit capture DOES land here).

Status

Step Description Status

(plan)

This execution plan + nav entry.

Done (2026-06-21) — the plan commit.

migration + contracts

household_member_versions table (per-(household_id, person_id) EXCLUDE) + backfill_household_member_versions_v1(); MemberFactValue/MemberClaimRequest (+provenance on HouseholdMember, remove AddMember); MemberBeforeWindow/MemberClaimedEvent/MemberClosedEvent (carry household_id); paths CLAIM/CLOSE_HOUSEHOLD_MEMBER.

Done (2026-06-22) — the A2 implementation commit.

store + events

household_member_versions.rs (mirror address, household-scoped; snap carries person_id); households.rs drop add/remove + get_with_members(as_of); models.rs drop AddMember/HouseholdMemberRow; publish_member_claimed/publish_member_closed.

Done (2026-06-22) — the A2 implementation commit.

api + as-of-aware person projection

claim_household_member/close_household_member_claim/persist_and_publish_member_claim/require_member_ownership; get_household(_full) as-of; remove add/remove; routes/openapi (19→19); map_claim_pg_error 23503→422; persons.rs as-of HOUSEHOLD_JOIN + update() one-off + resolve_as_of pub(crate) + thread as_of (incl. export.rs).

Done (2026-06-22) — the A2 implementation commit.

canopy-security + canopy-web + seed + consumers

parse_event_type arms (replace, no aliases) + tests; humanize_event keys (replace); members.rs claim/close bodies; repoint the 4 seed inserts to household_member_versions + reset list + integration check; applications + cli + test-lib (claim_member/close_member, given DSL).

Done (2026-06-22) — the A2 implementation commit.

tests

fact_versions/writes/reads/event_emission member coverage; per-(household_id, person_id) overlap test; backfill test (incl. active=false→closed, add→remove→re-add disjoint); 23503→422; person_id-mismatch→422; the as-of-consistency suite (the HIGH fix); contracts roundtrip + snapshots.

Done (2026-06-22) — the A2 implementation commit.

docs

data-models + api canopy-persons; CHANGELOG (Added/Changed/Removed); master plan A2 sub-note; this plan → Done + As-built; stale-doc sweep; api-docs --update.

Done (2026-06-22) — the A2 implementation commit.

Context

The income/asset/expense/address valid-time corpus is the proven pattern: per-fact _versions tables, a non-overlap GiST EXCLUDE where current-accepted, the snapshot-and-supersede + remnant re-tile correction algorithm, attributed .claimed/*.closed outbox events, and an as-of read filtered to determination_statuses(). The household_members table predates all of it: flat, mutable, active soft-delete, no provenance, no events, no as-of read. A2 brings membership to parity. The shared substrate (store/fact_versions.rs: compute_remnants, lock_fact, author_columns, reconstruct_provenance, determination_statuses, FactReadError) is reused unchanged; store/address_versions.rs is the line-by-line template.

Membership differs from income/address in three deliberate ways, each handled below: it is household-scoped, its non-overlap key is (household_id, person_id), and its versioned value (relationship) is not PII (no coarse event projection — the full MemberFactValue rides the event, like income). And because a person’s membership is denormalised onto every person read as Person.household_id, A2 also makes that person→household JOIN as-of-aware.

Decisions

Decision Resolution

EXPAND, don’t drop

A2 creates household_member_versions + a testable backfill_household_member_versions_v1(), keeps the legacy household_members table frozen (no Rust reads it; the seed writes the corpus), and flips all reads/writes to the corpus. The retained table is a data-migration artifact (the ephemeral harness runs every migration to head, so the backfill-transformation test needs the table present), NOT interface backward-compat. The destructive DROP is a tracked CONTRACT follow-up (mirrors A1’s #890).

Full claim/close mirror (user-confirmed)

The member API + events become the same authored claim/close contract every other fact uses. Breaking path + event rename. No pre-1.0 backward compat (user directive): the old POST/DELETE /members paths, the AddMember DTO, and the household.member_added/member_removed event names + their parse_event_type/humanize_event arms are deleted, not aliased. An in-flight/replayed old event falls through to the generic verb-split fallback — an accepted pre-1.0 break (fresh seed each deploy).

Schema

One forward-only migration (ADR-016) 20260624000000_create_household_member_versions.sql, SPDX first line. Shared columns + value column relationship TEXT NOT NULL + household_id UUID NOT NULL REFERENCES households(id). household_member_versions_valid_range CHECK (valid_to IS NULL OR valid_to > valid_from); the non-deferrable household_member_versions_no_overlap EXCLUDE USING gist (household_id WITH =, person_id WITH =, daterange(valid_from, valid_to, '[)') WITH &&) WHERE (superseded_at IS NULL AND claim_status LIKE 'accepted%') (btree_gist services both UUID equality columns — a migration comment calls this out; the existing fact tables used a single WITH =). No value CHECK (no numeric). Indexes on (fact_id), (person_id) WHERE superseded_at IS NULL (the PersonRow JOIN), (household_id) WHERE superseded_at IS NULL (the household read).

Identity + scope

fact_id = the legacy household_members.id (HouseholdMemberId), stable across corrections (ADR-025); version_id per write. Scope = household_id (ownership/correction key, matching the legacy WHERE id = member_id AND household_id); person_id is a stored, immutable-per-fact column (feeds provenance + the PersonRow JOIN; a correction may NOT reassign a membership to a different person). The non-overlap EXCLUDE is per-(household_id, person_id) — the valid-time form of the legacy UNIQUE (household_id, person_id) WHERE active = true. A double-add (overlapping membership, same household+person, via a new fact_id) — or a correction extending one fact to overlap another — trips 23P01 → 409.

Backfill

Idempotent backfill_household_member_versions_v1() (SQL), SELECT-called once. Per legacy row: fact_id=hm.id, person_id/household_id copied, valid_from=hm.effective_date, recorded_at=hm.created_at, author system, claim_source='self_attestation', claim_status='accepted_unverified', origin='backfill:v1', proposed_value=NULL, relationship copied. remove_member sets active=false but NOT end_date, so removed rows reconstruct as closed windows: valid_to = CASE WHEN hm.active THEN hm.end_date ELSE COALESCE(hm.end_date, (hm.updated_at AT TIME ZONE 'UTC')::date) END. Backfills both active and inactive rows; realistic add→remove→re-add data is non-overlapping per (household_id, person_id), so no EXCLUDE collision (a pathological overlap fails the migration loudly). WHERE (<computed valid_to> IS NULL OR <computed valid_to> > hm.effective_date) AND NOT EXISTS (… v.fact_id = hm.id).

Contracts

households.rs: add MemberFactValue { relationship } (Validate, ToSchema) + MemberClaimRequest (mirrors IncomeClaimRequest plus person_id — the URL is household-scoped, so person_id rides in the body; deny_unknown_fields); add provenance: Option<Provenance> (additive) to HouseholdMember; remove AddMember; add the Author/Provenance/VerificationSource/Uuid imports the file lacks. events.rs: MemberBeforeWindow { …, value: MemberFactValue } (full value, no PII projection) + MemberClaimedEvent (mirrors IncomeClaimedEvent plus household_id; from_claim(person_id, household_id, …)) + MemberClosedEvent (plus household_id; new(…, author: None)). paths.rs: add CLAIM_HOUSEHOLD_MEMBER/CLOSE_HOUSEHOLD_MEMBER_CLAIM, remove ADD_/REMOVE_HOUSEHOLD_MEMBER. Do not re-export MemberClaimRequest (parity).

Store

household_member_versions.rs mirrors address_versions.rs, household-scoped. Snap-person_id divergence: income/address snaps omit person_id (their ownership key, a param); here the ownership key is household_id, so MemberSnap includes person_id in its SELECT and reinsert_remnants copies s.person_id (so close_member_version needs no person_id param). snapshot_and_supersede(conn, household_id, fact_id, …) keys WHERE fact_id AND household_id. append_member_version(conn, household_id, person_id, fact_id, &req) (the new row uses the resolved person_id). list_by_household_as_of(pool, household_id, as_of) — WHERE household_id AND superseded_at IS NULL AND claim_status = ANY(determination_statuses()) AND daterange @> as_of ORDER BY fact_id (fact_id is UUID v7 = stable creation order surviving corrections; NOT recorded_at, which is per-version). No multi-person/export read needed (batch/export don’t touch members). households.rs: drop add/remove, get_with_members(as_of). models.rs: drop AddMember re-export + remove HouseholdMemberRow + its From.

As-of-aware person projection (the HIGH consistency fix)

Person.household_id must stay consistent with the as-of household reads. persons.rs: convert the shared HOUSEHOLD_JOIN (get/list_by_ids/list/list_for_export) from active = true to household_member_versions … daterange(valid_from, valid_to, '[)') @> $N (a household_join(placeholder) helper, since the JOIN is shared across queries with differing param counts); the update() inline subquery is a separate one-off (not a HOUSEHOLD_JOIN consumer). Every person read gains an as_of: NaiveDate param; handlers resolve it via resolve_as_of — today for current-state reads (get_person/list_persons/update_person/export_persons), the request as_of for the batch path (expand_persons_as_of → list_by_ids), so /full?as_of=X hydrates each nested household_id as-of-X. Make resolve_as_of pub(crate) so api/export.rs reaches it (it has two callers: persons::get + persons::list_for_export). B8 preserved (no new clock::today()). Two residual splits stay (pre-existing, by design): multi-household membership projects to one household_id via LIMIT 1; the unversioned active=true households-entity read can 404 a household the JOIN resolves.

API

Add claim_household_member (mirrors claim_income minus the negative-amount guard; for a correction require_member_ownership returns the stored person_id, the handler 422s on a body mismatch) + close_household_member_claim + persist_and_publish_member_claim + require_member_ownership(pool, household_id, fact_id) → PersonId (404 if not in this household). Convert get_household/get_household_full to pass as_of to get_with_members. Remove add_member/remove_member. Extend the shared map_claim_pg_error to map 23503 (FK) → 422 (client-supplied body person_id likely; today a 500). Update routes()/#[openapi]/the path-count assertion (19→19).

Events / audit

publish_member_claimed/publish_member_closed (routing keys household.member_claimed/member_closed; both take household_id). canopy-security parse_event_type: REPLACE the two arms (("claim"/"close", "household_member")), delete the old; the generic fact_id/household_id + nested-author extraction already applies. canopy-web humanize_event: REPLACE the keys (delete the old), keep the natural copy ("added/removed a household member").

Seed

Repoint the four tools/canopy-seed member inserts to household_member_versions, mirroring the address_versions seed block (active-only filter, origin='seed'; no inactive→closed handling — that is migration-backfill-only). Add household_member_versions to the reset/TRUNCATE list + the seed integration version-table enumeration.

Quality budgets

No new serde_json::Value (the proposed_value JSONB reuses A1’s // STRUCTURAL-VALUE carve-out; MemberFactValue typed) ⇒ B3a flat. list_by_household_as_of is a direct query (no unwrap_or_default) ⇒ B5 flat. Removing AddMember/households::add_member/remove_member/HouseholdMemberRow is a clean code-removal cluster: ratchet DOWN if a budget moves; surface-and-decide any rise (ADR-030).

OpenAPI

persons.json via api-docs --update: + MemberFactValue/MemberClaimRequest, + provenance on HouseholdMember, + the two member-claim paths, − AddMember + the POST /members + the DELETE /members/{member_id}. Path count 19→19.

Implementation

Single MR, Relates to #683 (the split work does not close the issue — Half B closes it). The (plan) commit is first (this .adoc + nav). Each commit build-green; per-commit the pre-commit token gate + a fresh J1–J8 subagent over the staged diff, reported as text.

  1. Migration — migrations/20260624000000_create_household_member_versions.sql: extension → CREATE TABLE household_member_versions → 3 indexes → backfill_household_member_versions_v1() → SELECT it. Keep the legacy household_members table (comment: frozen, dropped in the CONTRACT follow-up).

  2. Contracts — households.rs/events.rs/paths.rs per the Decisions (+ the new imports).

  3. Store — store/household_member_versions.rs (mirror address, household-scoped, snap carries person_id); register in store/mod.rs; households.rs drop add/remove + get_with_members(as_of); store/models.rs drop AddMember re-export + remove HouseholdMemberRow + its From.

  4. Events — publish_member_claimed + publish_member_closed.

  5. API — claim/close/persist/require_member_ownership + get_household(_full) as-of + remove add/remove + routes()/#[openapi]/19→19 + map_claim_pg_error 23503→422.

  6. Downstream reads (as-of-aware person projection) — persons.rs household_join(placeholder) helper + update() one-off + as_of param on the five reads + resolve_as_of pub(crate) + batch.rs/export.rs thread as_of + grep every other caller.

  7. canopy-security — REPLACE the two parse_event_type arms (no aliases) + full parse_event tests.

  8. canopy-web — audit/mod.rs REPLACE the humanize keys (no aliases); api/members.rs claim/close bodies.

  9. Seed — repoint the four member inserts (mirror the address seed block, active-only); reset list + integration check.

  10. Consumers — applications (persons_client + the 2 finalize sites); cli (cmd/household + main + cli_test); test-lib (claim_member/close_member client + the given DSL).

  11. Tests — the income/address test set mirrored for members + the per-(household_id, person_id) overlap test + backfill test (incl. active=false→closed + add→remove→re-add disjoint) + 23503→422 + person_id-mismatch→422 + the as-of-consistency suite + contracts roundtrip/snapshots.

  12. Docs — data-models/api canopy-persons; CHANGELOG (Added/Changed/Removed); master plan A2 sub-note; this plan → Done + As-built; stale-doc sweep; api-docs --update.

Verification

cargo build -p canopy-persons -p canopy-contracts-persons -p canopy-security -p canopy-test-lib -p canopy-seed -p canopy-applications -p canopy-web -p canopy-cli; set -a; source .ports.env; set +a; cargo nextest run -p canopy-persons -p canopy-contracts-persons -p canopy-security (devstack-gated). cargo xtask quality-budgets (expect flat; ratchet DOWN if the removal lowers one). cargo xtask api-docs --update + commit the persons.json delta + confirm the path-count assertion (19). cargo xtask docs plan-lint + cargo xtask check-docs. After cargo xtask dev refresh: confirm seeded members appear via GET /v1/households/{id} + /full and the canopy-web case-detail household display renders. Full pre-push battery (cargo xtask validate --skip-docker + e2e + cargo doc + k6 smoke). Load-bearing assertions: the per-(household_id, person_id) EXCLUDE rejects an overlapping membership; a correction re-tiles remnants; a close drops the fact from as-of-after reads but not as-of-before; the backfill produces the expected provenance + disjoint re-add windows; Person.household_id matches the as-of household read at the same as_of (and /full?as_of=PAST hydrates as-of-PAST).

As-built notes

Built as planned (plan commit 7f7c731 + one implementation commit), Relates to #683. The plan held; the deviations are mechanical, recorded here so the plan↔code diff stays zero.

  • MemberSnap carries person_id in its SELECT (the deliberate household-scoped divergence). Income/address snaps omit person_id (their ownership key, passed as a param). The member ownership key is household_id, so household_member_versions.rs::MemberSnap SELECTs person_id and reinsert_remnants copies s.person_id — so close_member_version needs no person_id param, and snapshot_and_supersede keys WHERE fact_id = $1 AND household_id = $N.

  • As-of-aware person projection — A2 touched the person read endpoints. To keep Person.household_id consistent with the as-of household reads (the HIGH consistency finding), HOUSEHOLD_JOIN became a household_join(placeholder) helper reading household_member_versions … daterange @> $N, and get / list_by_ids / list / list_for_export / update gained an as_of param. update() is a self-contained one-off (its inline subquery, not the helper). resolve_as_of was made pub(crate) so api/export.rs::export_persons reaches it for its two person reads via super::resolve_as_of — B8 preserved (no second clock::today()). batch.rs::expand_persons_as_of threads its as_of to list_by_ids. Two residual splits stay (pre-existing, by design): multi-household membership projects to one household_id via LIMIT 1; the unversioned active = true households-entity read can 404 a household the JOIN resolves.

  • No PII projection (unlike address A1). relationship is not PII, so the household.member_claimed/member_closed events carry the full MemberFactValue (the income pattern). publish_member_claimed carries an #[expect(clippy::too_many_arguments)] — the household-scope household_id pushes it to 8 args past the 7-arg lint (mirrors `reconstruct_provenance’s precedent).

  • Shared map_claim_pg_error gained 23503 (FK) → 422. The member claim’s client-supplied body person_id makes a bad FK an ordinary client mistake (income/address inherit the same correct mapping for a bad path person_id — a strict hardening).

  • No-backward-compat (user directive). The old POST/DELETE /members paths, the AddMember DTO, and the household.member_added/member_removed event names
    their parse_event_type/humanize_event arms were deleted, not aliased.

  • Legacy household_members table kept frozen (EXPAND only). The destructive CONTRACT drop + backfill_household_member_versions_v1() removal is a filed follow-up (kept so the backfill test runs on the ephemeral schema).

  • Quality budgets flat. No new serde_json::Value (the proposed_value JSONB reuses A1’s STRUCTURAL-VALUE carve-out); list_by_household_as_of is a direct query (no unwrap_or_default). ADR-027 unchanged — A2 fulfils §3.

Follow-ups

File each as a separate GitLab issue and /relate #683:

  • chore: CONTRACT — drop the legacy household_members table + backfill_household_member_versions_v1() once A2 has baked (forward-only migration; fold into / sibling of A1’s #890).

  • feat: household-member change-history — extend the T1-6 fact-history resource enum (+ worker UI) to members, if product needs the per-resource membership timeline (the generic audit capture already lands in A2).

  • on-behalf-of attribution for member close — MemberClosedEvent.author is None until the ADR-019 on-behalf-of plumbing lands (shared limitation with income/address close); track with the existing on-behalf-of follow-up.

Edit this page · default