ADR-038: Concurrency-Safe, Recoverable Applicant Finalization

Each finalize-tagged persons write records a finalize_receipts(operation_id, generation, step_key) row in the same transaction as the entity it creates and the outbox event it stages. operation_id is the reserved application id; step_key is a deterministic StepKey (person(0), household(), member(i), income(j), …). On ON CONFLICT (op,gen,step) DO NOTHING the handler reads and returns the stored stable id, so a replay is a no-op that yields the original entity. The stable id is the correction-surviving id (fact_id for facts, person_id/household_id for those), never a version_id. Because the receipt, the entity, and the event commit atomically, there is no crash window in which the write happened but the idempotency record did not.

finalize_operation_generations(operation_id, generation, state active|cancelled) records the live generation. Every finalize-tagged write, in its own transaction, takes FOR SHARE on the generation row and refuses (409/410) if it is absent or cancelled — closing the "a write from a compensated attempt lands after compensation" race. The generation is bumped on an aborted re-submit, so a fresh attempt’s receipts/tags never collide with the aborted attempt’s, and the aborted attempt becomes a new filing (fresh received_at/valid_from) — the aborted attempt created no application, so the successful re-submit is the filing.

A durable finalize_operations row (in_progress → completed | aborted, via compensating) holds the pinned basis_date, received_at, keyed request_digest, a lease (lease_holder + lease_expires_at), and — at completion — the household_id. claim_or_resume is one short transaction that locks the draft row FOR UPDATE (serialising with the ADR-026 §6 reaper) then the op row FOR UPDATE, and branches on the locked state: fresh insert, return the completed response, refuse a live/compensating op (503
Retry-After), steal an expired lease, or bump the generation on an aborted re-submit. Row locking — not a snapshot CTE — makes concurrent claims linearizable.

No lock or transaction is held across a network call (#1005 criterion g): the lease is a heartbeat’d row value, and the only transaction is the network-free final commit that inserts the application, deletes the draft, and marks the operation completed — all lease-fenced, so a stolen lease cannot double-complete. On a genuine 23505 the authoritative applications.household_id is read back and returned.

request_digest is a keyed HMAC over a deterministic canonical serialisation of the full typed FinalizeRequest, pinned per (operation_id, generation) and re-validated on every resume (mismatch ⇒ 409). Keyed ⇒ not offline-guessable from the PII it covers. This closes the "resume with an edited or reordered request skips already-receipted steps and runs the rest against different data" hole; a legitimate portal resend recomputes the identical digest.

A partial graph is undone by shredding each exclusively-owned entity’s DEK (ADR-036 tombstone) and deactivating it — under lock_fact(fact_id) / a person_id advisory lock, scoped to the dek_id captured at inventory (a new shred_with_dek_id variant, never shred_with(subject_kind, subject_id), which would match every live DEK for the subject and could destroy a later legitimate correction). An entity that cannot be proven exclusively owned under the lock (a fact with a later non-finalize version, a person in another active household) is terminally quarantined — left intact, recorded for a data steward — and never blocks the operation reaching aborted. Hard deletion is impossible by construction (`redaction_keys’ one-way trigger; version-row FKs) and is not attempted.

The persons event_outbox gains hold_operation_id / hold_generation; the drainer skips held rows (AND hold_operation_id IS NULL). Finalize graph events are staged held and released (un-held) only after the application commits, or dropped on compensation. Downstream therefore never observes the events of a partial or compensated finalize. The hold does not order events across services: application.submitted (the applications outbox) and the released persons events have no guaranteed relative order, as for all cross-service events on the bus. Because the persons rows are committed synchronously before either event drains, a consumer can always resolve a referenced entity by a synchronous lookup even if it has not yet seen that entity’s event; finalize-graph consumers are verified order-independent at MR0 and any order-dependent consumer is escalated, not silently relied upon.

The register / release / cancel / get internal endpoints — one of which can shred PII — are gated by require_service_caller()? then claims.service_id() == Some("canopy-applications"). The role-derived check follows the coarse one because service_id() falls back to azp, so gating on it alone would admit an azp-only OIDC client that require_service_caller rejects — a weaker trust class on a shred-capable endpoint. This is the codebase’s first specific-caller allow-list. Existing-orphan remediation (MR9) is gated to require_data_steward().