ADR-001: Program Service Isolation

Several federal data sources available to eligibility systems carry statutory restrictions on authorized use that cannot be satisfied by access controls alone — they require physical or logical isolation demonstrable to federal auditors:

By placing each program’s restricted data in an isolated service with its own database, Canopy makes federal audits of individual programs tractable: an IRS Pub 1075 audit of the TANF system examines canopy-tanf in isolation, without requiring explanation of why SNAP data is in the same schema.

Program policy changes — income limit adjustments, new categorical eligibility groups, work requirement modifications — can be deployed to a single program service without touching other programs. Under a shared schema model, schema migrations and deployment coordination across programs create fragile coupling.

DPH and DECAL can participate in the Canopy ecosystem without surrendering their program data to a shared platform administered by another agency. WIC data stays in canopy-wic under DPH’s effective control. CAPS data stays in canopy-caps under DECAL’s effective control. Participation means eligibility coordination, not data pooling.

Alternative 1: Shared schema with row-level security A single PostgreSQL database with per-program schemas and role-based access controls. Rejected because row-level security does not satisfy federal auditor expectations for FTI isolation, and a single database administrator has access to all program data by definition.

Alternative 2: Shared schema with separate audit database Rejected for the same reasons as Alternative 1, with additional complexity of keeping audit and primary data synchronized.

Alternative 3: Microservices with shared data layer A service mesh with a centralized data platform, as proposed in DCH’s IAPD (MEST-aligned architecture). Rejected because a shared enterprise data platform cannot satisfy the legal data isolation requirements of simultaneous Pub 1075, CMA, HIPAA, and FNS compliance. This alternative also creates dependency on a single vendor’s ecosystem and a single agency’s governance — specifically the risk that motivated Canopy’s creation.

B1 — One scope-parameterized read per resource; completeness is a per-response property. A cross-service list/universe read is ONE endpoint per resource. Called without a federal scope it is an interactive, best-effort, keyset-paginated list; called with the federal scope param (e.g. month / active_on) it is the completeness universe and carries total_in_scope. "This response is a universe" is signalled by the presence of total_in_scope (which a correct federal consumer requires, B3) — not by a distinct URL. This is deliberate: the truncation root cause is already fixed (keyset-to-exhaustion + a byte-identical COUNT), the store is already scope-parameterized, there is exactly one federal consumer (canopy-reporting), and total_in_scope is dual-use (the federal tripwire and interactive count tiles such as "N renewals due"), so a dedicated /universe endpoint would churn shipped handlers and regress those tiles for no completeness gain.

B2 — Keyset-page contract (normative; codifies the shipped lineage). Envelope {items, next_cursor: Option<Cursor>, total_in_scope: Option<i64>} — completeness reads populate total_in_scope; a purely interactive list (e.g. NoticePage) omits it. Cursor {after_<sortkey>, after_id}, id a UUID-v7 tiebreak giving a stable total order under sortkey ties; both cursor fields are required together (a lone one serves the first page). Ordering is a compound (sortkey, id) with a row-value predicate; direction is semantic. next_cursor = Some(last-row cursor) iff items.len() == limit, else None (a full final page costs one empty fetch, never a missed row). Page size default 50 / max 200 via shared named constants (#1251). Wire-exactness is contractual: timestamps re-encoded Z-suffixed RFC-3339 (SecondsFormat::AutoSi, use_z=true), dates %Y-%m-%d. Index standard: transactional CREATE INDEX IF NOT EXISTS … (sortkey <dir>, id <dir>), NOT CONCURRENTLY (#1196 sqlx-migrator advisory-lock deadlock) with the documented out-of-band-CONCURRENTLY escape hatch; the list SQL is a pub const backing a keyset index-regression test.

B3 — total_in_scope + completeness enforcement (fail-closed, #1042) — the core enforcement. A completeness read’s total_in_scope is a COUNT whose WHERE is byte-identical to the scope predicate (so page and count cannot diverge), populated on the first page. The consumer MUST loop next_cursor → None, assert pulled == total_in_scope, and fail closed when total_in_scope is absent — a federal/statutory extract that consumed a response lacking it MUST refuse to emit. Because B1 makes completeness a runtime property of one shared endpoint, a purely static lint cannot by itself tell a completeness fetch from an interactive one; therefore the contract makes completeness statically decidable via a typed marker: a completeness read yields a CompletenessRead<T> (newtype/trait) whose only constructor is the exhaustion-loop + tripwire, and every federal-extract assembly function takes that type — so skipping the tripwire fails to compile. A cargo xtask lint keyed on the marker is the backstop. Enforced once, at the consumer (#1249) — topology-independent, so it also catches a new resource whose interactive list silently becomes a universe.

B4 — :batchGet by-id bulk read + projection. Per-id enrichment of a universe’s rows is a single POST /v1/{resource}:batchGet (AIP-231 colon method; precedent #626 persons:batchGet): a server-enforced id cap (~500) + 422-over, a set-based WHERE id = ANY($1) with service-side assembly (never JSON_AGG — a decrypted field such as SSN-last-4 must be derived in-service, and JOIN-aggregating child tables cartesian-explodes). Projection is first-class: a per-request field-mask / ?view= on :batchGet and the list/universe reads — only requested fields are fetched, decrypted, and audited; unprojected restricted fields (e.g. SSN) are never decrypted and emit no access-audit event (this eliminates the M10/#1223 class: millions of spurious Pub-1075 ssn.accessed events + multi-GB payloads per run). Enrichment MUST NOT be serial per-row HTTP, and MUST NOT run under a held advisory lock or transaction spanning HTTP legs.

B5 — The anti-pattern rule + the completeness test. A completeness-required consumer MUST use a completeness read (B1 scope + B3 enforcement), never an interactive/unbounded read. A consumer is completeness-required iff (F) its output is a federal/statutory extract whose correctness depends on every in-scope row (FNS-388, FNS-7176/QC, ACF-199, T-MSIS, CMS-416, PAMMS-9000 overpayments), or (S) an SLA or legal deadline depends on processing every in-scope row (month-end enact closure, the appeals dead-action sweep, Medicaid ELE annual redetermination). If neither holds, an interactive read is fine. The rule is symmetric — it governs what reporting consumes and what it exposes (a federal CSV export streams via Body::from_stream; a QC-universe JSON keyset-paginates — neither materializes the whole set in memory).

B6 — Three distinct sanctioned patterns (shared principle: no unbounded fetch_all). (i) keyset read — one complete walk over a stable, immutable key to exhaustion (the universe reads); (ii) SKIP-LOCKED claim — a replica-shared, idempotent WORK sweep that is DB-local (FOR UPDATE SKIP LOCKED, bounded batch — not a read); (iii) progressing-cursor revisit — a continuous re-sweep over a mutable cursor (e.g. last_verified_at), which by design violates keyset’s immutable-key rule and is therefore a distinct pattern, not a keyset read. Each carries a when-to-use rule so an implementer picks correctly.

B7 — Reporting async job model (sanctioned execution for caseload-wide reads). The five cross-service universe extracts (FNS-388, QC, ACF-199, T-MSIS, CMS-416) return 202 + a run id and enqueue a reporting-owned report_runs row (status queued|running|completed|failed|cancelled, a resume_cursor, per-category skipped-row counters, a claimed_at/claimed_by lease + heartbeat_at); the three DB-local aggregations (ACF-196, WPR, CMS-64) stay synchronous. Safe multi-worker claiming comes from a FOR UPDATE SKIP LOCKED lease with expired-lease reclaim and a terminal update fenced on (claimed_by, claimed_at); exactly-once per row comes from idempotent ON CONFLICT upsert — not a global single-worker lock. The resume_cursor is the B2 keyset cursor, checkpointed atomically with each idempotent batch so a killed worker resumes. Snapshot GET reads are gated on run status — a partial or failed run is never served as complete — and a read-only extract never mutates program state (no fabricated fields, no get-or-create). report_runs and its cursor live only in the reporting database (isolation preserved). (Tracked as #1202.)

B8 — ADR-001 preserved; the Decision precisely scoped; a compliance gap surfaced. No direct cross-program DB access; bulk reads travel over internal HTTP; the FTI / SSA SOLQ-BINDEX / IEVS / HIPAA isolation map is unchanged. The Decision’s "return determination outcomes — never raw program data" is left byte-immutable; the Status-section NOTE scopes the operative prohibition to restricted data — FTI / SSA SOLQ-BINDEX / IEVS / HIPAA PHI (note: all Medicaid program data is HIPAA PHI) — so HTTP bulk reads of non-restricted program/reference data (household composition, certification windows, issuance amounts, determination outcomes — already the shipped reality) are in-bounds. This is an honest, bounded widening of the Decision’s literal text, not "no change." This amendment governs the read transport only; it does not re-classify data or authorize any service to hold restricted data.

Surfaced compliance gap (stated, not papered over). A federal extract can inherently be restricted data — T-MSIS and CMS-416 are HIPAA PHI — and canopy-reporting already persists person-level T-MSIS PHI at rest (services/canopy-reporting/migrations/20260409000000_tanf_medicaid_reporting_tables.sql, medicaid_tmsis_eligibility_extracts.person_id), yet reporting is entirely absent from this ADR’s isolation map — §Decision’s shared-services list (which asserts those services "hold no program-restricted data") does not name reporting at all, so it is neither a listed program service nor a listed shared service — and ADR-004 forbids restricted data being replicated to a service that is not an authorized consumer. So B5/B7’s mandate to build and persist those universes runs into a pre-existing, un-governed PHI-tenancy gap. This amendment records it plainly rather than deferring to an ADR-004 that does not yet cover reporting; the fix is an ADR-004 amendment authorizing canopy-reporting as a restricted-data consumer (its own Pub 1075 / HIPAA audit log + ADR-014 chain-v2 retention), filed as a hard prerequisite (blocker) for the T-MSIS + CMS-416 PHI-extract children (#1250). Until it lands, those two children are blocked; the non-PHI extracts (FNS-388 / ACF-199 / FNS-7176-QC over non-restricted data) proceed. Projection (B4) minimizes exposure but does not close the tenancy gap.

Precedents. #626 (persons:batchGet — id cap + 422, set-based ANY($1), service-side assembly) is the load-bearing bulk-read precedent carried into B4; #1195/#1204 are the ratified keyset + tripwire references (B2/B3), conformant and not re-split. ADR-025 is the existing HTTP-boundary precedent for cross-service ID existence-validation (a GET /v1/{entity}/{id} → 200/404 buffer_unordered validation fan-out) — a precedent, not superseded by :batchGet (validation vs data enrichment are different categories). The #320 deferral ("a bulk WHERE … = ANY($1) query if profiling later flags it") is superseded — #626 has generalized that idiom, so the deferred TANF work-activity N+1 is brought under it (#1252).