Plan: ADR-001 Amendment 1 — sanctioned bulk-read contracts + reporting job model (#1235, epic &73)

On this page
NOTE

Implements the ADR-001 Amendment 1 contract (B1–B8). The ADR pins the contract + anti-pattern rule + enforcement + job model; children own the byte-level (endpoint DTOs, SQL, migrations). Governed by ADR-004 (data tenancy) and ADR-003 / ADR-011. Bulk determinations are ADR-002 / #1237 — out of scope.

Status

Step Description Status

0

File the enforcement + follow-up issues (#1249/#1250/#1251/#1252) + epic + blocker graph; revise children ACs (#1202/#1203/#1219/#1220/#1221/#1222/#1223/#1224) and #1235’s own AC; commit this plan + nav.

Done (2026-07-27) — #1235 (this MR)

1

#1235 ADR-001 Amendment 1 — the bulk-read contract (B1–B8) + anti-pattern rule + enforcement + job model.

Done (2026-07-27) — this MR

2

Enforcement hardening (#1249) — typed CompletenessRead<T> marker + consumer fail-closed on missing total_in_scope + cargo xtask lint + param-level authz gate.

Not started

3

#1203 bulk contracts — :batchGet (one round-trip) + projection-first + universe reads reuse the shipped scope-parameterized keyset.

Not started

4

#1202 async report-run model — POST→202 + report_runs SKIP-LOCKED lease + heartbeat + resume cursor + status-gated GET (the 5 cross-service extract POSTs).

Not started

5

Universe-consumer children — #1219 (ELE), #1220 (enact), #1221 (CSV/QC), #1222 (overpayments), #1224 (appeals), #1223 (persons slim).

Not started

Epic: &73
Issue: #1235 (critical)
Branch: feature/1235-adr001-bulk-read

Context

Six scale-audit findings (C1/C2/H10/M5/M11/H12, epic &73; plus the adjacent H14/#1220 + M10/#1223) share one root: canopy-reporting and background sweeps treat an interactive or unbounded read as their completeness-required universe — a LIMIT 200 page (T-MSIS covered ~200 of a multi-million Medicaid roll) or an unbounded fetch_all silently became the federal-report / legal-sweep set. ADR-001 mandates HTTP-only cross-service reads but never defined a bulk-read contract. The keyset work (#1195/#1204/#1214) fixed the mechanical truncation; still missing were the codified contract, an enforceable anti-pattern rule, first-class projection (M10: ~3–6 GB + ~2M spurious Pub-1075 events/run), and an async job model for caseload-wide reads. This amendment is the keystone the implementation children build against.

The bulk-read contract (summary — full text in the ADR)

The authoritative contract is ADR-001 Amendment 1 B1–B8; a contextless implementer reads that first.

  • B1 — One scope-parameterized read per resource. Completeness is a per-response property (presence of total_in_scope), not a dedicated endpoint; the shipped #1195/#1204 are conformant (not re-split); total_in_scope is dual-use (federal tripwire + interactive count tile).

  • B2 — Keyset-page contract. {items, next_cursor, total_in_scope}; (sortkey, id) UUID-v7 tiebreak; default 50 / max 200 shared constants; NOT CONCURRENTLY index (#1196).

  • B3 — Completeness enforcement. Typed CompletenessRead<T> marker (skipping the tripwire fails to compile) + consumer fail-closed on missing total_in_scope + a cargo xtask lint. Enforced once, at the consumer.

  • B4 — :batchGet + projection. One round-trip for N ids (#626: cap + 422, ANY($1), service-side assembly); first-class field projection (unprojected restricted fields never fetched/decrypted/audited).

  • B5 — Anti-pattern rule + (F)/(S) test. Completeness-required consumers must use a completeness read; symmetric (governs consume + expose).

  • B6 — Three patterns. keyset read / SKIP-LOCKED claim / progressing-cursor revisit.

  • B7 — Async report-run job model. 5 cross-service extracts → 202 + report_runs SKIP-LOCKED lease + resume cursor ≡ keyset cursor + status-gated GET; 3 DB-local extracts stay 201.

  • B8 — ADR-001 preserved + scoped. Decision immutable; a Status NOTE scopes "raw program data" to restricted data; surfaces the reporting-PHI/ADR-004 gap.

Keyset-only (NDJSON dropped)

NDJSON was considered and rejected: no child needs it, keyset covers every read, and a raw stream reintroduces the #1042 silent-truncation defect unless it carries a termination sentinel (plus a reqwest stream feature + idle-timeout rework). Keyset is the sole cross-service bulk-read transport; terminal own-service file/CSV exports still stream via Body::from_stream (B5).

Surfaced compliance gap — reporting PHI tenancy (ADR-004)

canopy-reporting already persists person-level T-MSIS PHI at rest (services/canopy-reporting/migrations/20260409000000_tanf_medicaid_reporting_tables.sql, medicaid_tmsis_eligibility_extracts.person_id) but is absent from ADR-004’s authorized-consumer map, which forbids exactly that. B8 states this plainly; the fix is an ADR-004 amendment authorizing reporting’s PHI tenancy (its own Pub-1075/HIPAA audit log + ADR-014 chain-v2 retention), filed as #1250 — a hard prereq (blocker) for the T-MSIS/CMS-416 PHI-extract children. The non-PHI extracts (FNS-388/ACF-199/FNS-7176-QC over non-restricted data) proceed.

Issues (finding → issue → ownership)

Role Issue Owns / action

ADR contract

#1235

this MR; own AC revised (strike NDJSON D2 + #315 precedent D9)

enforcement hardening

#1249

CompletenessRead<T> marker + fail-closed + cargo xtask lint + param-authz

ADR-004 PHI tenancy

#1250

authorize reporting as a restricted-data consumer — blocker for the T-MSIS/CMS-416 PHI extracts

bulk contracts

#1203

:batchGet + projection-first + universe reads reuse the shipped keyset; reconcile vs #1219

async job model

#1202

report_runs SKIP-LOCKED lease + heartbeat + resume cursor + status-gated GET; 5→202 / 3→201; kill silent corruption

ELE sweep

#1219

cohort keyset + households:batchGet (not per-id /full) + bounded concurrency, no lock across HTTP legs

enact sweep

#1220

conformant SKIP-LOCKED claim (B6-ii) — AC note

CSV/QC export

#1221

conformant (stream CSV + keyset QC) under B5 symmetry — AC note

overpayments

#1222

keyset (or justified date-window) universe

persons slim

#1223

generalize projection into B4 (stop being persons-only)

appeals sweep

#1224

conformant progressing-cursor revisit (B6-iii) — AC note; carved out from keyset

page-size constants

#1251

one home for DEFAULT_LIMIT=50 / MAX_LIMIT=200 (D8)

TANF work-activity

#1252

bring the #320-deferred N+1 under B4 (ANY($1))

Precedents + supersessions

  • #626 (persons:batchGet) — the load-bearing bulk-read precedent, carried into B4.

  • #1195/#1204 — the ratified keyset + tripwire references (B2/B3); conformant, not re-split.

  • ADR-025 — the HTTP-boundary ID-existence-validation precedent; not superseded by :batchGet.

  • Supersede the #320 deferral — #626 generalized the ANY($1) idiom, so the TANF work-activity N+1 comes under it (#1252).

  • Drop #315 — a TANF ACF-199/WPR chore, not a bulk-read precedent.

Files touched (this MR — docs only)

File Change

adrs/adr-001-program-service-isolation.adoc

[#amendment-1] — B1–B8 + anti-pattern rule + enforcement + job model + precedents; Status-section forward NOTE scoping "raw program data" (Decision text untouched).

architecture.adoc

ADR-001 index line gains the Amendment 1 parenthetical.

CHANGELOG.adoc

== Unreleased › Changed (Closes #1235).

plans/scale-audit-adr001-bulk-read.adoc, nav.adoc

this plan + nav entry (Scale Readiness, epic &73).

Endpoint/table Antora pages flip in the child implementation MRs, not here.

Verification

  1. cargo xtask plan-lint + check-docs clean; the Antora build resolves the #amendment-1 xref + all issue refs.

  2. CHANGELOG.adoc == Unreleased carries Closes #1235; architecture.adoc ADR-index updated.

  3. Fidelity re-read: every B1–B8 clause maps to a real shipped shape/path or a named child; no ADR-001 isolation clause weakened beyond the stated bounded widening (B8); the ADR-002 boundary is stated; not-yet-built surfaces are phrased normatively.

  4. Docs-only ⇒ no functional battery; the children carry the code + tests. docs: MR to main.

Documentation updates

  • ADR-001 Amendment 1; architecture.adoc ADR-index; CHANGELOG.adoc.

  • Prereq issues filed (#1249/#1250/#1251/#1252) + blocker links; children revised (#1202/#1203/#1219/#1220/#1221/#1222/#1223/#1224); #1235 own AC revised.

  • Plan → Archive on completion (final MR of the stream).

Open decisions

All decisions resolved. During adversarial review the initial "dedicated universe endpoints" call was reversed to a typed-CompletenessRead marker + consumer fail-closed + CI lint + param-authz (no #1195/#1204 re-split) — enforcing the invariant once at the single federal consumer beats proliferating producer topology. The reporting-PHI/ADR-004 tenancy gap was surfaced and filed (#1250) as a hard-prereq blocker for the PHI extracts. (This plan’s committed form uses Closes #1235 — correcting the draft’s "Relates to": #1235 is the ADR-amendment issue itself, fully delivered by this MR, like #1236.)

Edit this page · default