Plan: ADR-002 Amendment 1 — async/bulk determination variant (mass-change machinery) (#1237, epic &73)

On this page
NOTE

Implements the ADR-002 Amendment 1 contract (D1–D10). The ADR pins the contract + invariants + acceptance criteria; children own the byte-level (event DTOs, checkpoint tables, SQL, concurrency). Governed by ADR-001 (isolation) and its Amendment 1 keyset shape (D5), ADR-004 (event allowlist), ADR-014 (the FTI chain accessed_by), ADR-028 / ADR-036 (the AEAD-seal-hash-then-sign ritual + signing-key retention), and ADR-019 (actor-JWT vs determination-JWS separation). Enrollment apply-semantics for an already-enrolled re-determination are #1133 — out of scope.

Status

Step Description Status

0

Claim #1237; re-point #1133 AC bullet 4 (the initial-vs-re-determination signal) at #1237 in both issues; commit this plan + nav.

Done (2026-07-27) — #1237 (this MR)

1

#1237 ADR-002 Amendment 1 — the async/bulk determination contract (D1–D10) + settled decisions + invariants + acceptance criteria.

Done (2026-07-27) — this MR

2

#1213 mass-change / October-COLA driver — the determination.requested consumer, bulk-enqueue admin surface (cohort selection, dry-run then enact), durable checkpoint/resume tables, bounded concurrency, per-dispatch stable idempotency keys, one-pending-slot skip/requeue. Decomposed into a program after external plan review: see the #1213 program plan.

Done (2026-08-16) — the program shipped: prerequisites #1467–#1472, then #1473, then the core (MR !1141, merge 68bab6c5)

3

#1133 enrollment apply-semantics — adjust-in-place vs supersede under the #1130 one-live-enrollment fence; 7 CFR 273.13 reduction-type adverse-action routing; consumes the initial-vs-re-determination signal specified in A1 (does not define it).

Not started

4

#477 make determination.completed atomic with the CombinedResult write (the D10 authoritative push edge). #477 closed the general publish_tx migration; the remaining eligibility edge is subsumed by #1471 (P5 of the #1213 program plan).

Done (2026-08-14) — delivered by #1471 (P5 atomic completion tx); the snap-side edge landed as #1473

Epic: &73
Issue: #1237 (priority::high)
Branch: feature/1237-adr002-async-bulk

Context

Scale-audit finding H6 (epic &73, 2026-07-25): a determination is only ever triggered by a synchronous, single-attempt, ~30-call HTTP fan-out per household inside one interactive request lifetime (services/canopy-eligibility/src/orchestrator.rs:1398-1412). There is no queued/bulk path and no async trigger, so a mandatory mass change — the canonical case being the annual 7 CFR 273.12(e) COLA rebudgeting of every ongoing SNAP case (canopy-policy pins snap-cola to Oct 1 with grace_days=0; GA ≈ 800K households ≈ 6.7h of saturated synchronous fan-out best-case) — has no contract-level home, and a naive bulk driver would collide with the one-pending-slot invariant (idx_unique_pending_request) and starve live interactive determinations. The bulk re-determination driver (#1213) needs the contract variant specified first so signing/verification semantics survive the async path. This amendment is that keystone; it pairs with the ADR-001 Amendment 1 bulk-read contract (#1235) — bulk determinations were explicitly deferred there to here.

The async/bulk determination contract (summary — full text in the ADR)

The authoritative contract is ADR-002 Amendment 1 D1–D10; a contextless implementer reads that first.

  • D1 — Async trigger. A new determination.requested command event (typed contracts-crate payload, ADR-004 no-PII/FTI allowlist), distinct from the determination.completed* fact events; canopy-eligibility owns producer + bulk consumer; binding-first (#1089) + write-ACL (#1122); consumer idempotent.

  • D2 — JWS byte-identical on the system-initiated path. Same boot-acquired signer + ADR-036 kid + signing_key_history + full ADR-028 seal-hash-then-sign; no separate "system" key; verify-before-accept preserved.

  • D3 — accessed_by for the FTI chain. Real originating-worker actor else a reserved system principal encoding the cohort_run/job id; ADR-019 actor-JWT vs determination-JWS separation.

  • D4 — Stable idempotency key. Deterministic hash(cohort_run_id, case/household, program, as_of/corpus); the #1003 HTTP middleware is transport-only; persistence idempotent on a natural key.

  • D5 — Checkpoint/resume. Driver-owned durable cohort-run + per-case state (renewals NOT EXISTS+ON CONFLICT), keyset cursor (ADR-001 A1 B2), per-unit state machine, single-active-run advisory lock. Table shape → #1213.

  • D6 — Retry/failure classification. Bounded retry + backoff + deadline + DLQ; 4xx terminal / 5xx transient; no synthetic pending_verification; deduped verification items.

  • D7 — One-pending-slot + interactive priority. Skip-and-requeue behind a live interactive determination (never clobber, never 409 a worker); bounded concurrency; queued/deferred representation; idempotent enqueue.

  • D8 — as_of/corpus pinning. Resolved once per cohort-run, stamped on every dispatch, folded into the key; requested_by + authz captured at enqueue.

  • D9 — Signal. Reuses the existing signed previous_determination_id (ADR-028 §57) as the supersession linkage (no new field); its None is tri-valued (first / legacy / not-yet-supersession-capable) so a typed trigger/reason enum is the authoritative initial-vs-re-determination classifier; already-enrolled re-determination blessed as a first-class outcome; completion events extended additively with the trigger classification.

  • D10 — Completion edge. 202 + request_id; determination.completed atomic with CombinedResult (#477) as the authoritative push edge; poll endpoints retained.

Boundary — what this amendment does NOT specify

The amendment governs the determination contract only. Explicitly out of scope (owned by #1133): enrollment’s adjust-in-place vs supersede-under-the-#1130-fence choice, the 7 CFR 273.13 reduction-type adverse-action routing for decreases, lifecycle_revision (#1095) fencing, current-month pending-issuance handling, and the continued-benefits / CB-on-appeal interaction. Two look-alike "409/park" invariants live at different services and MUST NOT be conflated: eligibility’s one-pending-determination-slot (idx_unique_pending_request, handled by #1213) vs enrollment’s one-live-enrollment-per-household #1130 fence (handled by #1133). The amendment may reference #1130 as the reason "emit does not imply enrolled" but does not specify how enrollment resolves the collision.

Issues (role → issue → ownership)

Role Issue Owns / action

ADR contract

#1237

this MR; owns the D1–D10 contract, the async re-determination signal (designating ADR-028 §57’s previous_determination_id + a trigger enum), the async-path JWS invariants

COLA driver

#1213

the determination.requested consumer + bulk-enqueue admin surface + checkpoint tables + bounded concurrency + per-dispatch keys + one-pending-slot skip/requeue (the byte-level)

enrollment apply

#1133

adjust-vs-supersede under #1130; 273.13 routing; consumes the initial-vs-re-determination signal specified in A1 (AC bullet 4 re-pointed at #1237)

273.13 reductions

#1002

the reduction-type adverse-action pipeline remainder (the timely-notice path a decreasing re-determination rides — D9)

atomic completion

#477

determination.completed atomic with CombinedResult (D10)

mq binding-first

#1089

the in-tree determination.requested consumer must land before the producer activates

mq write-ACL

#1122

extend canopy-eligibility’s topic-write ACL in devstack/rabbitmq/definitions.json

Files touched (this MR — docs only)

File Change

adrs/adr-002-black-box-determination-contract.adoc

[#amendment-1] — D1–D10 + settled decisions + consequences; Status-section forward NOTE (Decision text untouched).

architecture.adoc

ADR-002 index line gains the Amendment 1 parenthetical.

CHANGELOG.adoc

== Unreleased › Changed (Closes #1237).

plans/scale-audit-adr002-async-bulk.adoc, nav.adoc

this plan + nav entry (Plans, epic &73).

Endpoint/event/table Antora pages (the determination.requested contract, the async 202 shape, the eligibility 409/deferral doc flip) land in the child implementation MRs, not here.

Verification

  1. cargo xtask plan-lint + check-docs clean; the Antora build resolves the #amendment-1 xref + all issue refs.

  2. CHANGELOG.adoc == Unreleased carries Closes #1237; architecture.adoc ADR-index updated.

  3. Fidelity re-read: every D1–D10 clause maps to a real shipped shape/path or a named child; the §Decision block (lines 28-67) is byte-immutable; no ADR-002 trust clause weakened; not-yet-built surfaces are phrased normatively (MUST/SHALL). The load-bearing file refs (orchestrator.rs:1398-1412, auto_enroll.rs:143-171, idx_unique_pending_request) verified against the tree.

  4. Docs-only ⇒ no functional battery; the children carry the code + tests. docs: MR to main.

Documentation updates

  • ADR-002 Amendment 1; architecture.adoc ADR-index; CHANGELOG.adoc.

  • #1133 AC bullet 4 (the discriminator) re-pointed at #1237 in both issues.

  • Plan → Archive on completion (final MR of the stream).

Open decisions

All decisions resolved. Five user-facing calls were surfaced from the understand-phase synthesis and confirmed: (a) async transport = canopy-eligibility owns producer + bulk consumer, the consumer calls the existing POST /v1/eligibility/determine (the orchestration entry, which fans out to each program’s signed /v1/determine) — smallest ADR-002 delta, JWS path unchanged; (b) the initial-vs-re-determination signal reuses the existing signed previous_determination_id (ADR-028 §57) for the supersession linkage (no new field), with a typed trigger enum as the authoritative classifier since previous_determination_id’s `None is tri-valued; (c) an already-enrolled re-determination is blessed as a first-class contract outcome now, with the enrollment PARK/409 as the bridge until #1133; (d) a bulk member colliding with a live interactive determination defers-and-requeues (interactive always wins the slot); (e) accessed_by for the FTI chain = real originating-worker actor else a reserved system principal encoding the cohort_run/job id. The remaining nine decisions are mechanical defaults recorded in the ADR. Adversarial review corrected the initial draft’s headline supersedes framing: the signed field already exists as ADR-028 §57’s previous_determination_id, so A1 reuses (not owns) it, and the transport endpoint is the orchestrator entry /v1/eligibility/determine, not the program /v1/determine.

Edit this page · default