Plan: Exchange partner architecture + Gateway-derived interface mocks (epic &79)

On this page

Context

canopy’s partner surfaces are stubs and fabricated-data adapters. Epic &79’s investigation produced two fused evidence bases — the source-derived interface catalog and the 7.4 INT design-document review — establishing that wire-faithful mocking is achievable for most of the estate. Reproducible from the pinned review artifact: 44 of 51 documented packages carry at least one operation with a full field-level layout, and 107 of 168 operations enumerate their reason/disposition code values.

ADR-045 records the architecture this plan implements: three layers (canonical contracts / grant-checked aggregation + routing / per-partner wire connectors) with canopy-exchange as a blind broker — recipient public keys only, every durable or returned evidence object a signed, recipient-sealed HPKE envelope it cannot decrypt.

Outcome: partner integrations that are real code against real formats, tested end-to-end against a devstack simulator, so a live deployment is credentials
endpoints + the transport-hardening gate — never a re-modeling.

Evidence provenance (pinned)

Connector children derive their wire facts from these artifacts. The vendor-derived inputs are not in this repository; they are archived locally and hash-verified by the generator at run time (fail-closed; --allow-unpinned to regenerate from different inputs).

Artifact sha256

Source sweep round 1 (gateway-inventory-round1.json)

d252cf0240c09109f346a22a67159a71c0f4c16676352eebcad6ea9e1bf44440

Source sweep round 2 (gateway-inventory-round2.json)

ab830d66c21d6aac35c44a55fdff24ecf55150115a4ee9eecd358c1bbab6e813

7.4 INT design-doc review (interface-docs-review.json, 51 packages / 168 operations)

bfd400b21b74cb560c440f1ee341295c1b52116798738251737113246d704229

Catalog generator (docs/tools/gen-gateway-catalog.py, in-repo)

regenerate: see the header comment; census self-check must exit 0

Gateway worker-portal checkout

commit e222bd1026b4a5857295522636ed63957a2e9492

Gateway customer-portal checkout

commit 3e4932825ae3e32e6b9af4c09682b61c0a643dcc

Divergence adjudication. Design docs win on semantics and code values; Gateway source wins on wire bytes and names; where both are internally inconsistent, the connector implements the source XSD/BeanIO literal and the simulator asserts it. Record the adjudication in the connector’s module docs.

Sanitization rule for fixtures and code tables. Interface FACTS only — field names, layouts, code values, cadences, ack semantics. Never: credentials, concrete hostnames/IPs, MFT or filesystem paths, personal names/emails/phone numbers, or the identity of secret-bearing vendor files. Fixtures use injected synthetic namespaces and endpoints; each checked-in code table records its provenance (package + document) in a header comment. Auth is modelled by shape (OFX signon block, WS-Security token, hub token) with fake devstack-only credentials.

Design

See ADR-045 for the normative architecture: the three layers, the envelope protocol (HPKE seal + Ed25519 sign, authenticated header), the two independent registries (grants — legal; keys — crypto, never authorizing), the three exchange functions (live query facade, batch relay, origin-durable command routing), program-local matching via the persons identity projection
match tokens, the opt-in per-program response cache for metered sources, the operational-only bus events, and the transit-capability compliance gate.

Non-goals recorded here so children do not re-litigate them: signed/versioned deployment-config manifests (compiled legal maxima carry the invariant); SFTP/SSH transfer-auth fidelity in the simulator (the shared volume tests parser and protocol fidelity only — a documented limit); performance and backpressure suites before real connectivity exists.

Scope by phase

  • Phase 0 (done) — land the interface catalog + generator + publishability gate.

  • Phase 1 (this plan’s MR) — ADR-045, this plan, GitLab restructure.

  • Phase 2 — foundations — envelope protocol, compliance transit gate, L1 contracts, L2 registry + live-query facade, persons identity projection, batch-relay core, recipient-ingest kit, scheduler, simulator scaffold, command-routing core.

  • Phase 3 — SNAP-first dialects and cutovers — GDOL, SDX, BENDEX, SVES/SOLQ, SAVE, then the SNAP cutover and the verification rip-out.

  • Phase 4 — remaining estate — ingress layer, eDRS, NDNH/New Hire, Work Number, NAC/PARIS, EBT commands + EBTAS dialect, incarceration, dedup policy, catalog enrichment, observability baseline.

Children

Each child is one MR. Acceptance criteria below are the issue bodies' source; every child additionally satisfies the standing bar: SPDX headers, typed errors, no unwrap/expect/panic, quality budgets, cargo xtask validate, Antora doc updates in the implementing MR, and a CHANGELOG == Unreleased entry.

Universal criteria for connector/dialect children

Every dialect child (11–22, 24, 25) satisfies all of:

  1. Simulator personality implementing the partner’s documented wire dialect, selectable by source_id, serving deterministic fixtures keyed to seed personas.

  2. Exchange L3 connector: parser + builder, mapped to L1 kinds, registered with compiled legal maxima (classification, authority, maximum program set) and capabilities.

  3. Proptests are mandatory on every parser/builder (round-trip, malformed input, boundary widths, code-value coverage).

  4. Golden fixtures are source-derived bytes committed independently of the connector/simulator code; the connector must not be the fixture’s author.

  5. Cross-process integration test: connector ↔ simulator over the real transport (HTTP or file volume), asserting field-level fidelity and the documented code values.

  6. Negative paths: malformed payload, auth failure, timeout, partial file, unknown code value — each asserted, none silently swallowed.

  7. Hostile-input controls active: XXE disabled, size/record caps, filename path-traversal guards, checksum/control-total validation.

  8. Divergence adjudication recorded in module docs; sanitization rule honoured for all fixtures and tables.

Child acceptance criteria

# Child and acceptance criteria

1

Envelope protocol + key registry. New module in canopy-crypto-shred: HPKE base mode (RFC 9180 DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-256-GCM, pinned RustCrypto hpke) + an ECDSA P-256 detached JWS signature over the RFC 8785-canonicalized inner envelope, produced through the existing canopy-signing crate (alg: ES256) — the estate’s single signature primitive; introducing a second algorithm would need its own ADR. Authenticated header per ADR-045 (and note in the module docs that the header is AAD, i.e. cleartext at rest). Zeroizing, non-Debug, non-Clone key types. Key-registry enrollment API: proof-of-possession challenge under ADR-019 identity, key id = fingerprint(jurisdiction, service, public key), duplicate material rejected, append-only audited activation/revocation, rotation with bounded decryption window. Tests: RFC 9180 known-answer vectors; forgery (seal without valid signature) rejected; replay (duplicate delivery id) rejected; key substitution rejected; every authenticated-header field mutation rejected; suite/version downgrade refused; unknown/retired key id refused; rotation overlap accepted then expired; truncated/malformed input rejected; proptests for seal/open round-trip and AAD tamper. Security-operations runbook for key provisioning + rotation. Judgment-protocols crate comparison recorded in the MR (why the pinned hpke crate over age and hand-composition).

2

Compliance transit-capability gate. Extend compliance/data-tenancy-authorisation.toml schema and xtask/src/cmd/compliance.rs with a transit capability: protected patterns permitted in a transit service’s code, forbidden in its migrations/columns; queue/cache schemas type-enforced to opaque envelope blobs. Exchange rows added in this MR, with enforcement live — never to the blanket authorised_services lists. Reconcile the ADR-004 isolation map with the matrix as an ADR-004 revision; the three known SOLQ/BINDEX discrepancies to settle: the matrix authorizes canopy-medicaid where the map omits it; the matrix authorizes canopy-verification where the map grants verification only SAVE + Death Master File (lapses with child 17); and CHIP appears in ADR-004’s source table but in neither the map’s service rows nor the matrix. Tests: a plaintext protected column in an exchange migration fails the audit; the same pattern in exchange parsing code passes; existing services' verdicts unchanged.

3

L1 contracts crate (crates/canopy-contracts-exchange). EvidenceQuery (subject enum beyond PersonId, optional per-subject identity block, non-empty kinds, purpose, correlation, requested sources, history window); EvidenceResult (per-source SourceOutcome, tagged EvidencePayload, Completeness); Provenance (source, legal authority, classification, verification tier, record ref, connector + schema versions, received/as-of); workflow contracts for stateful partners (SAVE, eDRS); command contracts. SsaBenefitEvidence is verdict-complete — every field of today’s SolqRecord (SSI-active, COLA loss, benefit category, disability onset, disabled-child loss) survives. Newtypes for SourceId, ProgramPurpose, ClassificationFlags, LegalAuthority, VerificationTier — no parallel stringly-typed identities. Tests: serde round-trips, kind↔payload type-safety (an income payload cannot inhabit an identity envelope), proptests on the wire shapes.

4

L2 grant registry + live-query facade. Grant registry keyed by source × jurisdiction × recipient service × benefit program × purpose × authority; compiled legal maxima per connector with boot-time validation (config may narrow, never broaden). Live-query facade: ADR-019 caller identity mapped to program/purpose server-side, grant check, capability dispatch, in-memory merge, one signed sealed envelope response. Blind per_program response cache: HMAC fingerprints, TTL sweep, same-source invalidation, no caching of errors/no-hits, attestation block required for classified sources. OpenAPI: ApiDoc registered, service added to the api-docs census, snapshot committed, cargo xtask api-docs drift check green against a live exchange. Compose: exchange added to every program profile with ADR-005 degraded behavior documented; snap-only profile boots. Closes #1497 (the Georgia-literal IEVS source dispatch dies with the registry). Tests: cross-program authorization denial; caller cannot assert another program; config-broadening rejected at boot; cache hit/miss byte-shape parity; fingerprint excludes nothing result-affecting.

5

persons identity projection + match tokens. Audited, service-scoped internal endpoint exposing full SSN/name/DOB for authorized evidence purposes (closes #1496); match-token facility (keyed HMAC over normalized identity) so program services can match batch records locally. Exact normalization rules specified and tested. Every access audited with actor, purpose, elements. Tests: unauthorized caller denied; audit row per access; token stability across normalization-equivalent inputs; no token collision on the seeded corpus.

6

Batch-relay core. Parse → seal-per-entitled-recipient → enqueue → delete-file pipeline; delivery-scope enforcement (full_file vs matched_only); forensic quarantine sealed to a keypair whose private half is held by the deployment’s security function and is never deployed to exchange (asserted by a test: exchange has no path that opens a quarantine entry), with audited opening; push delivery with recipient-signed receipts; ciphertext-queue redelivery; uniqueness key (source, file checksum, record ref, recipient); per-source expiry policy, always alerted. Fault matrix as tests: crash before enqueue, crash mid-recipient-fan-out, crash after enqueue before delete, recipient commit before lost ack, duplicate file checksum, control-total mismatch after partial parse, forged receipt, late receipt after expiry, reconciliation without duplication.

7

Recipient-ingest kit. Shared crate: verify signature → decrypt → match via tokens → persist → audit → acknowledge, all in one transaction; wrong-recipient envelope rejected; unknown key id rejected. Adopted by canopy-snap first. Match outcomes defined and tested: zero / one / multiple / ambiguous; uncertain matches never drive adverse action; nonmatch leaves no residue in rows, logs, or events (but IS audited as an access).

8

Scheduler. UTC cron per source; DB lease with fencing; overlap policy skip + alert; audited manual trigger (created → pending/deletable → processed) mirroring the documented Gateway capability; event-driven triggers from canopy domain events. Tests: two schedulers, one runner; DST boundary correctness (UTC); missed-window behavior explicit; manual trigger authorization + reason recorded.

9

Partner-simulator scaffold. tools/canopy-partner-sim: dedicated Dockerfile build target (binary + tables + non-root ownership + volume permissions explicitly added — the existing image copies a fixed binary list), refuses to serve unless development mode, watermarked responses, personality framework keyed by source_id, shared partner-files volume with separate read/write inboxes, code-table data files with provenance headers. Compose service in the devstack profiles. Tests: production image does NOT contain the binary; the binary exits non-zero outside development; watermark present on every response.

10

Command-routing core. Origin-durable command contract (origin service owns the record and resubmits; exchange transforms per attempt in memory); UUIDv7 command id, idempotency key, actor; state machine submitted → accepted | rejected | unknown → confirmed; ack correlation; reconciliation surface for unknown. Routing uniqueness (kind, jurisdiction, direction, operation); more than one enabled route is a boot error. Tests: replay-stable idempotency across restart; duplicate ack; out-of-order ack; unknown never auto-resolves.

11

GDOL wage/UBI dialect + connector — SOAP (the documented getWages schema) + batch legs; universal criteria.

12

SSA SDX dialect + connector — 338-field fixed-width, three cadences (daily incremental / monthly recon / annual COLA), trailer reconciliation counts computed, the documented code tables checked in; snap + medicaid ingest adoption; universal criteria.

13

SSA BENDEX dialect + connector + ingest adoption; universal criteria.

14

SVES/SOLQ dialect + connector. Compiled program set {tanf, snap, medicaid, chip} per the tenancy matrix. Consumption relocates program-side: canopy-medicaid queries exchange during its own determination and snapshots the result per ADR-028; canopy-eligibility’s pre-dispatch fetch, ApplicationContext.ssa_solq, and the person-id-as-synthetic-SSN substitution are deleted in this MR (ADR-034 debt retired; ADR-002 affirmed). solq.rs’s incorrect "Medicaid-scoped" comment corrected or deleted with the module. Outcome mapping asserted: `Partial, Unavailable, TimedOut produce provisional/manual-review results, never today’s None → false gate. Universal criteria.

15

SAVE workflow contract + dialect + connector, including the named consumer migration that makes verification’s SAVE route deletable in child 17; universal criteria.

16

SNAP cutover. canopy-snap consumes exchange for live {GDOL, SOLQ-under-SNAP-purpose} plus locally stored batch-fed {SDX, BENDEX} — the batch sources have no live per-person index by construction, so the cutover waits for children 12 and 13. services/canopy-snap/src/verification_client.rs deleted; provenance columns migrated; the IEVS-discrepancy projection that replaces the ievs_hits producer lands here so the retained verification workflow endpoint keeps its data. Persona migration matrix (every existing persona and source outcome mapped to canonical behavior) reviewed before any snapshot re-anchor; historical determination-snapshot fixtures proven readable with stable hashes.

17

Verification rip-out. Delete internal ievs/save/ssa routes, the shared IEVS DTOs and #[serde(flatten)] compat, noop.rs, noop_save.rs, noop_solq.rs, scripted.rs, guard.rs, the noop-adapters feature, and AdapterSelection — all in one compilation-atomic MR (consumers were cut over in 14/15/16). Existing ievs_hits / raw-payload data migration
retention decision executed. Doc census: Dockerfile note, services.adoc, configuration-reference, security, authorization inventory, federal-requirements mapping, RBAC, ATO-readiness, roadmap, and the FFE account-transfer pages.

18

Partner ingress layer. Authenticated inbound endpoints for Gateway-as-server surfaces and vendor callbacks; replay protection; correlation to originating runs/commands; durable processing. Simulator drives the callbacks. Universal criteria where a dialect is involved.

19

FNS eDRS workflow + dialect + connector — stateful multi-operation workflow (query, add, modify, delete disqualification); universal criteria.

20

NDNH + GA New Hire dialects + connectors; universal criteria.

21

Work Number dual personality — the OFX-era flavor and the REST/JSON successor as two personalities under one partner; universal criteria.

22

NAC + PARIS dialects + connectors; universal criteria.

23

EBT commands. Typed create-account / issue / expunge on the command-routing core, balance as a query; canopy-enrollment adopts them and keeps the issuance ledger (settlement, expungement policy). Replay-stable idempotency preserved from the current EbtAdapter bar. Tests: crash/replay produces one issuance; duplicate and out-of-order acknowledgements; ledger state transitions on accepted/rejected/confirmed; unknown blocks until reconciled.

24

EBTAS file dialect — issuance file out, drawdown file in as a source-derived reconciliation signal (per the design docs, not an invented vendor ack), address sync; needs the ingress layer; universal criteria.

25

Incarceration kind — GA DOC + SVES prisoner-match dialects and connectors; universal criteria.

26

Dedup/conflict policy across same-kind sources. Blocks enabling any additional same-kind source beyond the status-quo set. Defines precedence, conflict surfacing to workers, and what a determination may consume.

27

Catalog enrichment + coverage-gap completion. Fuse the 7.4 INT design-review findings into the catalog (documented layouts, code tables, divergence notes); close the coverage gaps the page’s own section enumerates (JAX-RS inbound registry, SMS and SMTP channels, the worker-portal QAS ProWeb second deployment, IQ/CV outbound stubs) or record each as explicit, owned debt.

28

Observability + runbook baseline. Per-source outcome and latency metrics carrying no person or delivery identifiers; oldest-undelivered and receipt-age gauges; scheduler lease health; config/key-change events; simulator-activation signal; per-source kill switch (registry enabled) with an operator runbook. Blocks the first connector cutover (child 16).

29

Real-connectivity hardening (T4, partner-blocked). mTLS, endpoint allowlists + SSRF controls, PGP-at-rest, SSH host-key and credential rotation, vendor quota/cost handling, HA scheduler. Real restricted connectors are compile-gated on this feature set — enabling one without it is a boot error.

Dependency spine (encoded as GitLab blocked-by links at filing): children 1, 2, 3 → 4 → {5, 6, 7, 8, 9, 10} → dialect children → cutovers 16, 17. Child 28 blocks 16. Child 26 blocks enabling any second same-kind source. Children 23 and 24 need 10 and 18. Child 27 follows the catalog MR. Per-cutover deploy order: exchange + keys first, recipients enroll, then the caller flips (ADR-016 expand/contract for wire-shape changes).

Status

Step Description Status

0

Catalog page + generator + publishability gate landed (MR !1187, merge 8bf6e8c0)

Done (2026-08-22) — merged

1

ADR-045 + this plan + nav + CHANGELOG (MR !1188, merge eb312d60); GitLab restructure: children #1531–#1559 filed with acceptance criteria + blocked-by spine + epic links (read-back verified), epic &79 amended, #1527 closed with receipts. Tier ruling recorded: children are T5 New Features — implementation starts after the existing backlog burns down (#1559 is T4, connectivity-blocked)

Done (2026-08-22) — see epic &79

2

Foundations — children 1–10

Not started

3

SNAP-first dialects + cutovers — children 11–17

Not started

4

Remaining estate — children 18–28

Not started

5

Real-connectivity hardening — child 29

Blocked (partner connectivity does not exist; T4)

Verification

  • Phase 1 (this MR): cargo xtask docs plan-lint; cargo xtask check-docs; Antora build in the CI-pinned container (antora/antora:3.1.14 with the .gitlab-ci.yml extension pins) with no new diagnostics for the added pages; full pre-push battery. The tenancy matrix is deliberately untouched here — it changes in child 2, with the enforcement.

  • Per child: the criteria above plus the universal bar; cargo xtask validate; for API-bearing children, cargo xtask api-docs drift-clean against a live service.

  • GitLab state: children exist with the acceptance criteria above as their issue bodies, weights, milestone, labels, and real blocked-by links; epic &79’s task list carries actual IIDs; #1527 closed with both SHAs and a per-criterion checklist.

Edit this page · default