ADR-033: Constraint-Driven Generative Seeding + Journey Execution Model

On this page

Status

Accepted (2026-06-10). Extends ADR-031 (the scenario inventory becomes the seeder’s precondition registry) and ADR-032 (both corpora consume the same generative engine); depends on the ADR-027 valid-time substrate and respects ADR-028 (signed determinations cannot be fabricated — they must be produced by the engine). Re-specs #716 (seed-profile convergence). Implementation plan: generative-seed-harness; epic &61 MR4 (journey harness) depends on it.

Context

Three pressures converged:

  1. The seed-profile split (#716). Two mutually-exclusive datasets exist — the RNG default seeder and the hand-curated demo SQL with named personas (Maria HH-c8841a23, Carlos, Priya…). The split is the root cause behind a cluster of recurring failures (#595, #610, #577, #636: "reseed the right profile before X"), and the demo-gated specs that carry the only authed-surface WCAG coverage never run in the default gate. Worse, the demo-gated specs are value-coupled: they hardcode persona credentials and assert curated facts, so every dataset edit is a test edit.

  2. Epic &61 needs journey infrastructure. The scenario inventory (572 scenarios, ADR-031 §3) defines multi-life-event journeys whose starting states ("approved household with an active certification nearing expiry") no curated dataset can enumerate — the long tail is the point.

  3. Seeding bypasses the system’s own controls. SQL-injected rows skip every validation the endpoints enforce. The seeder therefore becomes a second, independent definition of "valid" — and a household with zero members, a certification ending before it starts, or a state production could never reach are all seedable. Silent seeder-vs-endpoint drift produces tests that pass against impossible worlds.

The standing requirement over all three: the harness must be as agile as canopy. Canopy’s core bet is that policy lives in data (jurisdiction.toml, rulesets/, the catalogues, the option registry). Federal and state law will change yearly; a harness that hardcodes a policy value anywhere — a threshold in a constraint, a dollar amount in an assertion — forfeits that agility and re-brittles on the next COLA. Legacy eligibility vendors cannot build this class of harness because their policy is hardcoded through the stack; the harness designed here is a dividend of the policy-as-data architecture, and it only pays out if the harness honors the same bet.

Decision

1. One generative engine, two constraint layers

The default and demo profiles converge on a single deterministic generative engine (canopy-seed, seeded StdRng — already deterministic). Generation is governed by two declarative constraint layers:

  • Domain invariants — always-on for happy-path seeding; the re-homed endpoint controls (households have ≥1 member; referential integrity; date ordering; age/category consistency; no negative money). Registered per program (auto-discovered, mirroring the per-program catalogue files), never a monolithic validator.

  • Scenario preconditions — per-scenario declarative requirements ("≥1 household: size 3, one member disabled, income in band X"), carried on the scenario inventory row (an optional preconditions extension of ScenarioEntry): the inventory is the registry of situation classes, so it is also the registry of seeding targets — one artifact, one audit.

  • Satisfiability fails loud. Constraints that cannot be met — mutually contradictory, or requiring a domain-invariant violation — are a build-failing finding, never a silent retry-until-timeout. This is itself a control: it catches contradictory test specs and invariant conflicts at authoring time.

  • A scoped chaos seam. Sad/bad/chaos-path testing deliberately violates domain invariants. The engine exposes an explicit, per-invocation opt-out (--violate <invariant-id>) so that work (future, separate concern) composes without weakening the happy-path default.

2. Constraints reference policy parameters by NAME, never by value

A precondition is income = { relation = "just-above", param = "snap.gross_income_limit", margin = 0.05 }, resolved at seed time against the target jurisdiction’s policy layer (jurisdiction.toml + rulesets/federal/ via canopy-policy). Literal policy values in constraints are forbidden — they would be a second copy of policy that drifts on every indexing cycle. This makes canopy-seed depend on canopy-policy: a deliberate coupling, stated here.

The same rule yields the oracle: a constraint spec that generates "just-above the gross-income limit" derives the expected classification ("ineligible, gross-income reason") from the same parameter reference. Generation and expectation are two views of one policy-derived spec; when the parameter moves, both move together.

3. Assertions are relational, never constant

Journey- and flow-tier tests assert invariants computed from the run:

  • Consistency — the portal’s number == the determination JWS’s number == the notice PDF’s number.

  • Conservation — continued benefits pending hearing == the pre-adverse-action allotment (two engine-produced values).

  • Monotonicity — income decrease ⇒ recomputed allotment ≥ prior.

  • State-machine — after "report change," the case is in the states policy permits and no other.

  • Derived classification — the §2 oracle, where the constraint pins the expected outcome class.

Exact-dollar arithmetic has exactly one home: the unit-tier JDM fixtures (deliberately golden, tiny, ADR-032 round numbers in the engine corpus). The inventory’s tier system already enforces the boundary; SNAP math never leaks into a Playwright spec. E2E specs MUST NOT assert generated names, credentials, household sizes, or dollar values — any random valid instance satisfying the preconditions must pass.

4. Execution model: endpoint-driven given, surface-driven when

  • A scenario’s "given" (prerequisites) is built by shared setup helpers that drive the real service endpoints — apply → screen → determine → certify — unasserted. This is not merely preferred: determinations carry ES256 JWS signatures and (post-ADR-028) input snapshots, so a valid mid-lifecycle state cannot be fabricated in SQL. Replaying endpoints is the only honest constructor, and it guarantees by construction that seeded states are reachable.

  • The "when/then" (asserted steps) drives the scenario’s target surface: the applicant portal BFF for intake journeys (intake is the behavior), the worker BFF for casework journeys. Each journey pays for its own surface; downstream journeys reach "approved" via fast service-token calls, not a portal walk.

  • Bulk background caseload (dashboard/search density) is the only raw-SQL seeding, and it is never asserted against.

  • The demo profile becomes a pinned seed of the same engine — a stable cast for human walkthroughs and screenshots — and no test asserts those instances' specifics. The committed devstack/demo-dataset/*.sql retires.

5. Time: three tiers, backdate-first, clock-last-and-gated

  1. Backdate the seed (default; zero new infrastructure). "The world has aged to state X" is constructed by creating state with past effective dates through the §4 helpers — a certification at today − 11 months is naturally near expiry under the real clock.

  2. Effective-date parameters (existing plumbing). "Time advances during the journey" passes explicit dates through the effective_date/as_of parameters the contracts already carry (persons income, program determinations, authorized-reps, notices — the ADR-027 valid-time substrate).

  3. Gated test clock (last resort). Only for decision-driving reads of wall-clock with no effective-date seam (the appeals 90-day clock and SNAP ABAWD month-counter already have clock structs to extend). Compile-gated behind a test-clock feature, devstack-only, stripped from production builds — a settable clock reachable in production is a correctness and audit-integrity hazard, so the gate is a safety control, not a convenience.

Amendment (#1561, 2026-08-24) — the interactive evaluation date is the jurisdiction’s LEGAL day. The tier-2 seam’s single stamped as_of is resolved by the orchestrator as legal_today([jurisdiction].timezone), never the UTC day: every ET evening (~19:00–24:00 local) the two calendars differ, program services select parameter sets and score ages on the legal calendar, and canopy-snap’s #1467 orchestrator pin compares the stamp against its own legal fallback. Program-service DIRECT-caller fallbacks read the same legal calendar in all four determine services — snap and medicaid from #1561, tanf and wic from #1573 (each loads [jurisdiction].timezone fail-fast at boot). legal_today rides the same gated clock, so test-clock journeys move the legal date in lockstep.

The effective-date ratchet: every NEW decision-driving time read must accept an effective date (tier 2) from birth; the tier-3 surface may only shrink. This is the quality-budgets ratchet discipline pointed at wall-clock reads, and it is what keeps decades of policy churn from re-growing the clock-fake burden.

6. Extensibility seams (built now, cheap; retrofitted later, rewrites)

  • Composable step-primitives. A journey is a sequence of reusable steps (drive-endpoint / advance-time / assert-invariant); the lifecycle state machine is data, not code. A new terminal state (a future suspension type, a new waiver pathway) is an additive step, never a harness rewrite.

  • Per-program registries. Invariants, setup helpers ("how to drive this program to approved"), and constraint vocabulary register per program and are auto-discovered. A sixth program touches zero core harness code. match program {} in harness core is forbidden.

  • Jurisdiction + program-subset parameterization from day one. The engine takes a jurisdiction and reads that jurisdiction’s policy values and deployed program subset (ADR-005/006), even while Georgia and the ADR-032 synthetic pair are the only inhabitants. Georgia’s shape baked into shared harness code is the default = georgia copy smell reborn.

  • Seed-sweep seam. CI pins one seed for speed; the harness can run the invariant suite across N seeds and report the failing seed for replay. The bridge from example-based-with-random-data to property-based testing — a flag now, a rewrite later.

7. The harness audits itself

cargo xtask scenarios audit extends to the harness’s own policy references: a constraint citing a parameter that no longer exists in the policy layer, or an election key absent from the option registry, is a finding — exactly as a dangling spec binding is today. The harness cannot silently drift from policy: retire a parameter and the harness fails loud, the same way the action catalogue fails when a bound endpoint vanishes. The ADR-031 discipline, turned on the test infrastructure itself.

8. Forbidden decisions

Recorded so review can cite them: literal policy values in constraints or assertions; persona names/credentials/sizes/dollars asserted in e2e specs; a monolithic validator or match program in harness core; a single hardcoded seed with no sweep seam; a hardcoded lifecycle state machine; SQL as a primary seed path for asserted state (forward-only migrations under ADR-016 make committed SQL datasets migration-brittle; endpoints are the schema-stable interface); Georgia assumptions in shared harness code; a test clock reachable outside the test-clock feature.

Consequences

  • canopy-seed gains a canopy-policy dependency — the deliberate coupling that buys policy-agility. The harness updates itself when values change; only structural policy changes (a new deduction type, a new program) require new constraint vocabulary, registered per §6.

  • #716’s convergence is realized: one engine, one profile axis (seed number), demo = pinned seed; the #595/#610/#577/#636 failure class (ambient-profile brittleness) loses its root cause; the demo-gated WCAG coverage joins the default gate.

  • Epic &61 reorders: MR4 (journey harness) consumes this plan’s step-primitive library and endpoint-driven given helpers; the generative-seed plan lands first.

  • Cost is real and accepted: the constraint engine, per-program registries, endpoint-driven setup library, and time-tier audit are a multi-MR build (sized in the plan). It is the foundation under all scenario testing — the 418-uncovered burndown runs through this engine.

  • The signed-determination property becomes a test asset: because determinations cannot be fabricated, every seeded prerequisite is proof of reachability — the harness exercises the real pipeline even while "just seeding."

Edit this page · default