Plan: Policy Currency & Federal Source Consumption (epic &59)

On this page
NOTE

Implements ADR-031 §1 for epic &59 (parent &58). Grounding below is code-verified (2026-06-09). Issues are cut from the Status rows per ADR-013 once this plan lands.

Status

MR Description Status

MR1 (federal audit)

Bring rulesets/federal/citations.toml under cargo xtask policy audit. The file already exists (FNS COLA / FPL / SMI / CMS entries) but nothing reads it — audit only loads rulesets/{jurisdiction}/citations.toml. Add a federal source family to the audit: completeness against the rulesets/federal/*.json data files' keys, consistency (cited value == data value), staleness, schema. New CI job adr-031-federal-audit (advisory first, blocking once clean), mirroring adr-011-policy-audit.

Done (2026-06-09) — canopy-policy::federal module (validate_federal + load_data_files), Citation.value now Option (file-level/subtree citations carry none; jurisdiction + federal leaf citations enforce it as a schema check), policy audit --source all|jurisdiction|federal (default all), 5 missing file-level citations backfilled, CI job blocking from birth. See Design — MR1 findings.

MR2 (source pinning)

Add source pinning to the citation schema: optional source_sha256 (hash of the cited source file at verification time) + a sync-cache manifest. policy sync-cache (currently unpinned git clone --depth 1, xtask/src/cmd/policy.rs:148-225) records per-repo HEAD commit + per-cited-file content hashes into rulesets/{jurisdiction}/.policy-cache/sync-manifest.toml (gitignored alongside the cache). A --pin mode back-fills source_sha256 onto citations whose source_ref resolves in the cache. Schema change is additive (Option<String> — existing manifests parse unchanged).

Done (2026-06-09) — Citation.source_sha256 + hex-format schema check, canopy-policy::sync_manifest types, sync-cache writes the manifest (3 repo HEADs + per-cited-file hashes) every run, --pin back-fills via format-preserving toml_edit (comments survive). Georgia back-filled live: 134 citations pinned across 46 distinct PAMMS files. See Design — MR2 findings.

MR3 (drift tool)

cargo xtask policy drift --jurisdiction georgia: re-sync (or read the cache), compare each pinned citation’s source_sha256 against the current content hash of its source_ref, and report changed-since-verified citations grouped by source file with their manual_transmittal/effective_date context. Mechanical hash comparison only — per ADR-011/ADR-031 it NEVER edits values and the human re-verification loop (re-read section → update value or bump verified_date + re-pin) is documented in the report output. CI job adr-031-policy-drift runs it advisory (scheduled/MR-visible warning, allow_failure: true — drift against a live upstream must not block unrelated MRs).

Done (2026-06-09) — canopy-policy::drift::detect_drift (pinned-current / drifted incl. missing-upstream / unpinned-notice; 5 unit tests), policy drift re-syncs by default (--no-sync for offline), exit 1 on drift with the 4-step re-verify→re-pin runbook in the output, CI job permanently allow_failure: true. Live-proven: a mutated cache file flagged all 4 of its citations grouped under the file; restore → clean.

MR4 (reverse completeness)

Extend citation::validate() (crates/canopy-policy/src/citation.rs:172-229) with reverse checks: (a) orphaned citations — cited keys absent from jurisdiction.toml (today silently ignored); (b) orphaned federal citations — cited keys absent from the federal JSON files. Orphans are errors (the citation lies). Wire into the existing policy audit output.

Done (2026-06-09) — OrphanedCitation error in both families (jurisdiction: key absent from jurisdiction.toml; federal: dotless non-file / unknown stem / unresolvable path), exempted only via compliance/adr-031-citation-orphan-allowlist.toml (reason required). First run found 7 jurisdiction orphans: 4 stale tanf.work_requirement_* duplicates from the pre-[tanf.wpr] key move (deleted — correct tanf.wpr.* citations already existed) + 3 genuine JDM/service-code-embedded values (allowlisted), and confirmed the 1 known federal case. See Design — MR4 findings.

MR5 (annual indexing model)

Model the federal indexing cycle explicitly: a small rulesets/federal/indexing.toml declaring each family’s cutover cadence (SNAP COLA Oct 1; FPL Jan 1; SMI Jul 1; static families like the 1977-88 budgeting factors marked static). policy audit checks each federal data file’s _effective_date against its family window and flags out-of-window tables (e.g. it is FY2027 but snap-allotments-2026.json is still the newest). This converts "someone remembers October" into a finding.

Done (2026-06-09) — canopy-policy::indexing (11 unit tests) + rulesets/federal/indexing.toml (4 indexed families: snap-cola 10-01/0d grace, fpl 01-01/90d, smi 07-01/60d, cms-416 10-01/90d; 8 static files). Coverage is total: every data file must belong to exactly one family (UNINDEXED/AMBIGUOUS errors); out-of-window warns during the family’s publication-grace window, errors after. See Design — MR5 findings.

MR6 (PolicySource trait + docs)

Code the PolicySource trait ADR-011 described but never built (doc-only today): resolve_section, content_hash, cache_sync — implemented by PammsGitSource (the existing clone behavior, now pinned) and ManualSource (returns manual-verification-required). Federal stays file-based (the JSON tables ARE the source snapshot; their upstream is PDF/memo — no parser). Update ADR-011’s drift section to point at ADR-031 + this plan; document the re-verification runbook in the Antora policy page.

Done (2026-06-09) — canopy_policy::source (trait + PammsGitSource + ManualSource + shared sha256_hex; 6 unit tests incl. a file:// fixture-repo clone/pull round-trip); xtask sync-cache/manifest hashing refactored onto the trait (no behavior change). ADR-011 gains an "Amendment 2026-06-09: currency tooling as built" (hash-based drift, not table parsing; trait as code). New compliance/policy-currency-runbook.adoc (sync → pin → drift → re-verify loops, per-finding remediation, annual cycle, new-jurisdiction guidance), nav-wired. Epic &59 complete.

Design — grounded current state (code-verified)

  • Citation schema (crates/canopy-policy/src/citation.rs:36-60): value, authority, source_ref (PAMMS file path, e.g. dfcs-snap/modules/snap/pages/3617.adoc), section, manual_transmittal, effective_date, federal_citation, state_citation, verified_date, verified_by, notes. No hash/pin/effective-until fields.

  • Validation (citation.rs:172-229): forward completeness, value consistency, staleness (warning ≥365d, is_error() false only for Stale), schema non-empty. Orphaned citations are silently ignored.

  • xtask policy Action enum (xtask/src/cmd/policy.rs:26-48): Audit, AuditUnwraps, AuditLiterals, SyncCache. No Drift. sync_cache reads [policy_source].repos from jurisdiction.toml, clones --depth 1 into rulesets/{jurisdiction}/.policy-cache/ — no commit pin, no hash, no last-synced marker.

  • Federal data (rulesets/federal/): fpl-2026.json (Jan 1), snap-allotments-2026.json / snap-deductions-2026.json / snap-income-limits-2026.json / cms-416-2026.json (Oct 1), smi-2026.json (Jul 1), snap-budgeting-factors.json (static 1977-88), wic-food-packages-2026.json, cross-program-2026.json. Each carries _source/_effective_date headers. rulesets/federal/citations.toml exists with entries (FNS COLA memo, FNS Handbook 501, CMS SHO letters as `source_ref`s) but no audit consumes it — the epic’s "federal aren’t citation-traced" is stale; the true gap is audit + currency coverage.

  • Staged-enforcement precedent: adr-011-policy-audit CI job went advisory → blocking after the burn-down ("205/201 keys cited — clean"); same .rust-base job pattern for the new gates.

Design — MR1 findings (2026-06-09)

  • The federal manifest could not even parse before MR1: Citation.value was a required field, but 12 of the federal entries are file-level citations (key = data-file stem, no single value). The schema change is additive — value: Option<toml::Value> — with strictness preserved per family: jurisdiction citations and federal leaf citations without a value are SCHEMA errors.

  • Three citation granularities, discovered from the live data: file-level (key = stem, e.g. "fpl-2026"), subtree (dotted key resolving to a JSON object, e.g. "snap-budgeting-factors.pay_periods" — cites a group of values, no single value), and leaf (dotted key resolving to a scalar/array — value required + consistency-checked). The first audit run flagged pay_periods before the subtree rule existed; the rule was added rather than forcing a fake aggregate value.

  • The audit found 5 genuinely uncited data files (cross-program-2026, ele-grant-2026, ele-lapse-2026, ele-renewal-2026, wic-food-packages-2026) — backfilled in the same MR with citations transcribed from the files' own _comment/_citation headers and the plans that authored them (epic &55 Plan 2 MR4, wic-eligibility plan).

  • CI job lands blocking from birth (deviation from "advisory first"): because the findings were fixed in the same MR, there was nothing to burn down — the degenerate case of the staged pattern. adr-011-policy-audit (default --source all) also covers the federal family; adr-031-federal-audit exists so a federal failure is named in the pipeline.

  • Unresolvable dotted paths are still skipped (e.g. snap-alien-eligibility.qualified_alien_five_year_bar, whose value lives inside JDM rule content with no addressable key) — that is MR4 orphan territory, by design.

Design — MR2 findings (2026-06-09)

  • 114 distinct source_ref`s in the Georgia manifest; 46 resolve in the cache. The rest are CFR/USC/memo references used as `source_ref by non-PAMMS authorities — they stay unpinned by design (no upstream file to hash; the federal family’s currency assurance is MR5’s indexing-window check).

  • Pinning is format-preserving (toml_edit, already in-tree via toml 0.8 — no new duplicate dep): --pin appends/updates one source_sha256 line per citation block; the file’s 55 comment lines survive. Pinning is idempotent.

  • The initial back-fill pins the cache content at pin time, not at verified_date. A value that drifted upstream between the last human verification (2026-04-07 full audit) and the first pin would be captured as-is; that residual risk is bounded by the staleness check and closed permanently by the MR3 drift→re-verify loop going forward. Re-pinning after re-verification is the documented loop.

  • A .git guard protects the HEAD pin: git rev-parse walks up the tree, so a non-clone directory in the cache would otherwise silently pin canopy’s own HEAD.

Design — MR4 findings (2026-06-09)

  • The reverse check found real rot on its first run: 7 jurisdiction citations targeted nothing. Four were stale duplicates of the tanf.work_requirement_* keys left behind when those keys moved under [tanf.wpr] — correctly-keyed citations already existed, so the orphans were silently double-counting the audit’s "citations" total (218 vs 211 keys). Deleted.

  • The one legitimate orphan class is structural: a citation documenting a value embedded in JDM rule content or service code (TANF sanction/PR wire codes in tanf-eligibility.json, the PAMMS 1540 itemized self-employment method in canopy-tanf, the PRWORA 5-year bar in snap-alien-eligibility.json). These go in compliance/adr-031-citation-orphan-allowlist.toml with a written reason naming where the value actually lives — the compliance/*.toml pattern, same as the ADR-011 allowlists.

  • Validate signatures gained an orphan_allowlist parameter rather than post-filtering in xtask, so the exemption logic is uniform across both families and the library’s tests pin it.

Design — MR5 findings (2026-06-09)

  • grace_days models real publication lag (deviation from a naive hard cutover): HHS publishes the FPL guidelines weeks after Jan 1, so a hard Jan-1 error would put main’s CI red for weeks every year through no fault of the repo. Inside a family’s grace window an out-of-window table is a warning (visible in the audit output); past it, an error. SNAP COLA gets grace_days = 0 — FNS publishes in August, so there is no excuse on Oct 1.

  • Two of the year-suffixed files are static, not indexed: cross-program-2026.json (TSNAP/TMA/ELE regulatory constants per 7 CFR 273.26 / 42 CFR 435.112 / 42 CFR 435.1102) and wic-food-packages-2026.json (7 CFR 246.10 — reg-driven, not annually indexed). The -2026 suffix reflects authorship vintage, not an indexing cadence; they live in the static family alongside the JDM rulesets and the budgeting factors.

  • Coverage is closed-world: a new data file dropped into rulesets/federal/ without an indexing.toml assignment fails the audit (UNINDEXED), so the model can’t silently rot as files are added; overlapping patterns are AMBIGUOUS errors.

Design — decisions

  • Hash unit = whole source file (source_ref), not the section anchor. Section-granular hashing requires AsciiDoc structural parsing for marginal benefit; a file-level hash with the section field for human navigation is robust and cheap. A drifted file with 12 citations reports 12 findings grouped under one file — correct, since each needs re-verification.

  • Federal upstream is not parsed. FNS/CMS/ACF publish memos and PDFs, not machine-readable tables; building parsers is fragile and out of scope. The federal JSON files in-repo ARE the verified snapshot; currency assurance for federal is the indexing-window check (MR5) + audit of the citations file (MR1), not upstream hash drift. PAMMS (git-backed AsciiDoc) gets true drift detection.

  • Drift stays advisory permanently (allow_failure: true): it compares against a live upstream, so it can turn red on a Tuesday with no repo change — blocking unrelated MRs on that would train people to ignore it. It is surfaced in CI + a scheduled pipeline instead. The audits (MR1/MR4/MR5) are deterministic against the repo and follow advisory→blocking.

  • Pin data is gitignored (sync-manifest lives in .policy-cache/), but the per-citation source_sha256 is committed in citations.toml — the pin travels with the citation; the manifest is just the comparison cache.

Verification

  • Unit tests in canopy-policy for the new validate checks (orphan jurisdiction/federal, schema with/without hash) + fixture-tree tests for drift (pinned hash vs mutated cache file → finding; unpinned citation → "unpinned" notice, not error).

  • Live: cargo xtask policy sync-cache --jurisdiction georgia && cargo xtask policy drift --jurisdiction georgia against the real PAMMS cache; policy audit green on both source families before the blocking flip.

  • Each MR through the standard gate (validate + D1-D8 + force-merge squash=false); CI jobs land allow_failure: true and flip per the burn-down.

Edit this page · default