ADR-036: Crypto-Shred Redaction & Signing-Key Retention

On this page

Status

Accepted (2026-06-25)

Realized by epic &56 T2-6 (#687), MR1–MR9 all merged; see the T2-6 plan. One as-built deviation from the proposed design is recorded inline in §5 (the audit-event redaction model is the single-owner shared per-fact DEK, not a cross-service fan-out — the fan-out is an antipattern under ADR-004/ADR-001).

Amends

  • ADR-027 §8 deferred a true purge of erroneous/expunged PII to "Track 2: a genuine purge via crypto-shredding … across facts, events, and snapshots, with an explicit Pub 1075 access-audit story for SSN." This ADR realizes it.

  • ADR-028 — the snapshot’s value leaves become sealed; snapshot_hash becomes required; schema_version advances to 4 (legacy plaintext formats dropped).

  • ADR-014 — the audit chain’s v1 hash + hash_version selector are retired (v2 is the sole formula); sealed before/after are hashed as ciphertext, preserving tamper-evidence over redacted values.

  • ADR-017 — the existing CANOPY_ENCRYPTION_KEY is reused as the per-service Key-Encryption-Key (KEK); SSN moves from the direct-KEK ssn_encrypted column to a per-value sealed envelope so it can be shredded independently.

Context

Append-only facts (ADR-027) + immutable signature-bound determination snapshots (ADR-028) + a tamper-evident audit chain (ADR-014) make a plain DELETE of an erroneous or court-expunged value either impossible (immutability triggers) or chain-breaking (deleting a hashed value rotates every downstream hash). ADR-027 §8 named the resolution — crypto-shredding — but deferred it. Separately, the JWS verification path retains rotated keys only in env vars and the rotation runbook deletes the previous key 30 days after rotation, so a determination signed with a retired key can never be re-verified — a gap for appeals/QC horizons that can be years long.

This is pre-1.0 with no production data, forward-only migrations (ADR-016), and a re-seeded devstack, so the realization collapses to a single canonical format and rips out the back-compat scaffolding rather than carrying legacy variants.

Decision

1. The crypto-shred envelope (hash-over-ciphertext)

A PII-bearing value is sealed in a SealedValue { v, alg, dek_id, ct } (the canopy-crypto-shred crate): the plaintext is AES-256-GCM-encrypted under a per-value Data-Encryption-Key (DEK); ct is base64url(nonce || ciphertext || tag). The SealedValue is the unit a canonical hash covers (RFC 8785 via serde_json_canonicalizer since #1281 → the hash is over ct). Redaction = destroying the DEK, never touching the SealedValue: the ciphertext + every hash over it (a snapshot’s snapshot_hash, the audit chain) stay intact and keep verifying — only the plaintext becomes unrecoverable.

Seal once; never re-seal on a read path. AES-GCM uses a random nonce, so re-sealing would change the ciphertext, change the hash, and break the signature. The envelope exposes open but no in-place re-seal; a value change is a new sealed envelope (the bitemporal fact model already appends, never mutates). This invariant is property-tested (seal → hash → shred → hash-stable → open-fails) and is a code-review checklist item for every sealing MR.

2. Key hierarchy: random per-value DEK wrapped under the service KEK; AAD-bound

The KEK is the existing per-service CANOPY_ENCRYPTION_KEY (ADR-017) — its fail-closed loader and EncryptionKeys{current, previous} rotation window are reused; no new env var. Each sealed value gets a fresh random 32-byte DEK (OsRng, held zeroize::Zeroizing), AEAD-wrapped under the KEK and stored in the service’s redaction_keys table. DEKs are independent random keys, not HKDF-derived from a shared secret, so destroying one reveals nothing about its siblings — the standard crypto-shred construction.

Both encryptions bind Additional Authenticated Data: the value-seal binds v:alg:dek_id; the DEK-wrap binds dek_id:subject_kind:subject_id. A swapped redaction_keys row therefore fails the auth tag — no confused-deputy/ciphertext-swap across values.

3. DEK granularity = redaction granularity

One DEK per fact version-row; per PII column per person for persons-table PII (ssn and date_of_birth get separate DEKs so one redacts without the other); per audit-event; per determination for snapshots (a frozen legal artifact is expunged wholesale). This bounds the redaction_keys row count and matches each redaction operation’s natural unit.

4. redaction_keys per-service store; shred = one-way tombstone

Each sealing service owns its own redaction_keys table (ADR-001). Shred overwrites wrapped_dek to a zero sentinel and sets shredded_at; a dedicated one-way-tombstone trigger permits only the INSERT and that single transition, rejecting any other UPDATE, any DELETE/TRUNCATE, and un-tombstoning. Redaction is idempotent (WHERE shredded_at IS NULL).

5. The redaction operation + Pub-1075 SSN access audit

Redaction is privileged + irreversible: a dedicated canopy:redact / data-steward role behind the #632 gate, a mandatory reason, the actor sub captured. It emits a plaintext-free *.redacted audit event that chains into the ledger.

Audit-event redaction = the shared per-fact DEK, not a fan-out (as-built, MR9). The proposed design had canopy-security re-shred a separate audit-event value-DEK on a fact.redacted fan-out. That is an antipattern: canopy-security would have to own a second copy of the value-key, which it can only obtain by receiving plaintext to re-seal (violating ADR-004) — and a second key turns redaction into a delivery-dependent distributed transaction (PII survives in the audit copy if the fan-out is lost). The realized model is single-owner: the persons store seals each fact event’s before/after PII leaves under the same per-fact DEK as the at-rest value (the envelope copied verbatim, never re-sealed), so the canopy-security audit copy is ciphertext under that one DEK. Redacting the fact tombstones that single DEK and the at-rest and audit-ledger copies become unrecoverable together — atomically, with no fan-out. canopy-security holds only sealed ciphertext + public keys, never the DEK (ADR-001), so the audit ledger is plaintext-free; its change-history renders a (sealed) marker for value leaves (the figure is read from the system-of-record, canopy-persons — follow-up #920).

Every SSN open (not just redaction) emits a plaintext-free ssn.accessed audit event (actor_sub, person_id, an enum purpose ∈ {case_view, search, batch_lookup, foia, portability}, source_service) at each of the seven persons SSN-decrypt sites (the persons_to_wire callers: create/get/list/update, batchGet, household-full, FOIA/portability export), per ADR-027 §8’s Pub-1075 story, obeying ADR-004 event scrubbing; a redacted SSN reads None (no plaintext open) and fires no event. Two-person integrity for expungement is a documented requirement, deferred to a follow-up (no approvals surface yet).

6. JWS verification-key retention

A persistent, INSERT-only signing_key_history table (canopy-security) records every signing public key by a stable program-bound kid. Each program derives its kid from the key itselfcanopy-{program}-{first-16-hex of SHA-256(public_key_pem)} — and registers its current public key on boot (idempotent: same key ⇒ same kid ⇒ ON CONFLICT (kid) DO NOTHING). A key-derived kid is collision-free by construction: a rotated or regenerated key automatically gets a new kid, so a key can never be silently shadowed by a stale registration under a reused slot name (the failure mode of an operator-supplied canopy-{program}-current). "Current vs. retired" is derived from registration order, so no mutable retired_at is needed and the table stays append-only.

The VerifyingKeyRegistry verifies async: it extracts the kid, rejects one whose program prefix ≠ the verifying program (defeats forged/cross-program kids), tries its in-memory keys, and on a miss lazy-loads the public key by (program, kid) via an injected KeyHistoryProvider (the orchestrator’s HTTP client against canopy-security’s JWKS endpoint). Old determinations stay verifiable forever; the env-var CANOPY_VERIFY_KEY_*_PREV dual-key mechanism is retired (the zero-downtime rotation window survives, sourced from the store).

Since #1232 the orchestrator wraps its provider in canopy_signing::memo::MemoizedKeyHistory — the cache this Decision originally shipped without: known kids cache for the process lifetime (key material is immutable per kid), unknown kids negative-cache with a bounded re-probe TTL (default 30s), fetch errors are never cached (an outage keeps failing loudly, not silently), and misses single-flight so a rotation-skew stampede costs one fetch per unknown kid per process. No periodic refresh loop: the lazy fetch is the admit path, so the skew window closes on the first post-TTL probe without a redeploy.

ADR-001 carve-out: signing_key_history lives in canopy-security (not per-program) because public verification-key material is cross-cutting compliance metadata, not program-tenant data; it holds public keys only — never private material, never FTI.

7. What is sealed

Sealed (PII-bearing): money amounts, employer_name, description, address street lines, persons-table ssn + date_of_birth, SOLQ dollar amounts, derived-fact values, program_input, cross_program_inputs. The same money/free-text leaves are sealed in the fact events' before/after windows (MR9 — the at-rest envelope copied verbatim into the income/asset/expense.claimed/closed payloads, so the audit ledger stores ciphertext under the fact DEK; address events already carry only the coarse, street-redacted value, and household.member events carry only relationship). Left plaintext (structural/non-PII, sealing costs queryability for no redaction value): type discriminators, frequency, relationship, household_size, all UUIDs, corpus_hash, policy_params, and the already-coarse address city/state/zip/county.

Threat model

Crypto-shred is application-layer redaction, and this ADR states its boundary honestly (Kerckhoffs — no overclaim). Destroying the DEK in the live database makes the plaintext unrecoverable through the application, but the wrapped-DEK plaintext can residue:

  • in PostgreSQL WAL and base backups until their retention expires;

  • in unencrypted storage pages until those pages are overwritten;

  • in the EncryptionKeys.previous KEK held in memory during a rolling KEK rotation.

The operational prerequisites that make shred effective are therefore: block-layer-encrypted storage (so reused/old pages are unreadable), bounded backup retention, and brief KEK-rotation windows. A post-grace secure-overwrite sweep of WAL/backup DEK residue is a filed follow-up, not a v1 deliverable.

Other boundaries: losing the KEK makes every wrapped DEK unrecoverable (total plaintext loss — the point of crypto-shred, but operationally catastrophic if accidental); KEK rotation re-wraps DEKs (unwrap-old/wrap-new via the EncryptionKeys window) without re-sealing values. Sealing happens inside each service’s store layer, so unsealed PII never crosses a service boundary; the orchestrator receives only the outcome + snapshot_hash (never ciphertext or keys); canopy-security receives only sealed bytes and public keys, never FTI (ADR-004). Sealed serde_json::Value fields become opaque blobs, so future materiality (T2-7) / overpayment (T2-8) consumers must open() before comparing.

Consequences

  • A new canopy-crypto-shred crate owns the envelope + key hierarchy + the RedactionKeyStore / KeyHistoryProvider traits, reusing canopy_common::crypto (which gains AAD-capable encrypt_with_aad/decrypt_with_aad + a random_key helper).

  • The determination snapshot becomes v4-only; snapshot_hash is required; the legacy NoInputSnapshot/tri-state read path is removed.

  • The audit chain collapses to its v2 formula (the hash_version column is dropped).

  • Each sealing service gains a redaction_keys table + a redaction endpoint + canopy CLI parity; canopy-security gains signing_key_history + a JWKS endpoint. canopy-security needs NO redaction_keys and NO KEK for the fact-event surface: the audit copies share the persons fact DEK (single-owner; §5 as-built), so the persons redaction expunges them with no security-side key or subscriber.

  • Fact events seal their PII value leaves before publish; the typed FactRedactedEvent / SsnRedactedEvent / SsnAccessedEvent payloads (with the SsnAccessPurpose enum) replace the MR8 inline json!. The canopy-security change-history renders (sealed) for value leaves (worker-facing value display is re-sourced from canopy-persons in follow-up #920).

  • cargo machete: hkdf is intentionally NOT added (random DEKs + KEK-wrap, §2) — it would be an unused dependency.

Alternatives considered

  • Commitment-in-hash (store a salted hash of the value in the signed blob; keep plaintext in an encrypted side store). Rejected: it keeps raw values out of the signed artifact, but reading any value (appeals replay, overpayment recalc) then requires the side store + a commitment check, and the snapshot no longer self-contains its inputs — losing ADR-028’s reproducibility property. Hash-over- ciphertext keeps the snapshot self-contained and matches ADR-027 §8’s wording verbatim.

  • HKDF-derived DEKs from the KEK + a per-value salt. Rejected for v1: re-deriving from a shared secret complicates true erasure (the derivation input persists). Independent random DEKs make destruction a single-row delete.

  • Hard DELETE of the redaction_keys row instead of a tombstone. Rejected: loses the tamper-evident "redacted-at" proof and complicates idempotency; a one-way tombstone keeps both.

Edit this page · default