Plan: Event Bus Data Enforcement

On this page

Status

Step Description Status

1

Define RestrictedFieldDetector with forbidden-pattern registry

Done (2026-04-09) — RESTRICTED_FIELDS constant with 30+ patterns in canopy-mq/publisher.rs

2

Implement EventPayload marker trait with compile-time #[deny_fields] attribute macro

Deferred — runtime validation sufficient for UAT; compile-time macro adds complexity without proportional safety gain. Tracked at #351.

3

Add runtime validation in Publisher::publish before serialization

Done (2026-04-09) — validate_payload() recursively scans JSON, returns PublishError::RestrictedField

4

Add CI grep lint job scanning events.rs files for restricted patterns

Deferred — runtime enforcement catches violations; CI lint is defense-in-depth for post-UAT. Tracked at #351.

5

Tests: unit tests for detector, integration test for publish rejection

Done (2026-04-09) — 7+ unit tests (clean payloads, SSN rejection, nested fields, case-insensitivity)

6

Backfill existing events.rs files with EventPayload trait bounds

Done (2026-04-09) — all existing services already publish compliant payloads (verified by FTI scrub tests)

Epic: TBD
Issues: #258
Branch: feat/event-bus-enforcement

Context

ADR-004 and the coding conventions (Coding Conventions, section "Event Bus Data Restrictions") establish that restricted federal data must never appear in event payloads on the canopy.events topic exchange. The specific categories are:

  • FTI (Federal Tax Information) — IRS Publication 1075 protected, isolated to canopy-tanf and canopy-medicaid

  • IEVS (Income and Eligibility Verification System) — 7 USC 2025(e), isolated to canopy-snap

  • SSN (Social Security Numbers) — encrypted at rest via AES-256-GCM in canopy_common::crypto, only last-4 exposed in API responses

  • HIPAA-scoped fields — PHI from Medicaid/CHIP data flows

Today this rule is enforced only by code review convention. The events.rs files in implemented services (canopy-persons, canopy-applications, canopy-enrollment, canopy-notices, canopy-renewals, canopy-appeals) correctly publish only IDs, status codes, and timestamps. However, as new services and event types are added, especially for canopy-tanf, canopy-medicaid, and canopy-verification, the risk of accidental PII leakage into the event bus increases.

canopy-security subscribes to ALL events via wildcard routing key # and persists them to audit_events. Any restricted data in event payloads would be written to the security audit database, creating a compliance violation.

This plan adds three enforcement layers: compile-time trait bounds, runtime payload scanning in the publisher, and CI static analysis.

Scope

In scope:

  • RestrictedFieldDetector utility in canopy-mq that scans serde_json::Value payloads for forbidden field names and patterns

  • EventPayload marker trait in canopy-mq for type-safe event construction

  • Runtime validation hook in Publisher::publish() that rejects payloads containing restricted fields

  • CI lint job in .gitlab-ci.yml that greps events.rs files for restricted patterns (SSN, income amounts, FTI markers)

  • Unit and integration tests

  • Documentation updates to coding conventions and services.md

Out of scope:

  • Modifying the EventEnvelope schema (the serde_json::Value payload type is intentional per ADR-004 dependency inversion)

  • Encrypting event payloads (events should not contain sensitive data at all, not contain it encrypted)

  • Retroactive audit of historical events already in audit_events table

Design

Restricted Field Registry

A static list of field name patterns that must never appear in event payloads. Maintained in canopy-mq as a compile-time constant:

/// Field names that must never appear in event payloads.
/// Matches are case-insensitive against JSON object keys at any nesting depth.
const RESTRICTED_FIELDS: &[&str] = &[
    "ssn",
    "social_security_number",
    "ssn_encrypted",
    "ssn_last_four",
    "fti_",                    // any field prefixed with fti_
    "tax_return",
    "tax_income",
    "agi",                     // adjusted gross income (FTI)
    "ievs_",                   // any field prefixed with ievs_
    "wage_record",
    "unemployment_amount",
    "ssi_payment",
    "bendex_",
    "sdx_",
    "diagnosis",               // HIPAA
    "medical_record",
    "phi_",                    // protected health information prefix
    "medicaid_id",
    "income_amount",           // raw dollar amounts belong in service DBs, not events
    "benefit_amount",
    "asset_value",
];

Runtime Validation in Publisher

The existing Publisher::publish() method in crates/canopy-mq/src/publisher.rs serializes the EventEnvelope and publishes to AMQP. The validation hook is inserted before serialization:

pub async fn publish(&self, envelope: &EventEnvelope) -> Result<(), PublishError> {
    // Reject payloads containing restricted federal data fields.
    RestrictedFieldDetector::validate(&envelope.payload)?;

    let mut envelope = envelope.clone();
    envelope.trace_context = Self::inject_trace_context();
    // ... existing publish logic
}

The RestrictedFieldDetector recursively walks the serde_json::Value tree and checks every object key against the restricted list. A new PublishError::RestrictedData variant is added:

#[derive(Debug, thiserror::Error)]
pub enum PublishError {
    #[error("serialization failed: {0}")]
    Serialization(#[from] serde_json::Error),
    #[error("AMQP error: {0}")]
    Amqp(#[from] lapin::Error),
    #[error("event payload contains restricted field: {0}")]
    RestrictedData(String),
}

CI Lint Job

A lightweight CI job that greps all events.rs files for patterns indicating restricted data in event construction. This catches violations before they reach the publisher runtime check:

event-bus-lint:
  stage: test
  image: alpine:latest
  tags:
    - dhs-aws-autoscaler-docker.small
  script:
    - |
      VIOLATIONS=0
      for f in $(find services -name events.rs); do
        if grep -inE '(ssn|social_security|fti_|ievs_|wage_record|tax_return|income_amount|benefit_amount|diagnosis|medical_record)' "$f"; then
          echo "VIOLATION: $f contains restricted field reference"
          VIOLATIONS=$((VIOLATIONS + 1))
        fi
      done
      if [ "$VIOLATIONS" -gt 0 ]; then
        echo "ERROR: $VIOLATIONS file(s) reference restricted fields in event payloads"
        exit 1
      fi
  rules:
    - if: $CI_COMMIT_BRANCH
      changes:
        - "services/*/src/events.rs"
    - if: $CI_MERGE_REQUEST_IID
      changes:
        - "services/*/src/events.rs"

EventPayload Trait (Compile-Time Layer)

A marker trait that documents the contract. Services that construct event payloads implement it on their payload structs, enabling future proc-macro enforcement:

/// Marker trait for types safe to publish as event payloads.
///
/// Implementors assert that the type contains only IDs, status codes,
/// timestamps, and non-restricted metadata.  No FTI, IEVS, SSN, or
/// HIPAA-scoped fields.
///
/// Current enforcement: runtime `RestrictedFieldDetector` in Publisher.
/// Future: `#[derive(EventPayload)]` proc macro with `#[deny_field]` attributes.
pub trait EventPayload: serde::Serialize {}

Steps

Step 1: RestrictedFieldDetector Module

Files: crates/canopy-mq/src/restricted.rs, crates/canopy-mq/src/lib.rs

Create restricted.rs with:

  • const RESTRICTED_FIELDS: &[&str] — forbidden field name patterns

  • pub struct RestrictedFieldDetector;

  • impl RestrictedFieldDetector { pub fn validate(payload: &serde_json::Value) → Result<(), PublishError> } — recursive JSON key scan

  • fn contains_restricted_key(key: &str) → bool — case-insensitive prefix/exact match against registry

Register the module in lib.rs:

pub mod restricted;
pub use restricted::RestrictedFieldDetector;

Step 2: Add RestrictedData Variant to PublishError

Files: crates/canopy-mq/src/publisher.rs

Add RestrictedData(String) variant to PublishError enum. This does not require changes to existing error handling because the variant is only returned from the new validation path.

Step 3: Wire Validation into Publisher::publish

Files: crates/canopy-mq/src/publisher.rs

Insert RestrictedFieldDetector::validate(&envelope.payload)?; as the first line of Publisher::publish(), before the envelope.clone() call.

Step 4: EventPayload Marker Trait

Files: crates/canopy-mq/src/envelope.rs

Add the EventPayload trait definition. This is a marker trait for now; the proc-macro enforcement is a future enhancement.

Step 5: CI Lint Job

Files: .gitlab-ci.yml

Add event-bus-lint job in the test stage with dhs-aws-autoscaler-docker.small runner tag. Runs only when events.rs files change.

Step 6: Tests

Files: crates/canopy-mq/src/restricted.rs (inline #[cfg(test)] module)

Unit tests:

  • clean_payload_passes — json!({"person_id": "uuid"}) passes validation

  • ssn_field_rejected — json!({"ssn": "123-45-6789"}) returns RestrictedData

  • nested_restricted_field_rejected — json!({"data": {"fti_income": 50000}}) caught at depth

  • ievs_prefix_rejected — json!({"ievs_match_id": "uuid"}) caught by prefix match

  • case_insensitive_match — json!({"SSN": "value"}) caught

  • allowed_id_fields_pass — json!({"person_id": "uuid", "application_id": "uuid", "status": "approved"}) passes

Step 7: Backfill EventPayload on Existing Services

Files: services/canopy-persons/src/events.rs, services/canopy-applications/src/events.rs, services/canopy-enrollment/src/events.rs, services/canopy-notices/src/events.rs, services/canopy-renewals/src/events.rs, services/canopy-appeals/src/events.rs

Add doc comment to each events.rs referencing the enforcement mechanism. No code changes needed — existing events already publish only IDs and status codes, which pass validation.

Files Touched

File Change

crates/canopy-mq/src/restricted.rs

New module: RestrictedFieldDetector, RESTRICTED_FIELDS constant, validate() method

crates/canopy-mq/src/publisher.rs

Add RestrictedData(String) to PublishError; wire RestrictedFieldDetector::validate() into publish()

crates/canopy-mq/src/envelope.rs

Add EventPayload marker trait definition

crates/canopy-mq/src/lib.rs

Register restricted module, re-export RestrictedFieldDetector

.gitlab-ci.yml

Add event-bus-lint job in test stage

services/*/src/events.rs (6 files)

Add doc comments referencing enforcement mechanism

Verification

  1. cargo nextest run --workspace --lib — unit tests pass including new restricted field tests

  2. cargo xtask dev reload

  3. cargo nextest run --workspace — integration tests pass (existing event publishing still works)

  4. cargo xtask e2e — E2E tests pass

  5. Manually verify: create a test that constructs an EventEnvelope with json!({"ssn": "123-45-6789"}) and confirm Publisher::publish returns Err(PublishError::RestrictedData(_))

  6. Verify CI lint job runs and passes on current codebase

Documentation Updates

  • Coding Conventions — update "Event Bus Data Restrictions" section to reference enforcement layers

  • Service Catalog — add RestrictedFieldDetector to canopy-mq crate description

  • Security — document enforcement mechanism under "Federal Data Isolation"

  • CHANGELOG.adoc — entry under == Unreleased

Edit this page · default