Plan: Event Bus Data Enforcement
On this page
Status
| Step | Description | Status |
|---|---|---|
1 |
Define |
Done (2026-04-09) — RESTRICTED_FIELDS constant with 30+ patterns in canopy-mq/publisher.rs |
2 |
Implement |
Deferred — runtime validation sufficient for UAT; compile-time macro adds complexity without proportional safety gain. Tracked at #351. |
3 |
Add runtime validation in |
Done (2026-04-09) — validate_payload() recursively scans JSON, returns PublishError::RestrictedField |
4 |
Add CI grep lint job scanning |
Deferred — runtime enforcement catches violations; CI lint is defense-in-depth for post-UAT. Tracked at #351. |
5 |
Tests: unit tests for detector, integration test for publish rejection |
Done (2026-04-09) — 7+ unit tests (clean payloads, SSN rejection, nested fields, case-insensitivity) |
6 |
Backfill existing |
Done (2026-04-09) — all existing services already publish compliant payloads (verified by FTI scrub tests) |
Epic: TBD
Issues: #258
Branch: feat/event-bus-enforcement
Context
ADR-004 and the coding conventions (Coding Conventions, section "Event Bus Data Restrictions") establish that restricted federal data must never appear in event payloads on the canopy.events topic exchange. The specific categories are:
-
FTI (Federal Tax Information) — IRS Publication 1075 protected, isolated to canopy-tanf and canopy-medicaid
-
IEVS (Income and Eligibility Verification System) — 7 USC 2025(e), isolated to canopy-snap
-
SSN (Social Security Numbers) — encrypted at rest via AES-256-GCM in
canopy_common::crypto, only last-4 exposed in API responses -
HIPAA-scoped fields — PHI from Medicaid/CHIP data flows
Today this rule is enforced only by code review convention. The events.rs files in implemented services (canopy-persons, canopy-applications, canopy-enrollment, canopy-notices, canopy-renewals, canopy-appeals) correctly publish only IDs, status codes, and timestamps. However, as new services and event types are added, especially for canopy-tanf, canopy-medicaid, and canopy-verification, the risk of accidental PII leakage into the event bus increases.
canopy-security subscribes to ALL events via wildcard routing key # and persists them to audit_events. Any restricted data in event payloads would be written to the security audit database, creating a compliance violation.
This plan adds three enforcement layers: compile-time trait bounds, runtime payload scanning in the publisher, and CI static analysis.
Scope
In scope:
-
RestrictedFieldDetectorutility incanopy-mqthat scansserde_json::Valuepayloads for forbidden field names and patterns -
EventPayloadmarker trait incanopy-mqfor type-safe event construction -
Runtime validation hook in
Publisher::publish()that rejects payloads containing restricted fields -
CI lint job in
.gitlab-ci.ymlthat grepsevents.rsfiles for restricted patterns (SSN, income amounts, FTI markers) -
Unit and integration tests
-
Documentation updates to coding conventions and services.md
Out of scope:
-
Modifying the
EventEnvelopeschema (theserde_json::Valuepayload type is intentional per ADR-004 dependency inversion) -
Encrypting event payloads (events should not contain sensitive data at all, not contain it encrypted)
-
Retroactive audit of historical events already in
audit_eventstable
Design
Restricted Field Registry
A static list of field name patterns that must never appear in event payloads. Maintained in canopy-mq as a compile-time constant:
/// Field names that must never appear in event payloads.
/// Matches are case-insensitive against JSON object keys at any nesting depth.
const RESTRICTED_FIELDS: &[&str] = &[
"ssn",
"social_security_number",
"ssn_encrypted",
"ssn_last_four",
"fti_", // any field prefixed with fti_
"tax_return",
"tax_income",
"agi", // adjusted gross income (FTI)
"ievs_", // any field prefixed with ievs_
"wage_record",
"unemployment_amount",
"ssi_payment",
"bendex_",
"sdx_",
"diagnosis", // HIPAA
"medical_record",
"phi_", // protected health information prefix
"medicaid_id",
"income_amount", // raw dollar amounts belong in service DBs, not events
"benefit_amount",
"asset_value",
];
Runtime Validation in Publisher
The existing Publisher::publish() method in crates/canopy-mq/src/publisher.rs serializes the EventEnvelope and publishes to AMQP. The validation hook is inserted before serialization:
pub async fn publish(&self, envelope: &EventEnvelope) -> Result<(), PublishError> {
// Reject payloads containing restricted federal data fields.
RestrictedFieldDetector::validate(&envelope.payload)?;
let mut envelope = envelope.clone();
envelope.trace_context = Self::inject_trace_context();
// ... existing publish logic
}
The RestrictedFieldDetector recursively walks the serde_json::Value tree and checks every object key against the restricted list. A new PublishError::RestrictedData variant is added:
#[derive(Debug, thiserror::Error)]
pub enum PublishError {
#[error("serialization failed: {0}")]
Serialization(#[from] serde_json::Error),
#[error("AMQP error: {0}")]
Amqp(#[from] lapin::Error),
#[error("event payload contains restricted field: {0}")]
RestrictedData(String),
}
CI Lint Job
A lightweight CI job that greps all events.rs files for patterns indicating restricted data in event construction. This catches violations before they reach the publisher runtime check:
event-bus-lint:
stage: test
image: alpine:latest
tags:
- dhs-aws-autoscaler-docker.small
script:
- |
VIOLATIONS=0
for f in $(find services -name events.rs); do
if grep -inE '(ssn|social_security|fti_|ievs_|wage_record|tax_return|income_amount|benefit_amount|diagnosis|medical_record)' "$f"; then
echo "VIOLATION: $f contains restricted field reference"
VIOLATIONS=$((VIOLATIONS + 1))
fi
done
if [ "$VIOLATIONS" -gt 0 ]; then
echo "ERROR: $VIOLATIONS file(s) reference restricted fields in event payloads"
exit 1
fi
rules:
- if: $CI_COMMIT_BRANCH
changes:
- "services/*/src/events.rs"
- if: $CI_MERGE_REQUEST_IID
changes:
- "services/*/src/events.rs"
EventPayload Trait (Compile-Time Layer)
A marker trait that documents the contract. Services that construct event payloads implement it on their payload structs, enabling future proc-macro enforcement:
/// Marker trait for types safe to publish as event payloads.
///
/// Implementors assert that the type contains only IDs, status codes,
/// timestamps, and non-restricted metadata. No FTI, IEVS, SSN, or
/// HIPAA-scoped fields.
///
/// Current enforcement: runtime `RestrictedFieldDetector` in Publisher.
/// Future: `#[derive(EventPayload)]` proc macro with `#[deny_field]` attributes.
pub trait EventPayload: serde::Serialize {}
Steps
Step 1: RestrictedFieldDetector Module
Files: crates/canopy-mq/src/restricted.rs, crates/canopy-mq/src/lib.rs
Create restricted.rs with:
-
const RESTRICTED_FIELDS: &[&str]— forbidden field name patterns -
pub struct RestrictedFieldDetector; -
impl RestrictedFieldDetector { pub fn validate(payload: &serde_json::Value) → Result<(), PublishError> }— recursive JSON key scan -
fn contains_restricted_key(key: &str) → bool— case-insensitive prefix/exact match against registry
Register the module in lib.rs:
pub mod restricted;
pub use restricted::RestrictedFieldDetector;
Step 2: Add RestrictedData Variant to PublishError
Files: crates/canopy-mq/src/publisher.rs
Add RestrictedData(String) variant to PublishError enum. This does not require changes to existing error handling because the variant is only returned from the new validation path.
Step 3: Wire Validation into Publisher::publish
Files: crates/canopy-mq/src/publisher.rs
Insert RestrictedFieldDetector::validate(&envelope.payload)?; as the first line of Publisher::publish(), before the envelope.clone() call.
Step 4: EventPayload Marker Trait
Files: crates/canopy-mq/src/envelope.rs
Add the EventPayload trait definition. This is a marker trait for now; the proc-macro enforcement is a future enhancement.
Step 5: CI Lint Job
Files: .gitlab-ci.yml
Add event-bus-lint job in the test stage with dhs-aws-autoscaler-docker.small runner tag. Runs only when events.rs files change.
Step 6: Tests
Files: crates/canopy-mq/src/restricted.rs (inline #[cfg(test)] module)
Unit tests:
-
clean_payload_passes—json!({"person_id": "uuid"})passes validation -
ssn_field_rejected—json!({"ssn": "123-45-6789"})returnsRestrictedData -
nested_restricted_field_rejected—json!({"data": {"fti_income": 50000}})caught at depth -
ievs_prefix_rejected—json!({"ievs_match_id": "uuid"})caught by prefix match -
case_insensitive_match—json!({"SSN": "value"})caught -
allowed_id_fields_pass—json!({"person_id": "uuid", "application_id": "uuid", "status": "approved"})passes
Step 7: Backfill EventPayload on Existing Services
Files: services/canopy-persons/src/events.rs, services/canopy-applications/src/events.rs, services/canopy-enrollment/src/events.rs, services/canopy-notices/src/events.rs, services/canopy-renewals/src/events.rs, services/canopy-appeals/src/events.rs
Add doc comment to each events.rs referencing the enforcement mechanism. No code changes needed — existing events already publish only IDs and status codes, which pass validation.
Files Touched
| File | Change |
|---|---|
|
New module: |
|
Add |
|
Add |
|
Register |
|
Add |
|
Add doc comments referencing enforcement mechanism |
Verification
-
cargo nextest run --workspace --lib— unit tests pass including new restricted field tests -
cargo xtask dev reload -
cargo nextest run --workspace— integration tests pass (existing event publishing still works) -
cargo xtask e2e— E2E tests pass -
Manually verify: create a test that constructs an
EventEnvelopewithjson!({"ssn": "123-45-6789"})and confirmPublisher::publishreturnsErr(PublishError::RestrictedData(_)) -
Verify CI lint job runs and passes on current codebase
Documentation Updates
-
Coding Conventions — update "Event Bus Data Restrictions" section to reference enforcement layers
-
Service Catalog — add RestrictedFieldDetector to canopy-mq crate description
-
Security — document enforcement mechanism under "Federal Data Isolation"
-
CHANGELOG.adoc— entry under== Unreleased