Worker portal redesign — Stage 3 MR1 (DB migrations + composition loader)

On this page
NOTE

MR1 of Stage 3 of group epic &51 (#460). Closes #489 (DB migrations) + #490 (composition loader). #491 (HTTP live-override APIs) ships as MR2 with its own plan.

GitLab MR labels: type::feature, priority::high, service::web, service::shared-crates, workflow::in-progress.

Status

Step Description Status

1

Crate scaffolding + types + proc-macro (single commit). Add crates/canopy-composition/ (Cargo.toml + src/{lib,types,source}.rs) and crates/canopy-plugin-macros/ (proc-macro crate). Register both in workspace Cargo.toml. Define Plugin trait, PluginRegistration struct, PluginSource trait, CompileTimePluginSource, #[linkme::distributed_slice] pub static CANOPY_PLUGINS: [PluginRegistration] = [..];, all ComposedSurface / ComposedItem / CompositionLoadError / newtype types. Gate: cargo nextest run -p canopy-composition -p canopy-plugin-macros && cargo deny check.

Done (2026-05-21)

2

Schemas + parsers + merge + role filter. manifest.rs (Plugin.toml schema, serde(deny_unknown_fields)), idp.rs (idp.toml v1, roles-only, serde(deny_unknown_fields)), defaults.rs (per-surface LazyLock<serde_json::Value> via include_str!), merge.rs (RFC 7396 + RFC 6902 wrappers around json-patch), role_filter.rs. 5 defaults/{surface}.json fixtures + 1 fixtures/Plugin.toml. Tests: 6 named test files (enumerated below). Gate: cargo nextest run -p canopy-composition clean.

Done (2026-05-21)

3

DB migrations + sqlx queries + loader + cache + jurisdiction fixtures. Forward-only migration at services/canopy-web/migrations/{YYYYMMDDHHMMSS}_create_composition_documents.sql (timestamp = day-of-landing). db.rs (runtime sqlx::query_as::<_, DbLayer>(…​) function form), cache.rs (tokio::sync::RwLock<HashMap<…>>), loader.rs (orchestrates merge → DB → manifest pre-validation → post-merge validation → role-filter → cache.insert). Fixture jurisdiction TOMLs at rulesets/georgia/composition/{worker_dashboard,case_detail,sign_in}.toml + rulesets/georgia/idp.toml. Gate: cargo nextest run -p canopy-composition + cargo xtask dev migrate on devstack.

Done (2026-05-21)

4

Plan finalize + parent plan + CHANGELOG + coding-conventions. Move this plan body to docs/modules/ROOT/pages/plans/archive/worker-portal-redesign-stage3-composition-runtime.adoc; Status cells Not startedDone (YYYY-MM-DD). Update parent plan Stage 3 row + acceptance + files-touched + Status. Append "Composition runtime patterns" subsection to .claude/docs/coding-conventions.md. CHANGELOG === Added entry (sample text below). Lint: asciidoctor-lint clean.

Done (2026-05-21)

Tracking issues: #489 + #490
Epic: &51
Parent plan: worker-portal-redesign.adoc
Branch: feat/wpr-stage3-composition-runtime (single MR)

Context

Stages 1 + 1.5 + 2 (2/3 ADRs) of epic &51 are shipped. The composability runtime — the heart of the worker portal redesign — is the next surface. ADR-021 ratified the runtime + plugin model; ADR-022 ratified the override-storage layering. Stage 3 implements both.

Stage 3 is fully greenfield: no crates/canopy-composition/, no crates/canopy-plugin-macros/, no json-patch workspace dep, no linkme workspace dep, no Plugin.toml schema implementation, no idp.toml (referenced by ADR-022 for role-slug validation; full design ships in Stage 4 #493 but the roles-only schema lands here), no rulesets/{j}/composition/ directories, no composition_documents migration. All scaffolding is net-new in this MR.

The MR1 split (#489 + #490) ships the persistence + runtime + schemas + fixtures as a coherent unit. The loader is unit-testable end-to-end against fixture jurisdictions without any HTTP layer. MR2 (#491) layers the 15-endpoint write API on top, with audit/auth/cache wiring as a separable concern.

Design

Decisions locked

  1. New crate crates/canopy-composition/ as the runtime home, NOT inline in services/canopy-web/src/composition/. Matches ADR-021 Decision shape. Stage 6 Studio (#499/#500) will reuse the merge + role-filter + manifest-validator logic. Doing the extraction now avoids the inevitable rework.

  2. New crate crates/canopy-plugin-macros/ for the #[canopy_plugin] proc-macro. v1 macro is minimal — it extracts slug + the include_str!-resolved Plugin.toml path from its args and emits the linkme::distributed_slice entry. It does NOT parse the TOML at expansion time (avoids a canopy-plugin-macros → canopy-composition dep chain that would pull canopy-composition’s sqlx/tokio deps into host-arch proc-macro builds). Compile-time validation of Plugin.toml (full schema check per ADR-021 lines 47-50) is deferred to a follow-up workflow::needs-spec issue filed in Step 4. The trade-off: runtime Plugin::manifest() returns Result<&Manifest, &ManifestError> (NOT panic!) and the loader surfaces invalid manifests via the existing CompositionLoadError::ManifestParse variant — Path B from the design review.

  3. Plugin.toml schema = full ADR-021 spec ([plugin], [plugin.exports], [panels.], [case_sections.], [data], [permissions], [i18n]). Parser uses serde + toml with #[serde(deny_unknown_fields)] per ADR-012 convention.

  4. idp.toml schema = roles-only for v1. Just [roles.<slug>] tables with display_name + description + a default_role field. #[serde(deny_unknown_fields)]. Stage 4 (#493) expands with IDP providers, email-discovery rules, local-accounts toggle. The schema’s doc-comment instructs Stage 4 to extend (not replace) the existing fields.

  5. system_defaults = LazyLock<serde_json::Value> (one per surface) materialized in canopy-composition::defaults from a checked-in defaults/{surface}.json file. std::sync::LazyLock is stable since rust 1.80; workspace pins ≥ 1.80.

  6. Loader API matches ADR-021 verbatim: async fn load_composition(jurisdiction, role, user_id: Option<&UserId>, surface) → Result<ComposedSurface, CompositionLoadError>. Errors are the four ADR-021 variants (UnknownPlugin, SpanOutOfRange, RowOverflow, RoleNotFound) + a fifth PatchFailed { layer, source: json_patch::PatchError } per ADR-022 line 195 (no op_index — json-patch returns one error per op list, not per op). Plus UnknownJurisdiction, PostMergeShape, ManifestParse, IdpParse, Db, Io for the cross-cutting concerns.

  7. PluginSource trait + CompileTimePluginSource impl in canopy-composition::source. v1 has one impl; the trait exists for v2 federation (WasmPluginSource) per ADR-021 Decision 1.

  8. Composition cache = tokio::sync::RwLock<HashMap<CompositionKey, Arc<ComposedSurface>>> in canopy-composition::cache. Invalidate-on-write API: cache.invalidate(key) + cache.invalidate_jurisdiction(jurisdiction_id). v1 is single-replica; multi-replica composition.invalidated event explicitly deferred (ADR-021 Decision 3). MR2 (#491) wires invalidation from write endpoints; in MR1 the cache exists + is unit-tested but only consumed via the loader’s own hot path.

  9. Merge implementation: json-patch crate, latest stable (added via cargo add json-patch in Step 1; verify API surface on adoption). Used for both RFC 6902 (json_patch::patch) and RFC 7396 (json_patch::merge). ADR-022 line 190 already references json_patch::patch.

  10. Jurisdiction fixtures shipped: rulesets/georgia/composition/{worker_dashboard,case_detail,sign_in}.toml (3 of 5 surfaces; shell-only, empty items in MR1 since no real plugins register yet — see Decision 5) + rulesets/georgia/idp.toml (roles-only). Supervisor + analyst dashboards stay defaults-only in v1. When Stage 5 ships the first real plugins, Georgia baselines extend to declare items pointing at those exports (per its plan).

  11. sqlx query scaffolds in canopy-composition::db, NOT canopy-web. canopy-composition takes a &PgPool parameter so the loader is callable from canopy-web OR future canopy-cli composition dump (filed as follow-up). The migration SQL lives at services/canopy-web/migrations/ per ADR-022 — canopy-web is the only service with composition tables today.

  12. No system_defaults for sign_in surface in v1. The sign-in surface composition is the list of IDP buttons the user sees on the sign-in page, sourced from idp.toml’s [providers] section (Stage 4 #493 scope). For MR1 the defaults/sign_in.json is {"shell": "", "items": []} (matches the uniform RawComposition shape) and loader.rs short-circuits with a doc-comment "Stage 4 (#493) wires real IDP list from idp.toml".

  13. Match existing canopy-web sqlx pattern: runtime queries via the function form sqlx::query_as::<_, DbLayer>(SQL).bind(…​).fetch_all(&pool) with #[derive(sqlx::FromRow)] struct DbLayer. NOT the sqlx::query_as!() macro (which requires compile-time DATABASE_URL or offline cache). The function form is what canopy-web’s existing handlers use. Introducing sqlx-offline as cross-canopy tooling is a separable follow-up.

  14. Real plugin Plugin.toml location: services/canopy-web/plugins/{slug}/Plugin.toml per ADR-021 line 139. Stage 5+ ships real plugins. MR1 ships a fixture-only Plugin.toml at crates/canopy-composition/fixtures/Plugin.toml for parser tests — NOT the production location. Plan documents both locations to prevent future confusion.

  15. Audit emission deferred to MR2. ADR-021 line 174 + ADR-022 line 127 require every composition mutation + every audit="read" panel render to emit a JWS-signed AuditEvent per ADR-014. The loader carries audit_emitter: Arc<dyn AuditEmitter> (defaulted to Arc::new(NoopAuditEmitter) in MR1 — the no-op satisfies the trait without emitting any events). MR2 swaps in the real emitter from canopy-security.

  16. Loader validation runs in 3 ordered passes after the 5 layers merge (split per the user’s MED-A finding from review round 5 so dropped-by-role items don’t trigger spurious overflows): (P1) Manifest pre-validation — walk every registered plugin’s manifest(); surface ManifestParse immediately on any Err. (P2) Export-resolution — for each item.item (export slug), surface-aware lookup via find_panel / find_case_section; surface UnknownPlugin { slug } on miss. NO span/row checks here. (P3a) Role filterfilter_items_by_role drops items whose plugin’s [permissions].required_roles excludes role (silent drop per ADR-021 line 135). (P3b) Span + row constraint validation — runs on the role-filtered items only: assert each item.span ∈ def.allowed_spans else SpanOutOfRange; group by row, sum spans, assert each row’s sum ≤ 12 else RowOverflow. RoleNotFound is enforced separately, before cache lookup, against idp.toml.

  17. Role filtering is fn(items: &mut Vec<ComposedItem>, role: &RoleSlug, surface: ComposableSurface, source: &dyn PluginSource) → () (no Result; takes surface so it picks the right export lookup). Per ADR-021 line 135, items are silently dropped if the role is not in the exporting plugin’s [permissions].required_roles. ADR-021 keeps [permissions] at the plugin manifest level (NOT per-panel), so role-filter accesses it via the plugin handle returned from find_panel / find_case_section using let Ok(manifest) = plugin.manifest() else { return false; }; (NOT ? — the fn returns (), not Result). Plugins with poisoned manifests are NEVER seen by role-filter because the loader’s Manifest pre-validation pass in Step 3 above surfaces ManifestParse and returns BEFORE role-filter runs; the let Ok defensive branch is unreachable in practice but keeps the type signature honest.

  18. Jurisdiction slug → UUID resolution via a new JurisdictionRegistry trait + StaticJurisdictionRegistry impl in canopy-composition::jurisdiction. The registry maps JurisdictionSlugUuid. v1’s StaticJurisdictionRegistry is hardcoded with {"georgia" → uuid_v7_for_georgia}. Stage 4+ may move this to rulesets/{j}/jurisdiction.toml lookup; the trait makes that additive. The loader takes Arc<dyn JurisdictionRegistry> as a constructor field and uses it to resolve JurisdictionSlugUuid before calling db::fetch_db_layers. If the registry doesn’t know the slug, surfaces CompositionLoadError::UnknownJurisdiction { slug }.

  19. AuditEmitter trait shape (minimal v1, MR2 wires real impl):

    #[async_trait::async_trait]
    pub trait AuditEmitter: Send + Sync {
        /// Called by `load_composition` BEFORE returning `Ok(...)` so the
        /// audit event includes the resolved `version` hash. v1 MR1 uses
        /// a no-op `NoopAuditEmitter` impl in unit tests. MR2 wires the
        /// real impl that calls into canopy-security per ADR-014.
        async fn emit_render(
            &self,
            jurisdiction: &str,
            role: &str,
            user_id: Option<&str>,
            surface: &str,
            version: u64,
        );
    }
    
    pub struct NoopAuditEmitter;
    #[async_trait::async_trait]
    impl AuditEmitter for NoopAuditEmitter {
        async fn emit_render(&self, _: &str, _: &str, _: Option<&str>, _: &str, _: u64) {}
    }

Plugin.toml schema (full, per ADR-021)

[plugin]
slug = "snap-overpayment-summary"        # required; ^[a-z][a-z0-9-]*[a-z0-9]$
name = "SNAP Overpayment Summary"        # required; human-readable
version = "1.0.0"                        # required; semver
author = "canopy-core"                   # required; free-form
license = "AGPL-3.0-or-later"            # required; SPDX
canopy_min = "0.1.0"                     # required; semver compat lower bound

[plugin.exports]
panels = ["snap-overpayment-summary-panel"]
case_sections = []

[panels.snap-overpayment-summary-panel]
display_name_key = "panels.snap_overpayment_summary.title"   # required; i18n key
icon = "💰"                                                  # required; unicode glyph
programs = ["snap"]                                          # required; ⊆ {snap, tanf, medicaid, caps, wic}
default_span = 4                                             # required; ∈ allowed_spans
allowed_spans = [3, 4, 6, 12]                                # required; ⊆ {1, 2, 3, 4, 6, 12} (ADR-021 line 192 breakpoint set)
required_states = ["empty", "loading", "error", "populated"] # required; ⊆ {empty, loading, error, populated}

[data]
source = "canopy-snap"                                       # required; canopy service slug
auth = "service_class"                                       # required; ∈ {none, service_class, user_jwt}
cache_ttl_seconds = 30                                       # required; u32 ≥ 0
timeout_ms = 5000                                            # required; u32 > 0
endpoints = ["/v1/overpayments/summary?household_id={household_id}"]  # required; non-empty

[permissions]
required_roles = ["eligibility_worker", "supervisor"]        # required; non-empty
audit = "read"                                               # required; ∈ {none, read, write}

[i18n]
default = "en"                                               # required
catalogs = ["en", "es"]                                      # required; non-empty; contains default

idp.toml schema (roles-only v1; expanded by Stage 4 #493)

# idp.toml — IDP and role configuration for {jurisdiction}.
# v1 (Stage 3 of #460): roles section only. Stage 4 (#493) adds
# [providers], [discovery], and [local_accounts] sections without
# replacing the existing [roles] schema. Uses serde(deny_unknown_fields).

default_role = "eligibility_worker"

[roles.eligibility_worker]
display_name = "Eligibility Worker"
description = "Front-line caseworker"

[roles.supervisor]
display_name = "Supervisor"
description = "Reviews caseworker decisions; can override"

[roles.jurisdiction_admin]
display_name = "Jurisdiction Admin"
description = "Manages live composition overrides for this jurisdiction"

[roles.qc]
display_name = "Quality Control"
description = "Read-only QC reviewer"

defaults/{surface}.json shape

Each defaults file deserializes into RawComposition { shell: String, items: Vec<ComposedItem> } (the intermediate type). The loader then maps shell to a typed ShellSpec based on surface. The string shell representation keeps the JSON simple and uniform across surfaces.

All MR1 defaults ship with empty items: [] because real plugin handlers don’t land until Stage 5/6 — CompileTimePluginSource is empty in MR1. Non-empty defaults would reference panel slugs that no plugin exports, and the loader’s UnknownPlugin check would reject them. Stage 5 ships real plugins + populates the defaults at the same time per its plan.

Example defaults/worker_dashboard.json (MR1 form):

{
  "shell": "grid",
  "items": []
}

When Stage 5 adds the first dashboard plugins, the defaults expand to reference their export slugs (a key in the plugin’s [panels.*] table, NOT the plugin’s [plugin].slug). See Decision 16 + the ComposedItem doc-comment.

Five MR1 files ship — all with empty items: []:

  • defaults/worker_dashboard.json{"shell": "grid", "items": []}

  • defaults/supervisor_dashboard.json{"shell": "grid", "items": []}

  • defaults/analyst_dashboard.json{"shell": "grid", "items": []}

  • defaults/case_detail.json{"shell": "tabs", "items": []} (shell carried so the Georgia case-detail tabs experience is locked from MR1; sections fill in Stage 5)

  • defaults/sign_in.json{"shell": "", "items": []} (Decision 12; shell ignored for sign-in)

Schema enforcement happens via serde deserialization into RawComposition at load time; any field outside the schema fails to deserialize. The further mapping of raw.shell to ShellSpec enforces surface-specific shell-value validity per the loader’s shell-mapping step in Step 3.

Test fixture plugins: loader_test.rs uses an in-test TestPluginSource impl (declared inline at the top of the test file) that returns hand-written Manifest values for "test-panel-a", "test-panel-b", etc. — used only by the validation-rejection tests (span/row/role/UnknownPlugin coverage). The production CompileTimePluginSource stays empty in MR1.

Core type vocabulary (crates/canopy-composition/src/types.rs)

// SPDX-License-Identifier: AGPL-3.0-or-later

use serde::{Deserialize, Serialize};
use std::sync::Arc;

#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize, sqlx::Type)]
#[sqlx(type_name = "composition_surface", rename_all = "snake_case")]
#[serde(rename_all = "snake_case")]
pub enum ComposableSurface {
    WorkerDashboard,
    SupervisorDashboard,
    AnalystDashboard,
    CaseDetail,
    SignIn,
}

/// Per-surface shell layout enum. Each surface's `ComposedSurface.shell`
/// uses a different variant set; we encode this as a tagged enum.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(tag = "surface", rename_all = "snake_case")]
pub enum ShellSpec {
    #[serde(rename = "worker_dashboard")]
    WorkerDashboard { layout: WorkerDashboardLayout },
    #[serde(rename = "supervisor_dashboard")]
    SupervisorDashboard { layout: WorkerDashboardLayout },
    #[serde(rename = "analyst_dashboard")]
    AnalystDashboard { layout: WorkerDashboardLayout },
    #[serde(rename = "case_detail")]
    CaseDetail { shell: CaseDetailShell },
    #[serde(rename = "sign_in")]
    SignIn,
}

#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum WorkerDashboardLayout { Grid, Stacked }

#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum CaseDetailShell { Scroll, CardGrid, Tabs }

/// Plugin identity from `[plugin].slug` in Plugin.toml. Composition items
/// do NOT reference this directly — they reference `ItemSlug`s (panel or
/// case_section keys from `[panels.*]` / `[case_sections.*]`). The mapping
/// from `ItemSlug` → plugin is established by the plugin's
/// `[plugin.exports]` table.
#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct PluginSlug(pub String);

/// Slug of a composition item — matches a key in `[panels.*]` (for
/// dashboard surfaces) or `[case_sections.*]` (for case_detail) of some
/// plugin's Plugin.toml. NOT the plugin slug. ADR-021 line 132's
/// "panel for dashboards, section for case-detail" uses this concept.
#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct ItemSlug(pub String);

#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct RoleSlug(pub String);

#[derive(Debug, Clone, Hash, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct JurisdictionSlug(pub String);

// UserId reuses canopy-common's `define_id!` convention (UUID v7, sqlx
// transparent, utoipa-schema-aware) so it composes with the rest of canopy.
canopy_common::define_id!(
    /// Worker user identity used as scope_key for the `user` layer.
    UserId
);

/// Loader-facing key. `Hash + Eq` is used by the composition cache.
#[derive(Debug, Clone, Hash, PartialEq, Eq)]
pub struct CompositionKey {
    pub jurisdiction: JurisdictionSlug,
    pub role: RoleSlug,
    pub user_id: Option<UserId>,
    pub surface: ComposableSurface,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct ComposedItem {
    /// Export slug — matches a `[panels.*]` or `[case_sections.*]` key
    /// in the exporting plugin's Plugin.toml. The composition system
    /// resolves this to a plugin via `PluginSource::find_panel` /
    /// `find_case_section`. Different from the plugin's `[plugin].slug`.
    pub item: ItemSlug,
    pub span: u8,
    pub row: u8,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ComposedSurface {
    pub surface: ComposableSurface,
    pub shell: ShellSpec,
    pub items: Vec<ComposedItem>,
    /// First 8 bytes of SHA-256 of canonical RFC 8785 JSON of the resolved
    /// composition doc (post-merge, post-role-filter, pre-version-stamp).
    /// Per ADR-021 line 133.
    pub version: u64,
}

/// Intermediate post-merge shape *before* `version` is computed and
/// *before* `shell` is mapped to its typed `ShellSpec` variant. The
/// loader deserializes the merged JSON tree into this; `shell` is a
/// raw string (e.g., "tabs", "scroll", "card_grid", "grid", "default")
/// because defaults / baseline TOMLs author shell as a string scalar.
/// The loader maps `raw.shell` to `ShellSpec` per the request's
/// `ComposableSurface` before assembling the final `ComposedSurface`.
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct RawComposition {
    /// Shell variant name as a string. Surface-specific valid values:
    /// - WorkerDashboard / Supervisor / Analyst: "grid" | "stacked"
    /// - CaseDetail: "scroll" | "card_grid" | "tabs"
    /// - SignIn: ignored (no shell)
    #[serde(default)]
    pub shell: String,
    pub items: Vec<ComposedItem>,
}

#[derive(Debug, thiserror::Error)]
pub enum CompositionLoadError {
    #[error("plugin slug `{slug}` referenced by composition is not registered")]
    UnknownPlugin { slug: String },
    #[error("item span {span} is outside plugin `{slug}`'s allowed_spans {allowed:?}")]
    SpanOutOfRange { slug: String, span: u8, allowed: Vec<u8> },
    #[error("row {row} exceeds 12-column budget: total span = {total}")]
    RowOverflow { row: u8, total: u32 },
    #[error("role `{role}` is not defined in `{jurisdiction}`'s idp.toml")]
    RoleNotFound { jurisdiction: String, role: String },
    #[error("jurisdiction slug `{slug}` not found in JurisdictionRegistry")]
    UnknownJurisdiction { slug: String },
    /// `json_patch::patch` returns one error for the entire op list (the
    /// crate does not expose per-op index); the `layer` field identifies
    /// which of {jurisdiction_live, role, user} failed.
    #[error("RFC 6902 patch failed at layer `{layer}`: {source}")]
    PatchFailed { layer: &'static str, #[source] source: json_patch::PatchError },
    /// Carries an owned `String` (not `&ManifestError`) because the
    /// underlying `toml::de::Error` is not `Clone`; the loader's
    /// pre-validation pass calls `.to_string()` on the ManifestError
    /// reference returned by `Plugin::manifest()`.
    #[error("manifest parse error: {0}")]
    ManifestParse(String),
    #[error("idp.toml parse error: {0}")]
    IdpParse(#[from] crate::idp::IdpError),
    #[error("post-merge doc failed to deserialize into ComposedSurface: {0}")]
    PostMergeShape(serde_json::Error),
    #[error("database error: {0}")]
    Db(#[from] sqlx::Error),
    #[error("io error: {0}")]
    Io(#[from] std::io::Error),
}

Plugin trait + PluginSource + plugin registration (source.rs)

ADR-021’s PluginSource::get returns Option<&dyn Plugin>. We define the Plugin trait minimally — slug + manifest accessors are all the loader needs in MR1. The Askama-partial render hook is Stage 5+ scope; the trait is forward-compatible (a default fn render method can be added without breaking existing `impl`s).

// SPDX-License-Identifier: AGPL-3.0-or-later

use crate::types::PluginSlug;
use crate::manifest::Manifest;
use std::sync::OnceLock;

/// Minimal trait every plugin handler implements. `PluginRegistration`
/// is the v1 implementer; Stage 5+ may add `render`, `audit_event`, etc.
/// methods with default impls so existing plugins don't break.
///
/// `manifest()` returns `Result` because v1's `#[canopy_plugin]` macro
/// does NOT validate the embedded TOML at compile time (Decision 2);
/// the first call parses + caches the result. A follow-up issue
/// promotes validation to compile-time per ADR-021 lines 47-50.
pub trait Plugin: Send + Sync {
    fn slug(&self) -> &str;
    fn manifest(&self) -> Result<&Manifest, &crate::manifest::ManifestError>;
}

/// Per-plugin runtime metadata. Populated by `#[canopy_plugin]`.
pub struct PluginRegistration {
    pub slug: &'static str,
    pub manifest_toml: &'static str,
    pub manifest_cache: OnceLock<Result<Manifest, crate::manifest::ManifestError>>,
}

impl Plugin for PluginRegistration {
    fn slug(&self) -> &str { self.slug }
    fn manifest(&self) -> Result<&Manifest, &crate::manifest::ManifestError> {
        self.manifest_cache
            .get_or_init(|| crate::manifest::Manifest::parse(self.manifest_toml))
            .as_ref()
    }
}

#[linkme::distributed_slice]
pub static CANOPY_PLUGINS: [PluginRegistration] = [..];

pub trait PluginSource: Send + Sync {
    /// Look up by plugin's `[plugin].slug`. Returns `None` if no plugin
    /// with this identity is registered.
    fn get_plugin(&self, slug: &PluginSlug) -> Option<&dyn Plugin>;

    /// Find the plugin exporting `panel_slug` in its `[plugin.exports].panels`
    /// list AND defining `[panels.<panel_slug>]`. Returns the plugin handle
    /// plus a reference to the panel's definition. Used by the composition
    /// loader to resolve dashboard items.
    fn find_panel(&self, panel_slug: &ItemSlug) -> Option<(&dyn Plugin, &PanelDef)>;

    /// Same shape as `find_panel` but for `[case_sections.<slug>]`. Used
    /// for the CaseDetail surface.
    fn find_case_section(&self, section_slug: &ItemSlug) -> Option<(&dyn Plugin, &CaseSectionDef)>;

    fn iter(&self) -> Box<dyn Iterator<Item = &dyn Plugin> + '_>;
}

pub struct CompileTimePluginSource;

impl PluginSource for CompileTimePluginSource {
    fn get_plugin(&self, slug: &PluginSlug) -> Option<&dyn Plugin> {
        CANOPY_PLUGINS.iter()
            .find(|p| p.slug == slug.0)
            .map(|p| p as &dyn Plugin)
    }

    fn find_panel(&self, panel_slug: &ItemSlug) -> Option<(&dyn Plugin, &PanelDef)> {
        for p in CANOPY_PLUGINS.iter() {
            // Silently skip plugins whose manifest fails to parse — they
            // can't contribute exports. Loader surfaces UnknownPlugin if
            // the requested slug is not exported by any *valid* plugin.
            let Ok(m) = p.manifest() else { continue };
            if m.plugin.exports.panels.iter().any(|s| s == &panel_slug.0)
                && m.panels.contains_key(&panel_slug.0)
            {
                return Some((p as &dyn Plugin, &m.panels[&panel_slug.0]));
            }
        }
        None
    }

    fn find_case_section(&self, section_slug: &ItemSlug) -> Option<(&dyn Plugin, &CaseSectionDef)> {
        for p in CANOPY_PLUGINS.iter() {
            let Ok(m) = p.manifest() else { continue };
            if m.plugin.exports.case_sections.iter().any(|s| s == &section_slug.0)
                && m.case_sections.contains_key(&section_slug.0)
            {
                return Some((p as &dyn Plugin, &m.case_sections[&section_slug.0]));
            }
        }
        None
    }

    fn iter(&self) -> Box<dyn Iterator<Item = &dyn Plugin> + '_> {
        Box::new(CANOPY_PLUGINS.iter().map(|p| p as &dyn Plugin))
    }
}

The proc-macro #[canopy_plugin(slug = "snap-overpayment-summary", manifest = "Plugin.toml")] expands roughly to:

#[linkme::distributed_slice(::canopy_composition::CANOPY_PLUGINS)]
static __CANOPY_PLUGIN_REGISTRATION_<slug-as-uppercase-underscored>: ::canopy_composition::PluginRegistration =
    ::canopy_composition::PluginRegistration {
        slug: "snap-overpayment-summary",
        manifest_toml: include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "/", "Plugin.toml")),
        manifest_cache: ::std::sync::OnceLock::new(),
    };

manifest = "…​" is resolved relative to the invoking crate’s CARGO_MANIFEST_DIR and embedded via include_str!. Per Decision 2, the v1 macro does NOT parse the TOML at expansion time — it only emits the registration. Manifest parsing happens at runtime via Manifest::parse on first Plugin::manifest() call (cached in OnceLock<Result<Manifest, ManifestError>>). If a manifest fails to parse at runtime, the plugin’s manifest() returns Err(…​); the loader surfaces this via a pre-item-validation pass (see loader’s Manifest pre-validation pass in Step 3 below).

Crate layout

crates/canopy-composition/
├── Cargo.toml
├── src/
│   ├── lib.rs                # SPDX header; #![forbid(unsafe_code)]; public re-exports
│   ├── types.rs              # see Core type vocabulary above
│   ├── source.rs             # see PluginSource + plugin registration above
│   ├── manifest.rs           # Plugin.toml schema (Manifest + nested types + ManifestError)
│   ├── idp.rs                # idp.toml schema (Idp + Role + IdpError)
│   ├── defaults.rs           # system_defaults(surface) -> &'static Value via LazyLock + include_str!
│   ├── merge.rs              # apply_merge_patch_7396 + apply_json_patch_6902 (thin json-patch wrappers)
│   ├── role_filter.rs        # filter_items_by_role
│   ├── cache.rs              # CompositionCache (tokio RwLock<HashMap<...>>)
│   ├── jurisdiction.rs       # JurisdictionRegistry trait + StaticJurisdictionRegistry impl
│   ├── audit.rs              # AuditEmitter trait + NoopAuditEmitter impl
│   ├── db.rs                 # fetch_db_layers(pool, ...) -> Vec<(layer, scope_key, patch_ops)>
│   └── loader.rs             # load_composition (orchestrates all of above)
├── defaults/
│   ├── worker_dashboard.json
│   ├── supervisor_dashboard.json
│   ├── analyst_dashboard.json
│   ├── case_detail.json
│   └── sign_in.json          # {"shell": "", "items": []} — Stage 4 fills
├── fixtures/
│   └── Plugin.toml           # canonical test fixture (NOT production location)
└── tests/
    ├── manifest_test.rs
    ├── idp_test.rs
    ├── merge_test.rs
    ├── role_filter_test.rs
    ├── cache_test.rs
    └── loader_test.rs

crates/canopy-plugin-macros/
├── Cargo.toml                # [lib] proc-macro = true
└── src/
    └── lib.rs                # #[canopy_plugin(slug=…, manifest=…)] expansion

Real plugin Plugin.toml files (Stage 5+) live at services/canopy-web/plugins/{slug}/Plugin.toml. The crates/canopy-composition/fixtures/Plugin.toml is for the parser test fixture only.

Migration SQL

File: services/canopy-web/migrations/{YYYYMMDDHHMMSS}_create_composition_documents.sql (timestamp = date -u +%Y%m%d%H%M%S at the moment of file creation).

-- Stage 3 of #460 / epic &51. Closes #489. ADR-022 storage layering.
-- Forward-only per ADR-016.

CREATE TYPE composition_layer AS ENUM ('user', 'role', 'jurisdiction_live');

CREATE TYPE composition_surface AS ENUM (
    'worker_dashboard',
    'supervisor_dashboard',
    'analyst_dashboard',
    'case_detail',
    'sign_in'
);

CREATE TABLE composition_documents (
    id              UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    jurisdiction_id UUID NOT NULL,
    layer           composition_layer NOT NULL,
    scope_key       TEXT NOT NULL,
    surface         composition_surface NOT NULL,
    patch_ops       JSONB NOT NULL,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
    updated_at      TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
    created_by      UUID NOT NULL,
    UNIQUE (jurisdiction_id, layer, scope_key, surface)
);

CREATE INDEX composition_documents_lookup_idx
    ON composition_documents (jurisdiction_id, surface, layer, scope_key);

CREATE TABLE composition_documents_archive (
    LIKE composition_documents INCLUDING DEFAULTS INCLUDING IDENTITY,
    archived_at  TIMESTAMPTZ NOT NULL DEFAULT clock_timestamp(),
    archived_by  UUID NOT NULL
);

CREATE INDEX composition_documents_archive_lookup_idx
    ON composition_documents_archive (jurisdiction_id, surface, archived_at DESC);

Verbatim from ADR-022 §Schema. INCLUDING DEFAULTS INCLUDING IDENTITY (NOT INCLUDING ALL) per the ADR’s archive table note.

Test enumeration

Every test below is a named [test] (or [tokio::test] for async) function. No "30+ tests, TBD" hand-waving.

tests/manifest_test.rs

  • manifest_parses_canonical_fixture — happy path against fixtures/Plugin.toml

  • manifest_rejects_unknown_fielddeny_unknown_fields

  • manifest_rejects_invalid_slug_regex

  • manifest_rejects_default_span_outside_allowed_spans

  • manifest_rejects_allowed_spans_outside_breakpoint_set (ADR-021 line 192 breakpoint set {1, 2, 3, 4, 6, 12})

  • manifest_rejects_program_not_in_known_set

  • manifest_rejects_required_state_not_in_four_set

  • manifest_rejects_audit_outside_enum

  • manifest_rejects_empty_required_roles

  • manifest_rejects_empty_endpoints

  • manifest_rejects_catalogs_missing_default

tests/idp_test.rs

  • idp_parses_canonical_fixture

  • idp_rejects_unknown_field

  • idp_rejects_default_role_not_in_roles_table

tests/merge_test.rs

  • merge_patch_7396_replaces_key

  • merge_patch_7396_null_removes_key

  • merge_patch_7396_passthrough_when_key_omitted

  • json_patch_6902_add_to_array_tail

  • json_patch_6902_remove_array_element

  • json_patch_6902_replace_value

  • json_patch_6902_test_op_passes

  • json_patch_6902_test_op_fails_returns_error

  • json_patch_6902_missing_path_returns_error

tests/role_filter_test.rs

  • role_filter_keeps_items_for_allowed_role

  • role_filter_drops_items_for_denied_role

  • role_filter_drops_items_when_role_unlisted_in_manifest_required_roles

tests/cache_test.rs

  • cache_get_returns_none_when_empty

  • cache_insert_then_get_returns_arc

  • cache_invalidate_removes_single_entry

  • cache_invalidate_jurisdiction_removes_all_entries_for_that_jurisdiction

  • cache_concurrent_read_under_rwlock_does_not_deadlock

tests/loader_test.rs

  • loader_returns_defaults_when_no_baseline_or_db_layers (uses EphemeralSchema-free pure-compute path)

  • loader_applies_jurisdiction_baseline_via_rfc7396 (no DB layers; baseline + defaults)

  • loader_applies_db_layers_in_jurisdiction_live_then_role_then_user_order (uses canopy_test_lib::db::EphemeralSchema if canopy_test_lib::infrastructure_available(); otherwise #[ignore = "needs devstack postgres"])

  • loader_post_merge_span_validation_rejects_out_of_range

  • loader_post_merge_row_overflow_rejects

  • loader_post_merge_unknown_plugin_rejects

  • loader_post_merge_role_not_found_rejects

  • loader_role_filter_applies_after_all_db_layers_merged — explicit ordering invariant per ADR-021 Decision: a user-layer patch adds a panel that role-filter then removes

  • loader_version_hash_is_deterministic (same inputs → same version)

  • loader_version_hash_changes_when_items_reorder

Total: 41 named test functions across 6 test files. The named-test discipline lets reviewers spot-check coverage without running the suite.

Files Touched

NEW:

  • crates/canopy-composition/Cargo.toml + the entire crate tree (14 source files: lib, types, source, jurisdiction, audit, manifest, idp, defaults, merge, role_filter, cache, db, loader, + the crate root + 5 defaults JSON + 1 fixture Plugin.toml + 6 test files = 26 files)

  • crates/canopy-plugin-macros/Cargo.toml + src/lib.rs

  • services/canopy-web/migrations/{YYYYMMDDHHMMSS}_create_composition_documents.sql

  • rulesets/georgia/composition/worker_dashboard.toml

  • rulesets/georgia/composition/case_detail.toml

  • rulesets/georgia/composition/sign_in.toml

  • rulesets/georgia/idp.toml

  • docs/modules/ROOT/pages/plans/archive/worker-portal-redesign-stage3-composition-runtime.adoc (Step 4 moves this scratch)

MODIFIED:

  • Cargo.toml (workspace) — adds crates/canopy-composition + crates/canopy-plugin-macros to [workspace.members]; adds linkme = "0.3" + json-patch = "4.0" to [workspace.dependencies] (versions are the latest stable as of plan authorship 2026-05-21; implementation runs cargo add --workspace linkme json-patch to pull the actual latest-at-MR-open, and updates these pins if newer majors exist). (toml, async-trait, serde, serde_json, sqlx, tokio, uuid, thiserror, chrono, anyhow already exist.)

  • services/canopy-web/Cargo.tomlno change in MR1. MR2 (#491) adds the canopy-composition dep when the write API handlers consume it. Adding the dep in MR1 without a consumer would trip clippy’s unused_crate_dependencies lint.

  • docs/modules/ROOT/pages/plans/worker-portal-redesign.adoc — Stage 3 row description + acceptance + files-touched + Status Not startedIn progress — MR1 !XXX merged; MR2 (#491) ahead

  • .claude/docs/coding-conventions.md — appends "Composition runtime patterns" subsection under "Worker portal patterns"

  • CHANGELOG.adoc=== Added entry under Unreleased (sample text below)

OUT OF SCOPE (deferred):

  • HTTP live-override APIs (#491) — separate MR with its own plan; needs audit/auth/utoipa wiring

  • Real plugin handlers — Stage 5/6

  • Full idp.toml schema (providers, discovery rules, local-accounts) — Stage 4 #493

  • Composition cache eviction (TTL, max-size) — v1 is unbounded; SNAP UAT single-replica makes this safe; post-UAT improvement (file follow-up issue)

  • Multi-replica composition.invalidated RabbitMQ fanout — ADR-021 Decision 3 explicit deferral

  • Deeper #[canopy_plugin] manifest↔handler-signature validation — follow-up issue filed in Step 4

  • canopy composition dump CLI subcommand — follow-up issue filed in Step 4

  • Studio promote-to-baseline affordance — deferred to #507 unified config backend ADR

  • Audit emission from the loader — deferred to MR2 (Decision 14); MR1 includes the Arc<dyn AuditEmitter> hook point defaulted to NoopAuditEmitter

CHANGELOG sample entry

* *Worker portal Stage 3 MR1 — composability runtime + DB migrations (closes \#489, \#490; refs \#460 / epic \&51).* New crate +crates/canopy-composition/+ wraps the 5-layer composition resolver (system defaults via +LazyLock+ → jurisdiction baseline TOML via RFC 7396 → jurisdiction_live → role → user via RFC 6902 op lists per ADR-022) + `PluginSource` trait + `CompileTimePluginSource` (linkme distributed_slice) + invalidate-on-write `tokio::sync::RwLock` cache. New +crates/canopy-plugin-macros/+ provides the +#[canopy_plugin]+ proc-macro for compile-time plugin registration. Forward-only migration creates +composition_documents+ + +composition_documents_archive+ tables (ADR-022 schema verbatim). Roles-only +idp.toml+ schema lands here (Stage 4 #493 extends with IDP providers). Fixture Georgia jurisdiction TOMLs ship for worker_dashboard + case_detail + sign_in surfaces. 41 named tests cover parsers + merge + role-filter + cache + loader end-to-end (including post-merge span/row/plugin/role validation). HTTP write APIs (#491) ship in MR2. Stage-3 implementation plan at +docs/modules/ROOT/pages/plans/archive/worker-portal-redesign-stage3-composition-runtime.adoc+.

Steps

Each step is a discrete commit; the MR rolls them up.

Step 1 — Crate scaffolding + types + proc-macro

Single commit. Lands the structural scaffolding without any I/O or merge logic.

  • crates/canopy-composition/Cargo.toml: name = "canopy-composition", version.workspace = true, edition.workspace = true, license.workspace = true. Deps: serde, serde_json, thiserror, tokio, uuid, chrono (all workspace = true); new workspace deps linkme = "<latest>", json-patch = "<latest>", async-trait = { workspace = true }, toml = { workspace = true }, sqlx = { workspace = true, features = […​] } (postgres + json + uuid + chrono). Dev-deps: canopy-test-lib = { path = "../canopy-test-lib" }.

  • crates/canopy-composition/src/lib.rs: SPDX header; #![forbid(unsafe_code)]; mod declarations; public re-exports.

  • crates/canopy-composition/src/types.rs: see Core type vocabulary section above. SPDX header.

  • crates/canopy-composition/src/source.rs: see Plugin trait + PluginSource + plugin registration section above. SPDX header.

  • crates/canopy-composition/src/jurisdiction.rs: SPDX header. pub trait JurisdictionRegistry: Send + Sync { fn uuid_for(&self, slug: &JurisdictionSlug) → Option<uuid::Uuid>; } + pub struct StaticJurisdictionRegistry { table: HashMap<JurisdictionSlug, uuid::Uuid> } with a StaticJurisdictionRegistry::default() constructor registering JurisdictionSlug("georgia".into())uuid::uuid!("019196a0-0000-7000-8000-000000000001") (a deterministic UUID v7 reserved for the Georgia seed jurisdiction; canonical reference for the seed row landed by canopy-seed Stage 3+. The exact const value is OPEN until canopy-seed lands; Step 3 implementation either (a) reads the actual seed UUID from canopy-seed crate if it has surfaced or (b) picks the literal above + opens a follow-up issue to align canopy-seed with this UUID before MR2).

  • crates/canopy-composition/src/audit.rs: SPDX header. See Decision 15 AuditEmitter trait shape above (AuditEmitter trait + NoopAuditEmitter).

  • crates/canopy-plugin-macros/Cargo.toml: name = "canopy-plugin-macros", [lib] proc-macro = true, deps syn = "2", quote = "1", proc-macro2 = "1". No canopy-composition dep — keeps the proc-macro build cheap (no sqlx/tokio transitive build for host arch). v1 macro does NOT parse TOML; it just extracts slug and emits the registration.

  • crates/canopy-plugin-macros/src/lib.rs: SPDX header; #[proc_macro_attribute] pub fn canopy_plugin(…​) parsing (slug = "…​", manifest = "…​") syntax via syn::parse_macro_input!. Resolves manifest arg as a path literal; emits:

    #[linkme::distributed_slice(::canopy_composition::CANOPY_PLUGINS)]
    static __CANOPY_PLUGIN_REG_<slug_uppercase>: ::canopy_composition::PluginRegistration =
        ::canopy_composition::PluginRegistration {
            slug: "<slug-from-args>",
            manifest_toml: include_str!("<manifest-path-from-args>"),
            manifest_cache: ::std::sync::OnceLock::new(),
        };

    That’s it. No TOML parse, no validation. The proc-macro is a structural sugar around the linkme::distributed_slice push.

  • Root Cargo.toml: add "crates/canopy-composition" + "crates/canopy-plugin-macros" to [workspace.members]. Add linkme + json-patch to [workspace.dependencies] via cargo add --workspace linkme json-patch (pin to whatever cargo add resolves at the time of MR open — latest stable; reviewer must verify version in code review). Verify json_patch::patch and json_patch::merge API surfaces match merge.rs assumptions before commit (Decision 9 risk-flag).

  • Tests (in this commit): smoke test in crates/canopy-composition/src/source.rs [cfg(test)] asserting CompileTimePluginSource::iter() returns an iterator (empty until plugins register). Type assertion tests in types.rs [cfg(test)] for CompositionKey Hash + Eq.

  • Step 1 gate (run before staging the commit):

    CARGO_TARGET_DIR=/home/bitskrieg/code/cargo-target cargo nextest run -p canopy-composition -p canopy-plugin-macros
    CARGO_TARGET_DIR=/home/bitskrieg/code/cargo-target cargo deny check

Step 2 — Schemas + parsers + merge + role filter

Single commit. Adds the pure-compute layers (no I/O).

  • manifest.rs: Plugin.toml schema. Manifest { plugin: PluginMeta, panels: HashMap<String, PanelDef>, case_sections: HashMap<String, CaseSectionDef>, data: DataMeta, permissions: PermissionsMeta, i18n: I18nMeta } where PluginMeta { slug: String, name: String, version: String, author: String, license: String, canopy_min: String, exports: Exports } and Exports { panels: Vec<String>, case_sections: Vec<String> }. The nesting mirrors the TOML’s [plugin.exports] table layout. ManifestError is a thiserror enum (variants per validation failure mode). Inherent method Manifest::parse(toml: &str) → Result<Self, ManifestError> (NOT from_str — avoids name collision with std::str::FromStr) uses toml::from_str + post-deserialization validation (slug regex, default_span ∈ allowed_spans, etc. per Decision 16).

  • idp.rs: Idp { default_role: String, roles: HashMap<RoleSlug, RoleDef> } + IdpError. Inherent method Idp::parse(toml: &str) → Result<Self, IdpError> + Idp::has_role(&self, role: &RoleSlug) → bool.

  • defaults.rs: pub fn system_defaults(surface: ComposableSurface) → &'static serde_json::Value returning LazyLock<Value> initialized via include_str!("../defaults/{surface}.json") + serde_json::from_str. Match arm per surface.

  • merge.rs: pub fn apply_merge_patch_7396(doc: &mut Value, patch: &Value) { json_patch::merge(doc, patch) } + pub fn apply_json_patch_6902(doc: &mut Value, ops: &[json_patch::PatchOperation]) → Result<(), json_patch::PatchError> { json_patch::patch(doc, ops) }. Verify API shape against the actual json-patch crate version installed in Step 1 — adjust signatures if needed.

  • role_filter.rs: pub fn filter_items_by_role(items: &mut Vec<ComposedItem>, role: &RoleSlug, surface: ComposableSurface, source: &dyn PluginSource) (returns ()). Implementation pattern:

    items.retain(|item| {
        let Some((plugin, _def)) = (match surface {
            ComposableSurface::WorkerDashboard
            | ComposableSurface::SupervisorDashboard
            | ComposableSurface::AnalystDashboard => source.find_panel(&item.item).map(|(p, d)| (p, d as &dyn std::any::Any)),
            ComposableSurface::CaseDetail => source.find_case_section(&item.item).map(|(p, d)| (p, d as &dyn std::any::Any)),
            ComposableSurface::SignIn => return true, // no role filter on sign-in
        }) else {
            return false; // export not found — drop silently (the post-merge UnknownPlugin pass already ran, so this shouldn't happen, but defensive)
        };
        // Manifest pre-validation pass in the loader already guaranteed Ok; use if-let-Ok for hygiene rather than unwrap.
        let Ok(manifest) = plugin.manifest() else { return false; };
        manifest.permissions.required_roles.iter().any(|r| r == &role.0)
    });

    ADR-021 keeps [permissions] at plugin level, not per-panel — Decision 16. Silent drop per ADR-021 line 135. The loader’s manifest pre-validation + post-merge UnknownPlugin pass run BEFORE role_filter, so every remaining item has both a valid manifest AND a known export; the defensive return false branches above are practically unreachable but type-safe.

  • 5 defaults/{surface}.json files per the defaults shape section.

  • 1 fixtures/Plugin.toml exercising every field per the Plugin.toml schema example.

  • Tests: 5 new test files per the test enumeration section (manifest_test.rs, idp_test.rs, merge_test.rs, role_filter_test.rs, plus cache_test.rs if cache lands in this step — or move cache to Step 3 since it depends on ComposedSurface).

  • Step 2 gate: cargo nextest run -p canopy-composition clean.

Step 3 — DB migrations + sqlx queries + loader + cache + jurisdiction fixtures

Single commit. Lands the I/O layer + the orchestrating loader.

  • Migration SQL file at services/canopy-web/migrations/{ts}_create_composition_documents.sql (timestamp generated at file-creation time). Content verbatim from the Migration SQL section.

  • db.rs: SPDX header. Declares named row struct:

    #[derive(Debug, sqlx::FromRow)]
    pub struct DbLayer {
        pub layer: CompositionLayer,
        pub scope_key: String,
        pub patch_ops: serde_json::Value,
    }
    
    #[derive(Debug, Clone, Copy, sqlx::Type)]
    #[sqlx(type_name = "composition_layer", rename_all = "snake_case")]
    pub enum CompositionLayer { User, Role, JurisdictionLive }
    
    impl CompositionLayer {
        pub fn as_static_str(self) -> &'static str {
            match self {
                Self::User => "user",
                Self::Role => "role",
                Self::JurisdictionLive => "jurisdiction_live",
            }
        }
    }

    And the function:

    pub async fn fetch_db_layers(
        pool: &PgPool,
        jurisdiction_id: Uuid,
        surface: ComposableSurface,
        role: &RoleSlug,
        user_id: Option<UserId>,
    ) -> Result<Vec<DbLayer>, sqlx::Error> {
        sqlx::query_as::<_, DbLayer>(
            r#"
            SELECT layer, scope_key, patch_ops
              FROM composition_documents
             WHERE jurisdiction_id = $1
               AND surface = $2
               AND ( layer = 'jurisdiction_live'
                  OR (layer = 'role' AND scope_key = $3)
                  OR (layer = 'user' AND scope_key = $4) )
             ORDER BY CASE layer
                WHEN 'jurisdiction_live' THEN 1
                WHEN 'role'              THEN 2
                WHEN 'user'              THEN 3
             END
            "#,
        )
        .bind(jurisdiction_id)
        .bind(surface)        // ComposableSurface: sqlx::Type with type_name="composition_surface", per types.rs
        .bind(&role.0)
        .bind(user_id.map(|u| u.into_inner().to_string()).unwrap_or_default())
        .fetch_all(pool)
        .await
    }

    Runtime form (sqlx::query_as::<_, DbLayer>(SQL).bind(…​).fetch_all(pool).await) per Decision 13 — NOT the compile-time sqlx::query_as!() macro. No DATABASE_URL build-time requirement.

  • cache.rs: SPDX header. pub struct CompositionCache { inner: RwLock<HashMap<CompositionKey, Arc<ComposedSurface>>> } with new, get(&self, key: &CompositionKey) → Option<Arc<ComposedSurface>>, insert(&self, key: CompositionKey, value: Arc<ComposedSurface>), invalidate(&self, key: &CompositionKey), invalidate_jurisdiction(&self, jurisdiction: &JurisdictionSlug). Internally tokio::sync::RwLock so async-friendly + read-concurrent.

  • loader.rs: SPDX header. pub struct CompositionLoader { plugins: Arc<dyn PluginSource>, jurisdictions: Arc<dyn JurisdictionRegistry>, cache: Arc<CompositionCache>, rulesets_root: PathBuf, audit_emitter: Arc<dyn AuditEmitter> } (defaulting audit_emitter to Arc::new(NoopAuditEmitter) for MR1 unit tests; MR2 swaps in the real impl). pub async fn load_composition(&self, pool: &PgPool, jurisdiction: &JurisdictionSlug, role: &RoleSlug, user_id: Option<&UserId>, surface: ComposableSurface, idp: &Idp) → Result<Arc<ComposedSurface>, CompositionLoadError> orchestrates:

    1. Validate idp.has_role(role)RoleNotFound if absent. Runs BEFORE cache lookup so an idp.toml that drops a role doesn’t keep serving stale cached compositions for it. (Note: idp.toml mutation outside this loader’s reach — Stage 4 #493 wires reload semantics. For v1, the loader trusts the &Idp snapshot the caller passes; cache invalidation on idp.toml mutation is the caller’s responsibility per the trade-off documented here.)

    2. let jurisdiction_id = self.jurisdictions.uuid_for(jurisdiction).ok_or(UnknownJurisdiction { slug: jurisdiction.0.clone() })?;

    3. Build CompositionKey { jurisdiction, role, user_id, surface }. Cache check via self.cache.get(&key) → early-return on hit.

    4. let mut working: serde_json::Value = system_defaults(surface).clone(); (mutable working document).

    5. Try-read rulesets_root/{jurisdiction.0}/composition/{surface_name}.toml. On Ok: toml::from_str::<serde_json::Value>apply_merge_patch_7396(&mut working, &baseline). On Err(io.kind == NotFound): // SILENT-OK: baseline absence falls through to defaults. Other `Err`s propagate.

    6. let db_layers = db::fetch_db_layers(pool, jurisdiction_id, surface, role, user_id.copied()).await?; Returns Vec<DbLayer> already ordered jurisdiction_live → role → user (the SQL ORDER BY CASE layer does it).

    7. For each DbLayer { layer, scope_key: _, patch_ops }: deserialize patch_ops into Vec<json_patch::PatchOperation>; call apply_json_patch_6902(&mut working, &ops); on Err, map to PatchFailed { layer: layer.as_static_str(), source }.

    8. Deserialize working into the intermediate RawComposition { shell: String, items: Vec<ComposedItem> } via serde_json::from_value. On Err, surface PostMergeShape.

    9. Manifest pre-validation pass: walk every plugin in self.plugins.iter() and call Plugin::manifest(). If any returns Err(parse_err), surface CompositionLoadError::ManifestParse(parse_err.to_string()) immediately (the error variant carries an owned String — see Decision 6 update + ManifestError Display impl via thiserror). After this pass, every registered plugin has a parsed Manifest cached.

    10. Export-resolution pass (surface-aware lookup — Decision 16; UnknownPlugin only, NO span/row checks yet):

      • Pick the lookup function by surface:

        • WorkerDashboard | SupervisorDashboard | AnalystDashboardself.plugins.find_panel(&item.item) returning Option<(&dyn Plugin, &PanelDef)>

        • CaseDetailself.plugins.find_case_section(&item.item) returning Option<(&dyn Plugin, &CaseSectionDef)>

        • SignIn → skip item validation (sign-in items are IDP entries; Stage 4 #493 wires)

      • For each item: call the chosen lookup; on None, surface UnknownPlugin { slug: item.item.0.clone() } (the variant’s slug field carries the export slug that didn’t resolve, not a plugin slug — error message reads "no plugin exports <slug>`"). After the pre-validation pass above, this can only mean "no plugin’s `[plugin.exports] lists this slug", not "a manifest was broken".

    11. role_filter::filter_items_by_role(&mut raw.items, role, surface, self.plugins.as_ref()); — surface-aware in-place silent-drop. Items whose plugin’s [permissions].required_roles (ADR-021 schema keeps [permissions] at the plugin level, NOT per-panel) excludes role are dropped. Signature takes surface so it picks the right lookup fn.

    12. Span + row constraint validation pass (runs AFTER role-filter so panels dropped for the requesting role don’t trigger spurious RowOverflow):

      • For each remaining item, look up the export def via the surface-appropriate find_panel / find_case_section.

      • Assert def.allowed_spans.contains(&item.span) else SpanOutOfRange { slug: item.item.0.clone(), span, allowed: def.allowed_spans.clone() }.

      • Group items by row; sum spans per row; assert each row’s sum ≤ 12 else RowOverflow.

    13. Map raw.shell (a String) to a typed ShellSpec per surface BEFORE hashing — so the hash reflects the resolved typed shell, not the raw string:

      • WorkerDashboard / Supervisor / Analyst: parse as "grid"WorkerDashboardLayout::Grid, "stacked"Stacked. Default Grid. Wrap in ShellSpec::WorkerDashboard { layout } (or Supervisor/Analyst variant).

      • CaseDetail: parse as "scroll"|"card_grid"|"tabs". Default Tabs. Wrap in ShellSpec::CaseDetail { shell }.

      • SignIn: ShellSpec::SignIn (raw.shell ignored).

    14. Build the unversioned pre-hash form: let pre_version = serde_json::to_value(serde_json::json!({"surface": surface, "shell": &shell, "items": &raw.items})).expect("pre_version serialization is total");.

    15. Compute version = u64::from_be_bytes(sha256(canonical_rfc8785_json(&pre_version))[..8]) — first 8 bytes per ADR-021 line 133. The hash input is the post-merge + post-role-filter + post-shell-normalization document.

    16. Assemble let composed = ComposedSurface { surface, shell, items: raw.items, version };

    17. let arc = Arc::new(composed); self.cache.insert(key.clone(), Arc::clone(&arc));

    18. self.audit_emitter.emit_render(…​) (NoopAuditEmitter in MR1 unit tests).

    19. Return Ok(arc).

  • rulesets/georgia/composition/worker_dashboard.tomlshell-only in MR1: shell = "grid" + empty items = []. Stage 5 extends with real items.

  • rulesets/georgia/composition/case_detail.tomlshell-only in MR1: shell = "tabs" + empty items = []. (items is the canonical field name across all surfaces per RawComposition; case-detail sections fill in Stage 5 as items referencing [case_sections.*] export slugs.) Stage 5 extends with real section items.

  • rulesets/georgia/composition/sign_in.toml — minimal stub (Stage 4 will fill)

  • rulesets/georgia/idp.toml — 4 roles per idp.toml schema section

  • services/canopy-web/Cargo.tomlno change in MR1 per the Files Touched MODIFIED note above. The migration ships under services/canopy-web/migrations/ (so sqlx::migrate!("./migrations") in services/canopy-web/src/main.rs picks it up at next startup), but no canopy-web source code references canopy-composition until MR2 wires the write APIs.

  • Tests: tests/cache_test.rs (5 named tests) + tests/loader_test.rs (10 named tests). The loader tests requiring DB use canopy_test_lib::db::EphemeralSchema for setup; gate via canopy_test_lib::infrastructure_available() per coding-conventions §Integration Tests. Tests that don’t need DB (defaults-only, baseline-only, validation rejections) run unconditionally.

  • Step 3 gate:

    CARGO_TARGET_DIR=/home/bitskrieg/code/cargo-target cargo nextest run -p canopy-composition
    cargo xtask dev migrate    # applies the migration on devstack

Step 4 — Plan finalize + parent plan + CHANGELOG + coding-conventions

Single commit. Doc + plan finalize.

  • Copy this plan body (the == Worker portal redesign — Stage 3 MR1 section onward) verbatim into docs/modules/ROOT/pages/plans/archive/worker-portal-redesign-stage3-composition-runtime.adoc. The body is already AsciiDoc syntax so no conversion needed.

  • Status table all four rows marked Done (YYYY-MM-DD).

  • Update docs/modules/ROOT/pages/plans/worker-portal-redesign.adoc Stage 3 row description + acceptance + files-touched per the Files Touched section above. Status: Not startedIn progress — MR1 !XXX merged; MR2 (#491) ahead.

  • Add the new plan to docs/modules/ROOT/nav.adoc per coding-conventions §Plan Authoring (line 222): "All plans MUST be saved as .adoc files under docs/modules/ROOT/pages/plans/ and linked in nav.adoc." Insert as a peer entry to the other Stage plans. (Stage 1 + Stage 1.5 plans are not currently in nav — backfilling them is out of scope for this MR; file as separate hygiene issue.)

  • Append "Composition runtime patterns" subsection to .claude/docs/coding-conventions.md under the existing "Worker portal patterns" section. Subsection covers: how to declare a Plugin.toml + use #[canopy_plugin]; how load_composition is invoked from a request handler (preview shape for MR2); the four-state primitives requirement from Stage 1.5 carries into composition items (every plugin’s [panels.*].required_states = ["empty","loading","error","populated"]).

  • CHANGELOG.adoc === Added entry under == Unreleased — use the sample text from the CHANGELOG sample entry section above verbatim.

  • File three follow-up GitLab issues via glab issue create referenced from CHANGELOG: (a) "feat: canopy composition dump CLI subcommand"; (b) "spec: deeper #[canopy_plugin] manifest↔handler validation"; (c) "feat: composition cache eviction (TTL + max-size) post-UAT". Use scoped labels per .claude/CLAUDE.md: type::feature/type::spike, priority::low, service::shared-crates, workflow::needs-spec.

  • Scratch plan file (~/.claude/plans/elegant-tinkering-pudding.md) is the only file outside the canopy repo; it gets cleaned up by the next plan-mode session (which writes a fresh plan over it) per the plan-mode workflow. No explicit rm required.

  • Step 4 gate:

    # Status-vocabulary lint (per ADR-013 + coding-conventions:250).
    cargo xtask docs plan-lint
    
    # AsciiDoc structural lint (in-house binary per
    # reference_asciidoctor_lint memory + coding-conventions §AsciiDoc lint).
    /home/bitskrieg/code/cargo-target/debug/asciidoctor-lint \
      docs/modules/ROOT/pages/plans/archive/worker-portal-redesign-stage3-composition-runtime.adoc \
      docs/modules/ROOT/pages/plans/worker-portal-redesign.adoc \
      docs/modules/ROOT/nav.adoc \
      CHANGELOG.adoc

Verification

Per-step gates

See each Step section above for the exact gate commands.

Stage acceptance

  • crates/canopy-composition/ compiles; all 41 named tests in tests/* pass

  • crates/canopy-plugin-macros/ compiles. Acceptance softened: no plugin consumers exist in MR1 (real plugins land Stage 5), so the proc-macro is only exercised by cargo build -p canopy-plugin-macros. A trybuild smoke test against a synthetic consumer is OPEN — file as a follow-up issue if Stage 5 doesn’t naturally exercise the macro by then.

  • cargo deny check clean after json-patch + linkme additions

  • Migration applies on a fresh devstack: composition_documents + composition_documents_archive tables exist with the right indexes + unique constraint + enums

  • load_composition(georgia, eligibility_worker, None, WorkerDashboard, &idp) against the MR1 fixture baseline returns a ComposedSurface with items: [] (Georgia MR1 baseline is shell-only since no plugins register) + a deterministic version hash; adding items to the baseline (a Stage 5 follow-up MR) will produce a different version. The named test loader_returns_defaults_when_no_baseline_or_db_layers verifies the empty-path; tests using TestPluginSource verify the populated-path with synthetic fixture plugins.

  • Role filtering: same call with role = RoleSlug("qc".into()) returns items: [] in MR1 (no items to filter). The named test loader_role_filter_applies_after_all_db_layers_merged uses TestPluginSource to verify the filter works against a synthesized item set.

  • Cache invariant: two consecutive calls with same key return the same Arc<ComposedSurface>; cache.invalidate(key) forces re-load

  • Manifest validation rejects allowed_spans containing values outside the breakpoint set {1, 2, 3, 4, 6, 12} per ADR-021 line 192. New test manifest_rejects_allowed_spans_outside_breakpoint_set covers.

  • Plan + parent plan + CHANGELOG + coding-conventions all updated

  • Three follow-up issues filed and linked in CHANGELOG

  • cargo xtask validate full pipeline clean before push: fmt + clippy + nextest + check-docs + Playwright E2E ≥ 139 green (no E2E changes expected — composition runtime is internal)

  • Zero new [allow] / unwrap outside tests / unsafe / TODO / FIXME tokens (verified by grep -rn "TODO\|FIXME\|unwrap()\|\[allow" crates/canopy-composition/src/ crates/canopy-plugin-macros/src/)

What this MR does NOT gate

  • HTTP write APIs — MR2 (#491)

  • Real plugin handlers — Stage 5 (#495-#498)

  • Studio UI — Stage 6 (#499-#501)

  • Multi-replica cache fanout — post-UAT

  • cargo composition dump CLI — separate canopy-cli issue (filed in Step 4)

  • Audit emission from loader — MR2

Risk + Rollback

  • Risk — json-patch crate’s current API surface differs from what merge.rs assumes. Mitigation: Step 2 explicitly verifies the API shape (json_patch::patch(&mut Value, &[PatchOperation]) and json_patch::merge(&mut Value, &Value)) on adoption; if either function moved or renamed, merge.rs adapts. The wrapper indirection makes the adaptation local.

  • Risk — sqlx compile-time query_as! macro requires DATABASE_URL at build time. Mitigation: Decision 13 picks the runtime function form sqlx::query_as::<_, DbLayer>(SQL).bind(…​).fetch_all(pool) (NOT the query_as! macro) to match existing canopy-web pattern. If team prefers compile-time, that’s a cross-canopy follow-up (sqlx-offline cache).

  • Risk — linkme + proc-macro hygiene issues across cargo profiles. Mitigation: ADR-021 documents a build.rs fallback (Option B) if linkme breaks. Step 1’s smoke test catches the common failure modes (empty slice, non-Send element, etc.). If linkme is a problem on Alpine musl builds (docker target), the fallback lands as a follow-up.

  • Risk — RFC 6902 patch ops can be authored to violate Plugin.toml constraints (e.g., {"op":"replace","path":"/items/0/span","value":999} setting a span outside allowed_spans). Mitigation: the loader’s post-merge validation walks every item; the four CompositionLoadError variants (SpanOutOfRange, RowOverflow, RoleNotFound, UnknownPlugin) surface these. Tests loader_post_merge_*_rejects exercise each.

  • Risk — fixture composition TOML drifts from defaults/{surface}.json over time. Mitigation: every loader_test.rs test that uses fixtures asserts against a specific expected ComposedSurface; drift fails the test. CHANGELOG entries for defaults JSON changes are required and flagged in code review.

  • Risk — Step 3 loader_test.rs DB tests fail in CI without devstack. Mitigation: canopy_test_lib::infrastructure_available() gate per coding-conventions skips the DB tests when no postgres is reachable. CI pipeline already runs with devstack up; only local runs without cargo xtask dev start hit the skip path.

  • Rollback: revert the MR. Migration drops via a fresh forward migration per ADR-016:

    DROP TABLE composition_documents_archive;
    DROP TABLE composition_documents;
    DROP TYPE composition_surface;
    DROP TYPE composition_layer;

    Since composition_documents is greenfield with no writes yet (MR2 ships writes), a revert in v0 is harmless: no data loss, no chain integrity concern.

Pre-commit subagent Q1-Q8 expectations

Aligned with the SUBAGENT-facing Q1-Q8 set inside .githooks/pre-commit (the set the subagent verifies against the staged diff at commit time):

  • Q1 — Have new code paths been added without corresponding tests? Every public function in Steps 1-3 ships with at least one named test. 41 named tests across 6 test files enumerated above. Smoke tests in source.rs + types.rs cover Step 1’s non-test-file code.

  • Q2 — Are there any hacks, bypasses, or // FIXME / // HACK comments? No unwrap outside [cfg(test)]; no unsafe; no [allow]; no FIXME/HACK comments. #![forbid(unsafe_code)] on every new lib.rs.

  • Q3 — Have any tests been weakened? No existing tests touched. All new tests use positive assertions; no #[ignore] added.

  • Q4 — Are there deviations from the plan? If material deviations surface, update plan Design section + file separate issues per ADR-013. Plan deviations encountered during Step 2-3 implementation must be reflected back into the durable plan in Step 4 before commit.

  • Q5 — Are there new endpoints, tables, events, or commands that aren’t reflected in services.md / CLAUDE.md / CHANGELOG.adoc? New tables (composition_documents, composition_documents_archive) + new types (composition_layer, composition_surface) are documented in Step 4’s CHANGELOG entry + parent plan files-touched. No new endpoints in MR1 (MR2 adds those). No new commands.

  • Q6 — Are there TODO/FIXME/stub tokens added without a linked GitLab issue? Zero new TODO/FIXME tokens. The audit_emitter: Arc<dyn AuditEmitter> field defaulted to NoopAuditEmitter is a trait seam, not a TODO — MR2 swaps in the real emitter from canopy-security.

  • Q7 — Are there any silently-discarded Result/Option values without a // SILENT-OK comment? load_composition’s baseline-TOML fallthrough deliberately swallows `std::io::ErrorKind::NotFound for the optional baseline file with an inline // SILENT-OK: baseline TOML absence falls through to defaults per ADR-022 comment. All other Result`s propagate via `? or are explicitly converted to CompositionLoadError.

  • Q8 — Are any new .rs files missing the SPDX header? Every new .rs file in crates/canopy-composition/src/, crates/canopy-composition/tests/, and crates/canopy-plugin-macros/src/ opens with // SPDX-License-Identifier: AGPL-3.0-or-later. lib.rs files additionally carry #![forbid(unsafe_code)] per coding-conventions.

References

Edit this page · default