Plan: OIDC Pluggability Refactor (Issue #422)

On this page

Status

Step Description Status

1

New module crates/canopy-auth/src/discovery.rs (SPDX header). Shape: pub struct OidcDiscovery { pub issuer: String, pub authorization_endpoint: String, pub token_endpoint: String, pub jwks_uri: String, pub end_session_endpoint: Option<String>, pub userinfo_endpoint: Option<String> } with Debug, Clone, Serialize, Deserialize derives. Async OidcDiscovery::fetch(issuer_url: &str, http: &reqwest::Client) → Result<Self, AuthError> GETs {issuer_url}/.well-known/openid-configuration (issuer URL already includes the realm path for Keycloak — e.g. http://keycloak:8080/realms/canopy/.well-known/openid-configuration resolves correctly). Honours Cache-Control: max-age=N from the response if present; otherwise defaults to 300 s. Parser is permissive: tolerates Okta-style extra fields (introspection_endpoint, etc.) via #[serde(default, deny_unknown_fields = false)]. Cache is per-issuer-URL via tokio RwLock<HashMap<String, CachedDiscovery>> at module level; CachedDiscovery { doc: Arc<OidcDiscovery>, expires_at: DateTime<Utc> }. Concurrent fetches single-flight via the same per-key mutex pattern used in canopy-mq’s reconnect (crates/canopy-mq/src/connection.rs). 6 unit tests: (a) Keycloak shape parses, (b) Okta shape parses, (c) Auth0 shape parses, (d) cache hit avoids second HTTP call, (e) cache expiry triggers re-fetch, (f) malformed JSON → AuthError::DiscoveryFailed.

Not started

2

JwksProvider::from_discovery(discovery: &OidcDiscovery) — async constructor that builds the provider with discovery.jwks_uri as the explicit fetch URL. The legacy JwksProvider::new(issuer) (with the /protocol/openid-connect/certs Keycloak-path fallback) gets a #[deprecated(note = "use from_discovery")] annotation but stays compileable for one MR cycle so devstack tests don’t break mid-flight. Drop the deprecated path in the dependent BFF token-refresh MR.

Not started

3

Rename ServiceSettings.keycloak_issuer → oidc_issuer, ServiceSettings.keycloak_url → oidc_internal_url. Field-level serde rename: keep accepting the old keycloak_* env-var name during this MR via #[serde(alias = "keycloak_issuer")] so docker-compose and .env files don’t have to land in lockstep. Drop the alias in the dependent MR.

Not started

4

Rewrite crates/canopy-api/src/bootstrap.rs:67-76 to: (a) fetch discovery via OidcDiscovery::fetch(&settings.oidc_issuer).await?, (b) build the JWKS provider via JwksProvider::from_discovery(&discovery) + with_audience("canopy"), (c) stash the discovery doc in BootstrapResult so canopy-web can reuse it without a second discovery fetch.

Not started

5

Refactor services/canopy-web/src/auth.rs to use the discovery doc. Rename oidc_external_url → oidc_external_issuer, oidc_internal_url → oidc_internal_issuer. The internal/external split is canopy-specific: the browser sees localhost:8180 while the BFF talks to keycloak:8080 over the docker network. Each issuer URL has its OWN discovery doc (Keycloak reports endpoints relative to the URL the discovery was fetched from). So canopy-web fetches discovery TWICE at startup: once at oidc_external_issuer (used for auth_endpoint redirect — browser-visible URL), once at oidc_internal_issuer (used for token_endpoint + end_session_endpoint — server-to-server). Both cache independently in OidcDiscovery::fetch’s per-URL cache. Replace the three hardcoded paths (`services/canopy-web/src/auth.rs:110, 150, 277) accordingly: line 110 uses external.authorization_endpoint, line 150 uses internal.token_endpoint, line 277 uses internal.end_session_endpoint.

Not started

6

Rename services/canopy-web/src/config.rs:13-15 config keys: keycloak_client_id → oidc_client_id, keycloak_external_url → oidc_external_issuer, keycloak_internal_url → oidc_internal_issuer. Use #[serde(alias = …​)] to accept the old names during this MR.

Not started

7

Update config/canopy-web/default.yaml keys (3 lines). Update config/canopy-applications/default.yaml if it sets keycloak_*.

Not started

8

Update docker-compose.yml env vars: CANOPY_<SERVICE>KEYCLOAK_ISSUER → CANOPY_<SERVICE>OIDC_ISSUER (17 services × 2 vars = ~34 lines). Update the top-level KEYCLOAK_ISSUER default to OIDC_ISSUER (the docker-compose variable, not the env var inside containers).

Not started

9

Update .env, .env.local, .env.example, .envrc references. Check pre-push hook for any KEYCLOAK_* validation.

Not started

10

Tests: keep JwksProvider::new tests working (they use the deprecated path until the dependent MR). New crates/canopy-auth/tests/discovery_test.rs (~6 tests) covering the discovery fetch + cache + JwksProvider construction across Keycloak / Okta / Auth0 shapes. Update crates/canopy-auth/src/middleware.rs:91 test setup to use the discovery path.

Not started

11

JWKS-rotation interaction with discovery cache. The existing JwksProvider::start_refresh_task() periodically re-fetches JWKS to handle key rotation. With Step 2’s from_discovery constructor, the JWKS URL is sourced from the cached discovery doc. If the IdP rotates AND moves the JWKS URL (rare but possible), the JwksProvider would still hit the old URL. Resolution: the JwksProvider’s refresh task consults OidcDiscovery::cached_or_fetch on each refresh (cheap because of the 300 s cache), then refreshes JWKS from the current jwks_uri. 1 unit test covering the URL-change-mid-runtime case.

Not started

12

Docs sync. New docs/modules/ROOT/pages/idp-integration.adoc covering: which OIDC providers are supported (Keycloak default, Okta and Auth0 confirmed via test fixtures in Step 1, Azure AD untested but should work), what config keys to set per provider, how to run devstack with a non-Keycloak provider for testing. CHANGELOG == Unreleased / === Changed. CLAUDE.md "Tech Stack" line that says "Identity: Keycloak…​" gets a "(default; any RFC 6749 + OIDC discovery provider works via config)" annotation. Plan moves to plans/archive/oidc-pluggability-refactor.adoc post-merge.

Not started

Issue: #422
Branch: refactor/oidc-pluggability
Labels: type::refactor, priority::high, program::infrastructure, service::shared-crates, service::web, workflow::ready

Context

Pre-existing tech debt: canopy-auth + canopy-web hardcode Keycloak-specific OIDC paths (/protocol/openid-connect/{auth,token,logout,certs}) and use keycloak_* config keys throughout. This bakes Keycloak as the only supported IdP — operators using Okta, Auth0, or Azure AD would have to fork the codebase to retarget endpoint paths.

The standard fix is OIDC .well-known/openid-configuration discovery: every compliant OIDC provider exposes a metadata document at the issuer URL with the actual endpoint URLs inside (RFC 8414). Use that, drop the hardcoded paths, and the codebase becomes provider-neutral.

This refactor is the foundation for the BFF token-refresh fix (#411). Without provider-neutral discovery in place first, the #411 fix would extend Keycloak coupling rather than repair it. Per user direction during Tier A planning (2026-05-05): "we’re already touching the code in that area, do the strategic refactor now."

Code references for the existing coupling

  • crates/canopy-common/src/settings.rs:23,30 — ServiceSettings.keycloak_issuer + keycloak_url field names.

  • crates/canopy-api/src/bootstrap.rs:67-69 — three uses of those fields.

  • crates/canopy-auth/src/jwks.rs:65 — Keycloak-path JWKS fallback (format!("{}/protocol/openid-connect/certs", …​)).

  • services/canopy-web/src/config.rs:13-15 — keycloak_client_id, keycloak_external_url, keycloak_internal_url config keys.

  • services/canopy-web/src/auth.rs:110, 150, 277 — three hardcoded path strings (/protocol/openid-connect/{auth,token,logout}).

  • docker-compose.yml — 17 services × 2 env vars (CANOPY_<SERVICE>KEYCLOAK_ISSUER, CANOPY_<SERVICE>KEYCLOAK_URL) ≈ 34 lines.

  • config/canopy-web/default.yaml, config/canopy-applications/default.yaml — YAML config keys.

  • .env, .env.local, .env.example, .envrc — env-var references.

  • Tests: crates/canopy-auth/src/jwks.rs:198-243, crates/canopy-auth/src/middleware.rs:91.

Net edit volume: ~25 file edits, mostly mechanical (rename + 1 new helper module).

Scope

In scope:

  • OidcDiscovery::fetch helper that GETs the standard discovery doc, parses, caches per-URL.

  • JwksProvider::from_discovery async constructor.

  • ServiceSettings.keycloak_* → oidc_* rename across the codebase (with serde aliases for one MR cycle of back-compat).

  • Rewrite services/canopy-web/src/auth.rs to use discovery-driven endpoint URLs (fetched separately for browser-visible vs server-side usage).

  • docker-compose, config YAML, .env* rename to OIDC_*.

  • New idp-integration.adoc documenting which providers are supported.

Out of scope:

  • Removing the back-compat #[serde(alias)] and the deprecated JwksProvider::new — those drop in the dependent MR (#411 BFF token refresh) once both have shipped together.

  • Azure AD smoke test — documented as untested. File a follow-up if a tenant becomes available.

  • Service-account / client-credentials flow — separate concern, file when needed.

  • Token refresh handling — that’s the dependent MR (#411).

Dependencies

  • crates/canopy-auth/Cargo.toml — reqwest + serde_json already present (no new deps).

  • crates/canopy-common/src/settings.rs — field rename source.

  • crates/canopy-api/src/bootstrap.rs — wires discovery + stashes OidcDiscovery in BootstrapResult.

  • services/canopy-web/src/config.rs, services/canopy-web/src/auth.rs — local consumers.

  • docker-compose.yml, config/*/default.yaml, .env — devstack glue.

No schema migrations. No new workspace dependencies.

Design

OidcDiscovery shape

// crates/canopy-auth/src/discovery.rs

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(default, deny_unknown_fields = false)]
pub struct OidcDiscovery {
    pub issuer: String,
    pub authorization_endpoint: String,
    pub token_endpoint: String,
    pub jwks_uri: String,
    pub end_session_endpoint: Option<String>,
    pub userinfo_endpoint: Option<String>,
}

impl OidcDiscovery {
    pub async fn fetch(
        issuer_url: &str,
        http: &reqwest::Client,
    ) -> Result<Arc<Self>, AuthError> {
        // 1. Cache lookup (per-URL, single-flight via mutex map).
        // 2. GET {issuer_url}/.well-known/openid-configuration
        // 3. Parse Cache-Control: max-age into expires_at; default 300 s.
        // 4. Insert into cache, return Arc.
    }

    pub async fn cached_or_fetch(
        issuer_url: &str,
        http: &reqwest::Client,
    ) -> Result<Arc<Self>, AuthError> {
        // Returns cached doc if not expired; otherwise calls fetch().
    }
}

Internal vs external discovery split

The internal/external URL split is canopy-specific (docker network has a different hostname than the host). Each issuer URL has its OWN discovery doc — Keycloak reports endpoints relative to the URL discovery was fetched from. So canopy-web fetches discovery twice at startup:

Each cached independently in `OidcDiscovery’s per-URL cache.

JWKS rotation + discovery cache interaction

The existing JwksProvider::start_refresh_task() periodically re-fetches JWKS. With from_discovery, the JWKS URL is sourced from the cached discovery doc. If the IdP rotates AND moves the JWKS URL (rare), the JwksProvider would still hit the old URL.

Resolution: the JwksProvider’s refresh task calls OidcDiscovery::cached_or_fetch on each refresh (cheap because of the 300 s cache), then refreshes JWKS from the current jwks_uri. This way a JWKS-URL change is picked up within one cache TTL.

Files Touched

File Change

crates/canopy-auth/src/discovery.rs

New module — OidcDiscovery + fetch + cache.

crates/canopy-auth/src/jwks.rs

Add from_discovery; deprecate new.

crates/canopy-auth/src/lib.rs, crates/canopy-auth/src/middleware.rs

Re-export discovery; update test fixtures.

crates/canopy-common/src/settings.rs

Rename keycloak_issuer → oidc_issuer, keycloak_url → oidc_internal_url. Add #[serde(alias = "keycloak_*")].

crates/canopy-api/src/bootstrap.rs

Use discovery, stash OidcDiscovery in BootstrapResult.

services/canopy-web/src/config.rs

Config-key rename + serde aliases.

services/canopy-web/src/auth.rs

Replace 5 hardcoded paths with discovery.{authorization,token,end_session}_endpoint.

config/canopy-web/default.yaml, config/canopy-applications/default.yaml

YAML key rename.

docker-compose.yml

Env-var rename across 17 services (~34 lines).

.env, .env.local, .env.example, .envrc

Env-var rename.

crates/canopy-auth/tests/discovery_test.rs

New (~6 tests covering Keycloak/Okta/Auth0 + cache + URL-change).

docs/modules/ROOT/pages/idp-integration.adoc

New page documenting supported providers + config.

CHANGELOG.adoc

== Unreleased / === Changed entry.

.claude/CLAUDE.md

Tech Stack line annotation: "Identity: Keycloak (default; any RFC 6749 + OIDC discovery provider works via config)".

docs/modules/ROOT/pages/plans/oidc-pluggability-refactor.adoc

This plan; moves to plans/archive/ post-merge.

No schema migrations. No new workspace dependencies (reqwest already present in canopy-auth).

Verification

Per-step

  1. cargo nextest run -p canopy-auth — discovery + JWKS tests pass against stubbed shapes.

  2. cargo xtask dev start — devstack still boots; every service successfully fetches discovery from Keycloak at startup.

  3. cargo xtask validate — full battery green; no service silently regresses on JWT validation.

End-to-end

  1. With devstack up, every existing endpoint that requires auth still accepts a worker JWT (regression: cargo xtask e2e baseline still passes).

  2. Boot a fresh devstack with KEYCLOAK_ISSUER env var unset and OIDC_ISSUER set instead — services come up clean. Old name still accepted via #[serde(alias)].

  3. Optional smoke: stand up a test Okta tenant + flip CANOPY_WEB__OIDC_INTERNAL_ISSUER to point at it. Login flow works. (Out-of-band; documented in idp-integration.adoc.)

Risk + Rollback

Risk: discovery fetch at service startup is a new external dep on the IdP being reachable when each service boots. If Keycloak is slow to come up in devstack, services block.
Mitigation: OidcDiscovery::fetch retries with exponential backoff (matches the existing JWKS fetch retry pattern in crates/canopy-auth/src/jwks.rs). 30-second wait is the upper bound — same envelope as today’s bootstrap.

Risk 2: a misconfigured discovery URL causes 17 services to fail-fast at boot.
Mitigation: cargo xtask dev start already health-checks every service before declaring devstack up; the health-check timeout surfaces the misconfiguration loudly. Bonus: cargo xtask validate adds a discovery-fetch smoke test in Step 10.

Rollback: this MR is mostly mechanical rename + 1 new helper. Revert the MR; no schema or wire-format changes to unwind.

Potential Improvements

(Out of scope; file separately if/when relevant.)

  • Azure AD smoke test — confirm Microsoft’s discovery + JWKS handling matches the implementation. Documented in idp-integration.adoc as untested. File when a tenant becomes available.

  • Per-IdP documentation — idp-integration.adoc covers Keycloak/Okta/Auth0; add Azure AD, Cognito, Ory Hydra etc. as operators need them.

  • Service-account / client-credentials flow — separate concern, file when the first background-job consumer needs it.

Errata

(none)

Edit this page · default