Plan: OpenAPI Contract Testing

On this page

Status

Deferred — APIs are still evolving pre-UAT. Contract testing on unstable APIs creates friction (constant baseline updates) without catching real bugs. Implement after SNAP UAT (September 2026) when API surfaces stabilize.

Step Description Status

1

Update service list in api_docs.rs to cover all 14 active services

Deferred (post-UAT — see plan-level Status; tracked at #352)

2

Add --check flag for CI enforcement (non-zero exit on diff)

Deferred (post-UAT — tracked at #352)

3

Add breaking-change detection (removed paths, changed types, removed required fields)

Deferred (post-UAT — tracked at #352)

4

Add CI job openapi-contract-check to .gitlab-ci.yml

Deferred (post-UAT — tracked at #352)

5

Commit current baselines for all 14 services

Deferred (post-UAT — depends on Steps 1-3 stabilising; baselines committed today churn on every API edit; tracked at #352)

6

Unit tests for diff detection logic

Deferred (post-UAT — depends on Steps 1-3; tracked at #352)

Branch: chore/openapi-contract-testing

Context

cargo xtask api-docs already exists at xtask/src/cmd/api_docs.rs. It fetches OpenAPI specs from running devstack services via HTTP and stores them as JSON baselines in test-results/openapi/. However:

  1. Incomplete service list: Only 11 services are listed. Missing: canopy-tanf (8014), canopy-medicaid (8015), canopy-web (8080 — BFF, no OpenAPI).

  2. No CI enforcement: The --update flag regenerates baselines, but there is no --check flag that exits non-zero when specs differ from baseline.

  3. Naive diff: Current implementation compares raw JSON strings. It cannot distinguish breaking changes (removed endpoint, changed required field) from non-breaking additions (new optional field, new endpoint).

The OpenAPI JSON endpoint for all API services is at /api-doc/openapi.json (served by utoipa-swagger-ui). BFF services (canopy-web, canopy-portal) do not serve OpenAPI — skip them.

Design

Service Registry

Update xtask/src/cmd/api_docs.rs service list:

const SERVICES: &[(&str, &str, u16)] = &[
    ("rules", "canopy-rules", 8001),
    ("persons", "canopy-persons", 8002),
    ("applications", "canopy-applications", 8003),
    ("eligibility", "canopy-eligibility", 8004),
    ("verification", "canopy-verification", 8005),
    ("enrollment", "canopy-enrollment", 8006),
    ("renewals", "canopy-renewals", 8007),
    ("notices", "canopy-notices", 8008),
    ("appeals", "canopy-appeals", 8010),
    ("reporting", "canopy-reporting", 8011),
    ("security", "canopy-security", 8012),
    ("snap", "canopy-snap", 8013),
    ("tanf", "canopy-tanf", 8014),
    ("medicaid", "canopy-medicaid", 8015),
];

Excluded: canopy-web (BFF, no OpenAPI), canopy-portal (stub BFF), canopy-exchange/caps/wic (stubs with no domain routes).

Breaking Change Detection

A breaking change is any modification that would cause existing API clients to fail:

  • Path removed: An endpoint in the baseline is absent in current spec

  • Required field added to request body: Client must now send a new field

  • Required field removed from response: Client relied on a field that no longer exists

  • Type changed: Field type changed (e.g., string → integer)

  • HTTP method removed on existing path

Non-breaking changes (allowed without CI failure):

  • New path added

  • New optional field in request or response

  • Description/summary text changes

--check Flag Behavior

When invoked with cargo xtask api-docs --check:

  1. Fetch current specs from running services

  2. Compare against baselines in test-results/openapi/

  3. Report differences categorized as breaking vs non-breaking

  4. Exit code 0 if no breaking changes; exit code 1 if breaking changes detected

  5. Print human-readable summary to stderr

Steps

Step 1: Update service list

File: xtask/src/cmd/api_docs.rs

Update the SERVICES constant to include all 14 API services with correct ports. Verify each service’s OpenAPI endpoint is at /api-doc/openapi.json by checking the utoipa-swagger-ui setup in each service’s main.rs or canopy_api::ApiServer::router().

Step 2: Add --check flag

File: xtask/src/cmd/api_docs.rs

Add a --check CLI flag via clap. When set:

  • Do NOT overwrite baselines

  • Compare fetched specs against existing baselines in test-results/openapi/

  • Exit with code 1 if any spec differs from baseline

  • Print summary of changed services to stderr

Step 3: Breaking-change detection

File: xtask/src/cmd/api_docs.rs (add detect_breaking_changes function)

Implement:

pub enum BreakingChange {
    PathRemoved { path: String, method: String },
    RequiredFieldAdded { path: String, field: String },
    ResponseFieldRemoved { path: String, field: String },
    TypeChanged { path: String, field: String, was: String, now: String },
}

pub fn detect_breaking_changes(
    baseline: &serde_json::Value,
    current: &serde_json::Value,
) -> Vec<BreakingChange> {
    // Walk paths object, compare methods, schemas, required arrays
}

Step 4: CI job

File: .gitlab-ci.yml

openapi-contract-check:
  stage: test
  script:
    - cargo xtask api-docs --check
  allow_failure: false
  rules:
    - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'

Step 5: Commit baselines

Run cargo xtask api-docs --update with all 14 services running. Commit test-results/openapi/*.json. These become the contract baseline.

Step 6: Unit tests

File: xtask/src/cmd/api_docs.rs

#[test]
fn no_changes_produces_empty_diff() { ... }

#[test]
fn removed_path_is_breaking() { ... }

#[test]
fn added_path_is_non_breaking() { ... }

#[test]
fn added_optional_field_is_non_breaking() { ... }

#[test]
fn removed_required_response_field_is_breaking() { ... }

#[test]
fn type_change_is_breaking() { ... }

Verification

  1. cargo xtask api-docs --update fetches specs from all 14 services

  2. cargo xtask api-docs --check exits 0 when baselines match

  3. Modifying a service’s schema causes --check to exit 1 with diff summary

  4. Unit tests pass for diff detection logic

Edit this page · default