Plan: Containerized Integration Tests
On this page
Status
| Step | Description | Status |
|---|---|---|
1 |
Author ADR-015 ( |
Done (2026-04-27) — ADR-015 authored. Renumbered from ADR-012 in plan (ADR-012 was already taken by |
2 |
URL-centralization close-out (A1–A3): export |
Done (2026-04-27) — |
3 |
Add |
Done (2026-04-27) — Single-stage Alpine ( |
4 |
Add |
Done (2026-04-27) — |
5 |
Wire |
Done (2026-04-27, partial — validate flip deferred) — |
6 |
Add |
Done (2026-04-27) — |
7 |
Documentation pass: |
Done (2026-04-27, scoped) — CHANGELOG.adoc Unreleased Added entry covers ADR-015 + new compose service + xtask flip + URL centralization + CI job + the validate-flip deviation. nav.adoc gains the ADR-015 link. |
Branch: feature/containerized-integration-tests
ADR: adr-015-containerized-integration-tests.adoc (authored in Step 1)
Labels: type::feature, priority::medium, program::infrastructure, service::devstack, workflow::ready
Context
Canopy’s Rust integration tests currently execute on the host, reaching devstack services through localhost:<mapped-port> (ephemeral since the port-allocation work in ephemeral-port-allocation.adoc).
The host-side model has three concrete operational gaps that surfaced during a fresh Linux Docker Engine setup:
-
Three
canopy-web::session_testcases followed a 303 tohost.docker.internal:8180(Keycloak) and failed withfailed to lookup address information: Name or service not known.host.docker.internalis injected into the host’s resolver by Docker Desktop on Mac/Windows but not by Docker Engine on Linux — so tests that resolve the redirect target on the host pass on Desktop and fail on Engine. The immediate workaround (TestClient::new_no_redirect()+assert_status(303)) shipped infeat: auto-handle first-run devstack friction in xtask(MR !65), but the parity fix subsumes it because the redirect target does resolve inside the docker network. -
canopy_test_lib::infrastructure.rs,canopy-db::pg_url, andcanopy-mq::amqp_urlreadCANOPY_PORT_POSTGRES_5432/CANOPY_PORT_RABBITMQ_5672for the port but hardcodelocalhostas the hostname. Inside the docker network the hostname ispostgres/rabbitmq, so these four spots need a URL-level env var (not a port-level one) to let the container override. -
Running the suite at all requires
cargo+cargo-nexteston every developer machine and every CI runner image, with host-side resolver quirks (systemd-resolved synthetic records, IPv6 preferences) affecting reproducibility.
Pattern precedent: the canopy-e2e Playwright container at docker-compose.yml:769-781 runs in-network via docker compose --profile e2e run --rm canopy-e2e.
This plan applies the same shape to the Rust integration suite.
Scope
In scope
-
New
Dockerfile.integration(multi-stage Alpine, non-rootappuser, pre-built nextest). -
New
canopy-integrationcompose service withprofiles: [integration]and explicitdepends_onfor every service the suite probes. -
New env vars in
write_ports_envso host-side test runs pick up a fullCANOPY_TEST__DATABASE_URL. -
Refactor
canopy-db::pg_url()andcanopy-mq::amqp_url()to read full URL env vars (localhost fallback preserved). -
Refactor
infrastructure_available()to parse the host from the DB URL (localhost default). -
xtask/src/cmd/{test,validate}.rs— default integration path runs in-container;--hostflag preserves the existing host-side path. -
.gitlab-ci.yml— newintegration-testsjob. -
ADR-015 + this plan file + documentation updates.
Out of scope
-
Dioxus applicant portal integration tests (portal is stub-only; revisit post-UAT per ADR-008).
-
Unit test runner (stays local, no devstack dependency).
-
Playwright E2E — already containerized; no changes required.
-
New unit-level integration tests using
testcontainers-rsper the convention; existing canopy suite is full-stack HTTP tests so docker-compose remains the right path (documented in ADR-015). -
Migration of the
new_no_redirectworkaround out of session_test; it still works under both runners and a future cleanup can remove it once the container path is canonical.
Design
A. URL-centralization close-out
Upstream’s TestConfig::from_env() (at crates/canopy-test-lib/src/config.rs) already centralizes all 20 service URLs via CANOPY_TEST__<NAME>_URL env vars with localhost fallback, and auto-loads .ports.env from the workspace root.
Three spots still bypass this abstraction with raw CANOPY_PORT_* port vars + hardcoded localhost:
A1. xtask/src/docker.rs::write_ports_env writes CANOPY_TESTKEYCLOAK_URL and CANOPY_TESTRABBITMQ_URL but not CANOPY_TEST__DATABASE_URL.
Adjacent to the existing RabbitMQ line, emit:
let pg_port = get_host_port(ports, "postgres", 5432);
writeln!(file, "CANOPY_TEST__DATABASE_URL=postgres://canopy:canopy@localhost:{pg_port}/canopy")?;
A2. Rewrite pg_url() / amqp_url() to read full URL env vars.
-
crates/canopy-db/tests/db_test.rs::pg_url: readCANOPY_TEST__DATABASE_URLwith the currentpostgres://canopy:canopy@localhost:{CANOPY_PORT_POSTGRES_5432}/canopyas fallback. -
crates/canopy-mq/tests/mq_test.rs::amqp_url: readCANOPY_TEST__RABBITMQ_URLwith the currentamqp://canopy:canopy@localhost:{CANOPY_PORT_RABBITMQ_5672}/%2fas fallback.
Host runs are unchanged — the env vars are already in .ports.env — and the container will override both to in-network URLs.
A3. Make infrastructure_available() network-aware.
crates/canopy-test-lib/src/infrastructure.rs currently does TcpStream::connect"127.0.0.1", pg_port.
Parse the host out of CANOPY_TEST__DATABASE_URL (or default to 127.0.0.1).
Inside the container this becomes postgres:5432; on host it stays localhost:<ephemeral>.
B. Compose service
Mirror the canopy-e2e shape.
The environment: block explicitly overrides every URL TestConfig::from_env() reads with an in-network address — TestConfig’s fallback chain (env var → `.ports.env → hardcoded localhost) is irrelevant inside the container because the env vars are always set.
canopy-integration:
profiles: [integration]
build:
context: .
dockerfile: Dockerfile.integration
user: app
environment:
CANOPY_TEST__KEYCLOAK_URL: http://keycloak:8080
CANOPY_TEST__RULES_URL: http://canopy-rules:8001
CANOPY_TEST__PERSONS_URL: http://canopy-persons:8002
CANOPY_TEST__APPLICATIONS_URL: http://canopy-applications:8003
CANOPY_TEST__ELIGIBILITY_URL: http://canopy-eligibility:8004
CANOPY_TEST__VERIFICATION_URL: http://canopy-verification:8005
CANOPY_TEST__ENROLLMENT_URL: http://canopy-enrollment:8006
CANOPY_TEST__RENEWALS_URL: http://canopy-renewals:8007
CANOPY_TEST__NOTICES_URL: http://canopy-notices:8008
CANOPY_TEST__EXCHANGE_URL: http://canopy-exchange:8009
CANOPY_TEST__APPEALS_URL: http://canopy-appeals:8010
CANOPY_TEST__REPORTING_URL: http://canopy-reporting:8011
CANOPY_TEST__SECURITY_URL: http://canopy-security:8012
CANOPY_TEST__SNAP_URL: http://canopy-snap:8013
CANOPY_TEST__TANF_URL: http://canopy-tanf:8014
CANOPY_TEST__MEDICAID_URL: http://canopy-medicaid:8015
CANOPY_TEST__CAPS_URL: http://canopy-caps:8016
CANOPY_TEST__WIC_URL: http://canopy-wic:8017
CANOPY_TEST__WEB_URL: http://canopy-web:8080
CANOPY_TEST__PORTAL_URL: http://canopy-portal:8090
CANOPY_TEST__DATABASE_URL: postgres://canopy:canopy@postgres:5432/canopy
CANOPY_TEST__RABBITMQ_URL: amqp://canopy:canopy@rabbitmq:5672/%2f
CANOPY_CI: ${CANOPY_CI:-}
volumes:
- ./test-results:/app/test-results
- ./.keys:/app/.keys:ro
- canopy-integration-target:/app/target
depends_on:
canopy-web: { condition: service_healthy }
canopy-rules: { condition: service_healthy }
canopy-persons: { condition: service_healthy }
canopy-applications: { condition: service_healthy }
canopy-eligibility: { condition: service_healthy }
canopy-verification: { condition: service_healthy }
canopy-enrollment: { condition: service_healthy }
canopy-renewals: { condition: service_healthy }
canopy-notices: { condition: service_healthy }
canopy-appeals: { condition: service_healthy }
canopy-security: { condition: service_healthy }
canopy-snap: { condition: service_healthy }
keycloak: { condition: service_healthy }
postgres: { condition: service_healthy }
rabbitmq: { condition: service_healthy }
garage: { condition: service_started }
volumes:
canopy-integration-target:
C. Dockerfile.integration
Multi-stage Alpine per .claude/docs/coding-conventions.md Container Runtime section.
Non-root app user.
Source COPY`ed (not bind-mounted) to match the existing `Dockerfile pattern.
# SPDX-License-Identifier: AGPL-3.0-or-later
# Multi-stage Alpine build for the Rust integration test runner.
# -- Stage 1: build --
FROM rust:1.94-alpine AS builder
RUN apk add --no-cache musl-dev curl
# Pre-built nextest (avoids ~3-min `cargo install`).
RUN curl -LsSf https://get.nexte.st/latest/linux-musl | tar -xzf - -C /usr/local/bin
WORKDIR /app
COPY . .
# Pre-compile test binaries so first `nextest run` inside the container is
# execution-only. Cached by docker layer.
RUN cargo nextest list --workspace --test '*' --profile integration
# -- Stage 2: runtime --
FROM rust:1.94-alpine
RUN apk add --no-cache musl-dev curl
RUN curl -LsSf https://get.nexte.st/latest/linux-musl | tar -xzf - -C /usr/local/bin
RUN addgroup -S app && adduser -S -G app -s /sbin/nologin app
WORKDIR /app
COPY --from=builder --chown=app:app /app /app
USER app
ENTRYPOINT ["cargo", "nextest", "run", "--workspace", "--test", "*", "--profile", "integration"]
Create or extend .dockerignore with:
target/
.git/
node_modules/
test-results/
.devstack/
.ports.env
D. xtask wiring
xtask/src/cmd/test.rs and xtask/src/cmd/validate.rs route integration tests through the compose service by default.
A new --host flag preserves the existing host-side path for IDE iteration.
// In test::Args
#[arg(long)]
pub host: bool,
// In test::run(), integration branch:
if args.host {
run_integration_tests_host()?; // the existing nextest call
} else {
run_integration_tests_container(&project)?;
}
fn run_integration_tests_container(project: &str) -> Result<()> {
crate::docker::compose_cmd(project, &[
"--profile", "integration",
"--profile", "snap-only", // transitively starts canopy-web and its deps
"run", "--build", "--rm",
"canopy-integration",
])
}
In validate.rs step [10/11] cargo nextest run …: replace with the same router.
Keep the step label stable.
E. Nextest profile selection
.config/nextest.toml defines integration (local) and ci-integration (adds terminate-after = 2 on 120s slow timeout).
The container ENTRYPOINT hardcodes --profile integration.
For CI, the canopy-integration service is invoked with a compose command override that swaps the profile — wire this in the CI job by appending --profile ci-integration to the compose run arguments when CANOPY_CI=true.
The xtask router should do the same switch so cargo xtask test running under CI uses ci-integration without manual override.
F. CI — .gitlab-ci.yml
Add integration-tests under the test stage.
Per .claude/docs/coding-conventions.md CI/CD Runners — explicit tags:, dhs-aws-autoscaler-docker.xlarge (multi-service docker-compose is xlarge territory).
integration-tests:
stage: test
tags: [dhs-aws-autoscaler-docker.xlarge]
image: $CI_REGISTRY/gadhs/standard/ci-images/rust-docker:1.94
variables:
CANOPY_CI: "true"
DOCKER_HOST: tcp://docker:2375
DOCKER_TLS_CERTDIR: ""
services:
- docker:dind
script:
- cargo xtask test --integration
artifacts:
when: always
reports:
junit: test-results/integration/results.xml
paths:
- test-results/integration/
expire_in: 1 week
G. Filesystem-bound tests
-
canopy-typst (
crates/canopy-typst/tests/render_test.rs): reads fonts from system dirs. Addapk add --no-cache font-nototoDockerfile.integration. Verify font availability in a smoke-test rendering after build. -
canopy-seed (
tools/canopy-seed/tests/integration.rs):tempfile::tempdir()— works in container, no change.
Steps
Step 1: ADR-015
Author docs/modules/ROOT/pages/adrs/adr-015-containerized-integration-tests.adoc.
Standard ADR format (Status / Context / Decision / Consequences).
Cite ADR-001 (program service isolation) and ADR-005 (modular deployment profiles).
Document the testcontainers-rs vs docker-compose reconciliation explicitly: canopy’s integration suite is full-stack HTTP tests against running canopy services, not unit-level DB/broker tests, so docker-compose remains the correct path; new unit-level integration tests against DB/broker should still use testcontainers-rs.
Step 2: URL-centralization close-out
Implement Design section A (A1, A2, A3).
Verify host-side tests still pass after this step alone with cargo xtask test --integration (no container changes yet).
Step 3: Dockerfile.integration + .dockerignore
Implement Design section C.
docker build -f Dockerfile.integration . should produce an image tagged canopy:integration without errors; verify with docker run --rm canopy:integration --help (nextest prints help and exits 0).
Step 4: canopy-integration compose service
Implement Design section B.
docker compose --profile integration config should validate without errors.
docker compose --profile integration --profile snap-only build canopy-integration should succeed.
Step 5: xtask wiring
Implement Design sections D and E.
cargo xtask test --integration (no --host) should route through the container; cargo xtask test --integration --host should run nextest on the host.
cargo xtask validate step [10/11] should route through the container.
Files Touched
New files
-
Dockerfile.integration -
.dockerignore(if not already present) -
docs/modules/ROOT/pages/adrs/adr-015-containerized-integration-tests.adoc -
(this plan file)
Modified files
-
xtask/src/docker.rs— addCANOPY_TEST__DATABASE_URLtowrite_ports_env -
crates/canopy-db/tests/db_test.rs— readCANOPY_TEST__DATABASE_URLwith localhost fallback -
crates/canopy-mq/tests/mq_test.rs— readCANOPY_TEST__RABBITMQ_URLwith localhost fallback -
crates/canopy-test-lib/src/infrastructure.rs— parse host fromCANOPY_TEST__DATABASE_URL -
docker-compose.yml— newcanopy-integrationservice +canopy-integration-targetnamed volume -
xtask/src/cmd/test.rs—--hostflag, route default to container,CANOPY_CIprofile swap -
xtask/src/cmd/validate.rs— route step[10/11]through container -
.gitlab-ci.yml— newintegration-testsjob -
.claude/docs/testing.md— document container-first runs +--hostescape hatch -
.claude/docs/local-dev.md— commands list; host-side nextest now optional -
.claude/CLAUDE.md— Tech Stack note about container-first integration runs -
docs/modules/ROOT/pages/developer-guide.adoc— commands + ADR-015 reference -
CHANGELOG.adoc— entry under== Unreleased→=== Added
Verification
-
Cold start, container path.
rm -rf .devstack/ .ports.env && cargo xtask validate. Expect:ensure_readycold-starts devstack with ephemeral ports →canopy-integrationbuilds and runs → all 877 tests pass →test-results/integration/results.xmlwritten → exit 0. -
Host path.
cargo xtask test --integration --host. Expect: nextest runs on host, hits loopback ephemeral ports viaTestConfig::from_env()fallback, all tests pass. -
Parity test. Locally revert the
new_no_redirectpatch inservices/canopy-web/tests/session_test.rs. Runcargo xtask test --integration. The three session tests pass under the container runner. Restore the patch. -
No-leakage grep.
grep -rn 'localhost:[0-9]' services//tests crates//tests tools/*/testsreturns zero matches — every URL flows through an env var. -
Image cache reuse. Run
cargo xtask test --integrationtwice. Second run hitscanopy-integration-targetvolume and completes nextest in <30s for unchanged code. -
CI green. Push branch, watch the
integration-testsjob on the MR pipeline. -
IDE iteration loop. From VSCode, "Run Test" on a single integration test runs on the host via loopback fallback. No docker invocation.
Acceptance Criteria
-
cargo xtask validateon a fresh clone (no.devstack/) cold-starts the devstack, builds the integration image, runs the full 877-test nextest integration suite insidecanopy-integration, writes JUnit XML totest-results/integration/results.xml. Exit 0. -
cargo xtask test --integration --hostruns the same 877-test suite on the host against loopback ephemeral ports. Exit 0. -
With the container runner selected, reverting the
new_no_redirectpatch locally keeps the threecanopy-web::session_testcases passing — the redirect target resolves inside the docker network. Restore the patch (don’t ship the revert). -
grep -rn 'localhost:[0-9]' services//tests crates//tests tools/*/testsreturns zero matches. -
CI
integration-testsjob is green on the MR pipeline. -
Pre-push hook (
cargo xtask validate) passes locally. -
cargo xtask check-docsis clean.
Documentation Updates
Per .claude/docs/delivery-protocol.md Documentation Update Checklist:
-
.claude/docs/services.md— N/A (no new endpoints or events). -
.claude/CLAUDE.md— one-line note under "Tech Stack" or "Conventions" about container-first integration runs. Since this is a single-MR plan, the status-table update per.claude/docs/git-workflow.mdMulti-MR Plans lands in this MR. -
CHANGELOG.adoc— entry under== Unreleased→=== Added, bullet format* feature-title: short description.Include: container runner,--hostescape hatch,CANOPY_TEST__DATABASE_URLexport,db_test.rs/mq_test.rsrefactor to URL-env-vars. -
.claude/docs/testing.md— "Container Runner" subsection under "Integration Tests"; document the--hostescape hatch. -
.claude/docs/local-dev.md— commands list; host-side nextest now optional for the default flow. -
docs/modules/ROOT/pages/developer-guide.adoc— commands + ADR-015 reference. -
docs/modules/ROOT/pages/adrs/adr-015-containerized-integration-tests.adoc— the ADR itself.