Agent Service Toolkit Optional-Dependency Smoke Test Skill
Run one command to verify locally that the Postgres/MongoDB checkpointers, AG-UI endpoint, and LangFuse tracing integrations actually work, without waiting for a full CI run.
The project's own optional-dependency smoke-test skill, explicitly framing whether to run it at all as a first decision rather than blindly executing — a cost-conscious design.
What does this skill do, and when should you use it?
This skill wraps scripts/smoke_test.sh from the agent-service-toolkit repository, covering optional-dependency integrations that unit tests and default CI don't exercise because they need real infrastructure. It starts each dependency in Docker (Postgres, MongoDB, or LangFuse's full self-host stack), runs the service on the host against it, verifies the integration end-to-end, then tears everything down. Its core value is guarding against the 'green but ineffective' trap: beyond an API-level check, each target independently queries the dependency itself to confirm this run's data actually landed in the intended backend rather than silently falling back to SQLite. It also tells you explicitly when to run a target and when to skip it, avoiding needless container startup cost.
The skill instructs the agent to run ./scripts/smoke_test.sh and interpret its results. By target (postgres, mongo, agui, langfuse, all), the script starts the dependency in Docker, runs the service on the host via uv with the right env, makes API-level requests, then uses a unique per-run thread id (SMOKE_THREAD_ID) to query the dependency directly (checkpoint rows, trace counts) to prove the real backend was exercised. Built-in guards include refusing to start when port 8080 is occupied, failing fast with the service log on startup death, and polling LangFuse's async traces API. The SKILL.md also documents sandbox workarounds: starting dockerd, allowlisting cgr.dev, using fuser -k 8080/tcp instead of pkill, and running the script in the background with log polling.
- A developer who changed checkpointer code under src/memory/ or initialize_database and needs to confirm Postgres/MongoDB persistence still works.
- A maintainer who bumped langgraph-checkpoint-postgres, pymongo, or psycopg versions and wants to verify the real integration wasn't broken.
- A contributor who touched src/service/agui.py or the AG-UI adapter and needs end-to-end confirmation the AG-UI streaming endpoint works.
- An engineer adjusting LangFuse CallbackHandler wiring or the /health auth check who must confirm traces actually reach LangFuse.
- A maintainer wanting a deliberate full pre-release confidence pass via ./scripts/smoke_test.sh all.
- An agent working in a Claude Code cloud sandbox that must decide whether a change warrants a smoke test at all, and how to navigate restricted-egress pitfalls.
What are this skill's strengths and limitations?
- Verifies dependency identity, not just API responses — catches silent env drops and SQLite fallbacks.
- Targeted design runs only the affected path, keeping single-target cost low.
- SKILL.md includes an unusually thorough sandbox/cloud troubleshooting guide (dockerd startup, registry allowlist, the pkill self-match trap, etc.).
- A clear should-I-run decision table avoids wasting time and tokens.
- Requires Docker and real infrastructure; the langfuse target pulls roughly 5GB of images.
- A run takes minutes, so it's unsuitable for frequent triggering.
- It's specific to the agent-service-toolkit codebase; porting to other repos means rewriting targets per its extension guide.
- The host-run-service/Docker-dependencies architecture exists because sandboxed builds can't reach package registries — a plain containerized workflow can't copy it verbatim.
How do you install this skill?
The skill lives at .claude/skills/smoke-test/ in the agent-service-toolkit repository. Clone the repo and the skill is used in-repo (it pairs with scripts/smoke_test.sh in the same project): git clone https://github.com/JoshuaC215/agent-service-toolkit.git, then uv sync --frozen. To reuse it elsewhere, copy the smoke-test skill folder into your project's .claude/skills/ directory (cross-repo installation steps beyond this are not documented in the source).
How do you use this skill?
From the repo root: first decide whether your change touches the checkpointer/memory layer, service startup/health, the AG-UI adapter, or LangFuse tracing — most changes (agent graphs, prompts, docs, client-only work) should skip it. When warranted: ./scripts/smoke_test.sh (default: postgres, mongo, agui); a single target like ./scripts/smoke_test.sh mongo; a subset like ./scripts/smoke_test.sh postgres agui; ./scripts/smoke_test.sh langfuse (~5GB heavy stack, run alone); ./scripts/smoke_test.sh all. A green run ends with '--- All smoke tests passed ---', but you should still confirm each '✓ verified: N …' dependency-identity line; a '✗ FAIL' means the dependency was never actually hit and should be treated as a real failure.