What does this skill do, and when should you use it?
This skill lives at .claude/skills/smoke-test/SKILL.md in the JoshuaC215/agent-service-toolkit repository and drives scripts/smoke_test.sh. The script exercises the optional integrations that the fast unit suite and default CI don't cover — the Postgres and MongoDB checkpointers, the AG-UI endpoint, and LangFuse tracing — by spinning each dependency up in Docker, running the service on the host via uv, verifying the integration end-to-end, and tearing everything down. Its core design is a dependency-identity check: a passing API test alone doesn't prove the intended backend was used, so each target also queries the dependency directly for this run's data to rule out silent fallbacks like SQLite. The skill is equally explicit about when NOT to run, saving minutes and tokens on unrelated changes.
- Starts the real dependency in Docker per target (postgres / mongo / agui / langfuse / all) and runs the service on the host with uv
- Runs an API-level persistence check plus a direct dependency query (using a per-run unique SMOKE_THREAD_ID) to confirm data actually landed in that backend
- The langfuse target pulls LangFuse's full 6-service self-host stack (~5GB of images) and is excluded from the default run
- Refuses to start if :8080 is occupied so the health check can't pass against an unrelated process; fails fast with the service log if the service dies on startup
- Documents sandbox/cloud pitfalls for agents: starting dockerd, allowlisting cgr.dev, avoiding the pkill self-match, backgrounding the run
- Tests live in tests/smoke/ marked @pytest.mark.docker and are outside CI's test-docker job scope
- A developer who touched src/memory/*, checkpointer wiring, or a saver/store class and wants to confirm the Postgres/MongoDB checkpointers still work for real
- After bumping langgraph-checkpoint-postgres/-mongodb, psycopg, or pymongo versions, verifying the integration didn't break
- Modified src/service/agui.py or ag-ui-* dependencies and need to confirm the AG-UI streaming endpoint works
- Changed LangFuse CallbackHandler wiring or the langfuse dependency and want a quick trace-pipeline check outside full CI
- A maintainer or agent doing a periodic pre-release confidence pass across all optional dependencies (run all)
- An agent working in a Claude Code cloud sandbox that needs the skill's documented sandbox workarounds to run the script cleanly
- Pure frontend, docs, prompt, or agent-graph changes: the skill states most changes don't need it, and running it then wastes minutes and tokens
- Environments without Docker access or container pulls (e.g. restricted egress without cgr.dev allowlisted): the script needs real infrastructure in Docker
- Teams wanting exhaustive integration coverage: these are quick is-this-obviously-broken checks, not full validation
How do you install this skill?
- Shell / CLI
- Network access
- Local filesystem
DockeruvPostgreSQL/MongoDB/LangFuse images via docker compose
The skill ships with the agent-service-toolkit repository; clone it first:
git clone https://github.com/JoshuaC215/agent-service-toolkit.git
cd agent-service-toolkit
uv sync --frozen
The source documents no dedicated installer for porting skills out of .claude/skills/ to other platforms; copying the .claude/skills/smoke-test/ folder into your own skills directory is the straightforward route.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/JoshuaC215/agent-service-toolkit.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/.claude/skills/smoke-test" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Once installed, send your agent any of these to trigger it:
- I just changed the checkpointer wiring in src/memory/ — run the postgres and mongo smoke tests and interpret the output
- I bumped pymongo; verify the MongoDB checkpointer didn't silently fall back using the smoke test
- My change only touches the Streamlit UI — decide whether running smoke_test.sh is even warranted
- Do a full pre-release confidence pass on all optional dependencies, including langfuse
The skill is triggered when a change hits a covered path or when the user explicitly asks for a smoke test. The core flow is running scripts/smoke_test.sh with the narrowest target covering the change:
./scripts/smoke_test.sh # default: postgres, mongo, agui
./scripts/smoke_test.sh mongo # a single target
./scripts/smoke_test.sh postgres agui # a subset
./scripts/smoke_test.sh langfuse # heavy: ~5GB stack, run on its own
./scripts/smoke_test.sh all # everything, incl. langfuse
A green run ends with "--- All smoke tests passed ---"; any failure or unknown target exits non-zero. When interpreting results, trust the "✓ verified: N …" dependency-identity lines; a "✗ FAIL: … was NOT exercised as intended" is a real failure, not flakiness. LangFuse ingestion is async, so the script polls the traces API for a few seconds before deciding.
What are this skill's strengths and limitations?
- The dependency-identity check catches the classic trap of an API test passing while the intended backend was never touched
- Fine-grained opt-in targets let you run only what your change affected, keeping cost low
- Extensive, battle-tested sandbox guidance makes it practical for autonomous agents
- Explicit guards for port conflicts, fast startup failures, and async ingestion ambiguity
- The langfuse target pulls a ~5GB six-service stack and is heavy to run
- Hard dependency on Docker plus a host uv setup; container builds fail behind restricted egress proxies
- Smoke-level only — not exhaustive integration coverage
- Designed for the agent-service-toolkit repo's layout; reusing it elsewhere requires following its extension guidance
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| Optional-Dependency Smoke Tests (agent-service-toolkit) this page | 64 · Recommended | ★ 4.5k | 21d ago | MIT |
| Model Catalog Refresh Skill (agent-service-toolkit) | 60 · Recommended | ★ 4.5k | 21d ago | MIT |
| VS Code Smoke Test Assistant ✓ Microsoft · Official | 53 · Use with care | ★ 194k | 11d ago | MIT |
| NV-Reason-CXR Chest X-ray Reasoning Test ✓ NVIDIA · Official | 52 · Use with care | ★ 3.6k | 11d ago | Apache-2.0 |
| Jetson BSP Image Validator ✓ NVIDIA · Official | 50 · Use with care | ★ 3.6k | 11d ago | Apache-2.0 |
The source contrasts this script with the regular uv run pytest suite and CI's test-docker job: those run tests/integration/ and never exercise the optional dependencies that need real infrastructure; this skill fills that gap and is deliberately not part of CI.
How did FollowSkills review this skill?
The project's own optional-dependency smoke-test skill, explicitly framing whether to run it at all as a first decision rather than blindly executing — a cost-conscious design.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
This review was completed under FSRS 1.0. Dimension scores are not converted; it is queued for re-review under FSRS 2.0.
See the full review method →