Harness Creator: Harness Engineering for AI Coding Agents
When your coding agent forgets context, drifts out of scope, or claims "done" before tests pass, this skill scaffolds, audits, and improves the harness that makes it reliable.
SKILL.md explicitly requires user confirmation before overwrites, minimal context collection, and forbids hidden destructive behavior; design rules show strong safety intent. However, the bundled scripts (create-harness.mjs etc.) were not included in the review material, so actual behavior and least-privilege could not be statically verified — deducted accordingly.
Instructions are self-consistent; README claims scripts use only Node built-ins and ships 10 eval cases plus a benchmark self-check. But script source is not visible, there is no CI evidence of evals running, and failure-feedback quality on abnormal input cannot be statically confirmed; capped at 10 per static calibration — deducted.
Audience and scenarios are clear (unreliable cross-session coding agents), non-fit boundaries are explicit, trigger terms are rich, and metadata declares zh support. But installation depends on npx/GitHub paths with no mainland-China reachability note, and evidence for real trigger precision is limited — deducted.
Well-layered docs (SKILL.md + references + evals + bilingual README), version number, MIT license, metadata. and a clear file layout. But maintenance ownership and update path are unclear, the publisher is unverified, and there is no changelog — deducted.
The five-subsystem model and deliverable checklist are concrete; the skill honestly states its structural benchmark does not replace real before/after agent sessions. But static review cannot confirm outputs are directly usable, and comparative-benefit evidence is only third-party experiments retold — deducted.
evals., schemas/templates, and a self-checking benchmark design are auditable primary material. But there is no third-party execution evidence, no reproducible CI run of evals, and reference claims are not linked to original sources; static cap of 5 — deducted.
- Bundled script sources were not visible in this review; manually inspect create-harness.mjs and others before use, especially --force overwrite behavior.
- The structural score and self-checking benchmark do not prove real effectiveness; run your own before/after agent sessions for important projects.
- Installation relies on npx and GitHub; users on mainland-China networks may need mirrors or manual directory copying.
- Publisher identity is unverified; there is no external endorsement of maintenance, so pin the version you adopt.
- Metadata declares Chinese support, but the skill body is in English; actual zh support is limited.
What does this skill do, and when should you use it?
harness-creator is a reusable skill bundled with the walkinglabs/learn-harness-engineering course, designed to build a reliable working environment around AI coding agents. It organizes every harness around five subsystems: instructions (AGENTS.md/CLAUDE.md), state (feature_list., progress.md), verification (init.sh and runnable commands), scope (feature dependencies and done criteria), and session lifecycle (session-handoff.md). Three bundled Node.js scripts let you scaffold a minimal harness, audit an existing one, and render a shareable HTML assessment in one command each. It fits teams whose agents are unstable across sessions, forget context, or start each session inconsistently.
The skill performs three concrete operations. First, create-harness.mjs scaffolds a minimal harness in a target project (AGENTS.md or CLAUDE.md, feature_list., progress.md, init.sh, optionally session-handoff.md). Second, validate-harness.mjs scores an existing repository against the five subsystems, reporting the lowest-scoring area and the first 2-3 changes that would improve reliability. Third, render-assessment-html.mjs and run-benchmark.mjs produce shareable HTML reports; the benchmark first runs a self-check (scaffolds and validates a throwaway harness) before scoring the target and eval coverage. Script options include --agent-file, --package-manager, --commands, and --force. Seven reference documents cover memory persistence, tool safety, context budgets, multi-agent coordination, and more, loaded on demand.
- An engineer using Claude Code or Codex on a real project finds the agent starts fresh every session and forgets prior progress; they need persistent state and handoff files.
- A maintainer of a legacy repo that already has an AGENTS.md but whose agent still overreaches or declares victory early runs the audit script to find the weakest subsystem.
- A tech lead wants to standardize agent workspaces across a team and needs a shareable HTML assessment report as a discussion baseline.
- A newcomer to harness engineering learns by running the seven progressive projects on the course's Electron knowledge-base app, comparing prompt-only vs. rules-first results.
- An intermediate user wants to move from manual prompting to automated loops (goal, timer, maker-checker), following Lecture 13 and Project 07 templates.
What are this skill's strengths and limitations?
- Theory grounded in OpenAI and Anthropic engineering write-ups, backed by a full 13-lecture / 7-project course in the same repo
- Batteries-included Node.js scripts make create, audit, and report each a one-line command
- Audit output names the lowest-scoring subsystem and concrete first fixes, not just a generic score
- Design rules explicitly require verification evidence and user approval before overwriting files
- Course content translated into 15 languages, plus a zero-dependency shell audit script (audit-harness.sh)
- The benchmark is explicitly structural only; real effectiveness requires before/after agent sessions on representative tasks, which the repo does not provide data for
- The create/audit scripts require a Node.js runtime (though tools/audit-harness.sh is a zero-dependency shell alternative)
- Course code snippets like init.sh are conceptual templates; adapting them to your stack takes real work
- SKILL.md does not declare a test suite for the scripts, and cross-platform script behavior is not verified in the source
- No evidence is shown of the skill being tested on specific platforms; real results depend on how well your agent follows instruction files
How do you install this skill?
Copy the skills/harness-creator/ folder into your agent's skills directory (e.g., the Claude Code skills folder). The course itself needs no installation to read; for local preview, clone the repo and run npm install && npm run docs:dev.
How do you use this skill?
Create a harness: node skills/harness-creator/scripts/create-harness.mjs --target /path/to/project (options: --agent-file CLAUDE.md, --package-manager npm|pnpm|yarn|bun, --commands "cmd one,cmd two", --force after confirming overwrites). Audit: node skills/harness-creator/scripts/validate-harness.mjs --target /path/to/project. Report: node skills/harness-creator/scripts/render-assessment-html.mjs --target /path/to/project, or run-benchmark.mjs --target ... --html report.html. You can also trigger the skill in natural language, e.g., "scaffold a minimal harness for this repo" — even without saying the word "harness".
How does this skill compare with similar options?
The source positions the course alongside OpenAI's and Anthropic's harness-engineering articles and the Awesome Harness Engineering list as references; sibling courses from the same team (Hands-on Modern RL, Modern LLM Notebook) cover different domains and are not substitutes.