Dev & Engineering ai-benchmarkeconomic-simulationgdpvalnanobot-integrationcost-trackingmulti-agent-evaluation

ClawWork: AI Economic Survival Benchmark & Coworker

Turns an AI assistant into an economically accountable coworker that pays for its own tokens and must earn money completing real professional tasks to survive.

FollowSkills review · FSRS-2.0
Not recommended
43/ 100 5-point scale 2.2 / 5
1 2 3 4 5 6
1Trust12 / 25 · 2.4/5

The SKILL.md is a pure instruction protocol with no direct credential or exfiltration behavior, but it is force-injected via always:true and instructs the agent to use web search and code sandboxes without disclosing data flows, confirmation gates, or rollback; deducted for missing least-privilege scoping and external-effect disclosure.

2Reliability8 / 20 · 2.0/5

Instructions are internally consistent (workflow, thresholds, iteration caps), but the skill fully depends on external nanobot/clawwork tools with no described tests, error handling, or failure feedback; deducted because key paths are not reproducible statically and abnormal-input behavior is unknown.

3Adaptability7 / 15 · 2.3/5

Scenario is clear (economic survival loop in a benchmark), but non-fit boundaries are undeclared, and core function depends on overseas services (OpenAI, E2B, Tavily) with no mainland-China reachability guidance; deducted for thin boundary and environment-fit evidence.

4Convention8 / 15 · 2.7/5

Repository has MIT license, README, troubleshooting, and roadmap, but the skill file itself lacks versioning/changelog, its description is somewhat marketing-flavored, and maintenance ownership is not stated at the skill level; deducted for incomplete versioning and skill-level governance.

5Effectiveness5 / 15 · 1.7/5

As a benchmark simulation prompt it plausibly guides the agent loop, but end-user value depends on running the full ClawWork system; the $19K/$15K earning claims are not verifiable from static files, deducted for limited evidence of direct usability and marginal benefit.

6Verifiability3 / 10 · 1.5/5

README offers a leaderboard and GDPVal references plus a Pages deploy workflow, but none cover the skill's key paths with reproducible tests; earning figures lack auditable primary material, deducted for single-source, non-reproducible evidence.

Evidence confidence:Low Reviewed Sep 09, 2026 Reviewed revision 9c73ac05fdb0
Before you use it
  • Static assessment only; no code was executed. All scores derive from source reading with low confidence.
  • The skill is injected via always:true and instructs the agent to perform web searches and sandboxed code execution; understand cost and external-call implications before use.
  • Earning figures in the README ($15K/$19K) are self-reported and not independently verifiable from static material.
  • Core function depends on overseas services (OpenAI, E2B, Tavily); mainland-China reachability is not addressed.
  • The skill file has no version or changelog; updates may change agent behavior without notice.
  • Publisher is not verified by the FollowSkills curated registry; identity is treated as unknown.
Review evidence [1][2][3][4][5]
See the full review method →

What does this skill do, and when should you use it?

ClawWork is an MIT-licensed open-source economic survival benchmark from HKUDS. AI agents start with just $10, pay real-time token costs on every API call, and earn $82–$5,000 per completed task drawn from OpenAI's GDPVal dataset — 220 real professional tasks across 44 occupations — with payment scaled by quality score and BLS wage estimates. The system ships with a React dashboard, an economic tracker, and a multi-model leaderboard where top agents reach roughly $1,500+/hr equivalent earnings. It runs standalone as a simulation or plugs into an existing Nanobot gateway via ClawMode, converting a chatbot into a cost-aware AI coworker.

In each daily cycle the agent calls decide_activity to choose 'work' or 'learn'. When working, it reads a GDPVal task and may use web search, a code sandbox (E2B by default), file creation (txt/xlsx/docx/pdf and more), and video generation tools to produce deliverables, then submits via submit_work for LLM evaluation against category-specific rubrics and payment; learning saves knowledge to persistent memory via learn. A TrackedProvider intercepts every LLM call and deducts token costs; the economic tracker logs balance, income, costs, and survival tier (Thriving/Stable/Struggling/Bankrupt), a FastAPI + React dashboard streams metrics over WebSocket, and accounting is written to local files like token_costs.l.

  1. AI researchers benchmarking different models (GLM, Kimi, Qwen, Gemini, etc.) on real work quality, cost efficiency, and long-term economic survival
  2. Engineering teams adding economic accountability to an existing Nanobot gateway spanning nine channels (Telegram, Discord, Slack, etc.) so every conversation carries a cost
  3. Educators or demo builders wanting to visualize how AI makes work-vs-learn career decisions under budget pressure
  4. Benchmark developers extending the system with new task sources, evaluation rubrics, or LLM providers using the documented extension points

What are this skill's strengths and limitations?

Pros
  • Grounded in real GDPVal tasks with BLS-wage-based pricing tied to quality scores — evaluation dimensions mirror production environments
  • Genuine cost accounting: token costs are read from API responses rather than estimated, and economics are transparently logged
  • Lightweight architecture — one pip install plus a config file deploys an economically accountable agent, with clearly documented extension points
  • Ships with a React dashboard and a public multi-model leaderboard for strong visualization
Limitations
  • Requires your own OpenAI API key and real API spend; the default E2B code sandbox is an external dependency with rate limits
  • Agents must submit by iteration 10–12 of 15 or time out, and quality scores below 0.6 earn nothing — failure costs fall on the operator
  • No evidence in the docs of an automated test suite; stability must be verified yourself
  • Headline earning figures ($15K/11 hours, $19K/8 hours) come from the project's own leaderboard and lack independent verification

How do you install this skill?

Clone and enter: git clone https://github.com/HKUDS/ClawWork.git && cd ClawWork. Create a Python 3.10+ environment (conda or venv) and run pip install -r requirements.txt. For the dashboard, cd frontend && npm install. Copy cp .env.example .env and fill in OPENAI_API_KEY (required); optionally set E2B_API_KEY, WEB_SEARCH_API_KEY, and provider variables. Full ClawMode integration steps are in clawmode_integration/README.md (not reproduced on this page).

How do you use this skill?

Standalone mode: run ./start_dashboard.sh in one terminal to launch the FastAPI backend and React frontend, and ./run_test_agent.sh in another to start the agent; watch live balance and task completions at http://localhost:3000. Agent configuration lives in JSON files under livebench/configs/, where you set initial balance, token pricing, date ranges, and multiple agents. For ClawMode, run python -m clawmode_integration.cli agent|gateway to add economic tools and the /clawwork command to an existing Nanobot instance; note balance tracking only works through the ClawMode gateway — direct nanobot agent runs bypass the tracker.

How does this skill compare with similar options?

The source positions ClawWork as an alternative to traditional technical benchmarks, which measure only technical metrics; ClawWork additionally measures work quality, cost efficiency, and long-term economic sustainability, built on OpenAI's GDPVal dataset. No other named competitors appear in the source.

FAQ

How much does it cost to run?
You supply an OpenAI API key and pay for all LLM calls; per-task costs are logged in token_costs.l. Web search is also billed (Tavily ~$0.0008/call, Jina ~$0.05/1M tokens). Payment is quality-gated — scores below 0.6 pay nothing.
How is this different from a plain chatbot?
With ClawMode integration, every reply carries a cost footer and deducts from the agent's balance; the agent must earn more from real professional tasks than it spends to keep operating.
What are the known failure modes?
Documented issues include a stale dashboard (hard refresh needed), port conflicts on 8000/3000, E2B sandbox 429 rate limits, and ClawMode balances not decreasing — the latter happens when running nanobot agent directly, bypassing the economic tracker.
Which models and integrations are supported?
Standalone mode accepts any base model (e.g., GPT-4o, Claude Sonnet 4.5) via LangChain/LiteLLM; ClawMode connects to Nanobot's nine channels including Telegram, Discord, Slack, WhatsApp, Email, Feishu, and DingTalk.

Related skills