What does this skill do, and when should you use it?
This is an Agent Skill covering the full arc of a deep-learning experiment — a SKILL.md hub plus on-demand reference docs and script templates, not a CLI or SDK. Its deepest layer is remote-GPU operations: dedicated profiles for AutoDL, RunPod, vast.ai, Lambda, Paperspace, the Chinese rental platforms, bare SSH, Slurm, and Kubernetes, each capturing stop-vs-destroy billing traps, spot grace windows, and mirror quirks. The VERIFY layer offers a 14-probe methodology for classifying a metric as bug, effect, or noise; DELIVER locks every shipped figure and table as a deterministic function of one immutable evidence layer. It fits researchers who can already write training code but keep getting burned by billing, dropped SSH sessions, and untrustworthy results.
- Generates a six-phase operational flow per rental profile: env/storage audit, SSH credentials, CPU smoke test before renting, detached launch, durable monitoring, verified teardown.
- Enforces a teardown Iron Law with a SHA-256 pull manifest: no release/terminate/delete until every reported number re-reads from the local copy.
- Ships an 8-file training debug layer mapping symptom → root cause → fix for OOM, multi-GPU hangs, NaN/loss spikes, throughput, resume bugs, convergence, and dataloader correctness.
- Verifies metric trustworthiness with probes for leakage, fair comparison, representation collapse, and variance, plus wandb forensics.
- Seals results into an immutable export/<run-id> capsule and generates single-source figures/tables with cross-document reconciliation.
- Handles Windows + Clash/Mihomo high-port SSH banner timeouts, fake-IP, and TUN routing interference.
- You rent an AutoDL GPU for a long training run and worry about dropped checkpoints, confusing stop-vs-destroy billing, and getting results off before teardown — you want detach + idempotent resume + a safe meter stop.
- A result looks surprisingly good and you need a systematic check for whether it's a bug, leakage, or a real effect before writing it up.
- A post-revision slide still shows a stale headline number — you need every figure generated from one evidence layer with automatic cross-document reconciliation.
- You train on RunPod/vast.ai spot instances and need preemption-aware checkpoint cadence plus byte-level verification before an irreversible destroy.
- A DDP launch hangs, losses go NaN, or VRAM OOMs — you want a symptom-indexed debug manual.
- You juggle many SSH rentals and fear a forgotten box billing 24/7 — you want durable monitoring and health patrols.
- Teams wanting managed multi-cloud price-shopping with automatic spot recovery across Western clouds — use SkyPilot; this skill does not orchestrate clouds.
- Users wanting zero-ops serverless inference — use Modal; this skill provides no inference service.
- Users who don't want to drive torchrun/accelerate/deepspeed themselves — the skill teaches how to operate them, it does not replace them.
How do you install this skill?
- Static review only, nothing executed: re-verify teardown/billing facts against current platform docs before relying on them — the author states six profiles are researched, not live-tested.
- Termination actions are irreversible: even with the Iron Law gate, personally confirm local PULL_VERIFIED. before approving any meter-stopping action.
- The skill deeply operates remote hosts and billing actions; trial on a small, short rental first. Publisher identity is unverified.
- Cross-skill references (research-artifact-hygiene, mirror-research-artifacts, etc.) are optional; without them some DELIVER guidance is explicitly legacy/non-canonical.
- Shell / CLI
- Network access
- Local filesystem
SSH access to remote GPU hostsrsync/scpuv (optional, for install validation)
Standard Agent Skill: clone the folder into your agent's skills directory and restart the agent. Keep the directory named remote-gpu-trainer.
Claude Code
git clone https://github.com/Hanyuyuan6/remote-gpu-trainer.git ~/.claude/skills/remote-gpu-trainerOpenAI Codex
git clone https://github.com/Hanyuyuan6/remote-gpu-trainer.git ~/.agents/skills/remote-gpu-trainerCursor · Trae · Gemini CLI · VS Code / Copilot · Goose · Kiro and other compatible agents
Clone into that agent's skills directory; see each agent's docs or agentskills.io for the exact location. Validate the install with the official validator:
uvx --from skills-ref agentskills validate ~/.claude/skills/remote-gpu-trainerHow do you use this skill?
Once installed, send your agent any of these to trigger it:
- I rented a 4090 on AutoDL for a long training run — set up checkpoint/resume and durable monitoring, and tell me how to confirm results are safely pulled before I shut the box down.
- My loss just went NaN — walk me through the precision-stability playbook to check my fp16/AMP setup.
- Is this 1.2% ablation win real? Run the VERIFY probes for leakage and unfair comparison.
- Training finished — take me through the pull verification checklist before destroying this RunPod instance.
No manual invocation — the skill triggers on run/debug/verify/deliver tasks. After the SKILL.md hub loads, follow the routing: for RUN, pick profiles/<platform>.md first (it binds every concrete path, billing verb, proxy rule), then walk the six-phase lifecycle, each phase ending in a runnable check; the core invariant is checkpoint-to-durable + unconditional load-latest-on-startup + idempotent resume. For VERIFY, apply the 14-probe methodology in references/verifying/ — classify bug/effect/noise before reporting. The Iron Law: before any release/terminate/destroy, the pull manifest must verify byte-for-byte and you must explicitly approve the cost action. Companion skills (figures, mirroring, orchestration) are optional; the skill works standalone.
What are this skill's strengths and limitations?
- Genuinely grounded platform detail: stop-vs-destroy billing differences, inode caps, cgroup memory, mirror traps all come with concrete commands and citations.
- VERIFY and DELIVER layers are platform-agnostic methodology that also applies to GPUs you own.
- Includes an evals regression harness and a U1–U44 gotcha catalog; money-affecting facts carry a verified-month stamp.
- MIT-licensed, standalone, with no mandatory companion skills.
- Only the AutoDL profile reflects the author's daily hands-on use; the other six rental profiles are researched from docs and community reports, not independently live-tested.
- It is knowledge and templates, not a CLI/SDK — monitoring and scheduling recipes depend on mapping your host agent's background-task and scheduler capabilities.
- Platform billing rules change; teardown/billing facts still need re-checking against current official docs.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| Remote GPU Trainer Skill this page | 65 · Recommended | ★ 42 | 12d ago | MIT |
| NeMo MBridge GPU Memory Tuning ✓ NVIDIA · Official | 61 · Recommended | ★ 3.6k | 11d ago | Apache-2.0 |
| NeMo AutoModel Recipe Development ✓ NVIDIA · Official | 55 · Use with care | ★ 3.6k | 11d ago | Apache-2.0 |
| NeMo-RL Auto Research ✓ NVIDIA · Official | 51 · Use with care | ★ 3.6k | 11d ago | Apache-2.0 |
| Megatron Bridge Activation Recompute ✓ NVIDIA · Official | 59 · Recommended | ★ 3.6k | 11d ago | Apache-2.0 |
Complementary to infrastructure orchestrators like SkyPilot, dstack, and Modal rather than competing: they move and price-shop the box; this skill then makes your code resume correctly so recovery continues the run, and continues into the VERIFY and DELIVER phases those tools don't touch. It specializes in the raw rented instances orchestrators skip — AutoDL, the Chinese platforms, cheap bare SSH — and in disk budgeting, inode caps, mirror stalls, and spot grace windows.
How did FollowSkills review this skill?
Evidence shows mature safety design: an Iron Law teardown gate (PULL_MANIFEST, SHA-256, byte-level roster match, explicit user approval) before any irreversible action, bans on silent deletes and route/DNS mutation, secrets via stdin, strict host-key checking, and an evidence-gated Paramiko fallback that fails closed. Deducted for: broad operational surface (SSH, billing verbs, cleanup scripts) and a complex single-socket Paramiko path whose safety implementation cannot be verified statically.
High internal self-consistency: clear progressive disclosure, uniform platform verb model, CI with structural reachability evals and reconcile unit tests. Deducted for: static review cannot reproduce key paths; consistency across many profiles/scripts is unexecuted; the README itself states six profiles are researched not live-tested, so failure-feedback quality is only partly inferable.
Audience and scenarios are explicit (owned/rented GPU, AutoDL, SSH, debug/verify/deliver), non-fit ranges are declared (SkyPilot/dstack/Modal territory), triggers include Chinese terms and China platforms/mirrors, fitting mainland-China users well. Deducted for: broad trigger surface (nearly any training task could match) with no evidence of semantic trigger precision in practice.
Excellent layered documentation: hub + references + profiles + scripts + examples + evals; explicit MIT license, v1.0.0 release, SECURITY.md, maintenance path (self-improvement protocol, staleness checks, verified stamps). Deducted for: no standalone changelog, a legacy/non-canonical DELIVER layer creating historical baggage and cross-skill hidden assumptions.
High methodological value: full RUN→VERIFY→DELIVER arc, a worked example, single-source delivery architecture; clear marginal value over platform docs. Deducted for: static cap; direct usability of outputs (export capsules, figures) is unverified, representative outputs not confirmed.
Repo contains a real CI workflow (structural evals, reconcile unit tests, staleness advisory), claimed RESULTS.md navigation runs, and inline-cited, date-stamped billing facts. Deducted for: tests cover structural drift and one script, not end-to-end reproduction of the skill's key paths (train, pull, teardown gate); author admits six profiles are untested.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →