NeMo-RL Brev Runbook
Helps NeMo-RL agents manage storage, caches, logs, and authentication safely on Brev instances.
The skill explicitly moves large artifacts to /ephemeral, forbids printing or committing .env secrets, limits cleanup scope, and documents cache and log locations, providing partial least-privilege and data-flow transparency. Eight points are deducted because it lacks explicit user confirmation before expensive runs, a complete rollback plan, and sufficient disclosure of W&B/Hugging Face external transfer and authentication side effects.
Capacity checks, missing-.env feedback, cache variables, storage paths, and logging locations are reasonably specific, and BENCHMARK reports static checks and evaluation results. Twelve points are deducted because dependencies, platform prerequisites, abnormal recovery, and graceful-stop procedures are incomplete; commands contain unresolved placeholders, and static files cannot reproduce key paths. The static cap also limits this to at most 10 points.
The trigger scope is clear for NeMo-RL training, auto-research, caches, and logs on Brev, and the positive/negative evaluation cases support basic routing boundaries. Eight points are deducted because /home/ubuntu/RL and /ephemeral are hard-coded, non-Brev alternatives are not defined, Chinese-language support is absent, and mainland-China reachability for Hugging Face, W&B, and related services is not assessed.
SKILL.md provides frontmatter, licensing, and clear operational sections; skill-card adds owner, use case, risks, version 1.5.4, and evaluation information. Seven points are deducted for missing author and tags, missing recommended Instructions and Examples sections, no changelog, no explicit maintenance responsibility or update path, and limited installation and troubleshooting guidance.
It gives directly reusable guidance for Brev disk layout, shared caches, logs, and authentication variables, while the evaluation report indicates some benefit on core tasks. Nine points are deducted because examples contain placeholders, representative complete outputs or verifiable execution records are absent, and adaptation to varying Brev configurations is limited. Static review caps this at 7 points.
The files include a dated evaluation report, evaluation tasks, metrics, results, and positive/negative trigger cases, providing limited auditability. Six points are deducted because raw execution logs, CI tests, and independent reproduction artifacts are absent; the BENCHMARK is primarily report-level evidence and does not provide cross-source corroboration. Static review caps this at 5 points.
- Confirm that the host is actually a Brev instance and replace all placeholders with verified paths before execution.
- Before enabling HF_TOKEN, WANDB_API_KEY, or other credentials, confirm external data-flow implications and network reachability.
- The skill lacks complete recovery procedures for disk exhaustion, interrupted jobs, or failed cache cleanup; cleanup targets should be confirmed individually.
- The evaluation results lack raw logs and independent reproduction artifacts and should not be treated as completed runtime verification.
What does this skill do, and when should you use it?
This skill provides operating guidance for NeMo-RL agents running on Brev, with `/home/ubuntu/RL` treated as the source checkout and `/ephemeral` as working storage. It covers auto-research campaigns, training jobs, model and dataset downloads, shared caches, Ray temporary files, checkpoints, logs, and authenticated W&B or Hugging Face workflows. It requires disk checks before expensive runs and keeps bulky generated state outside the Git checkout. It is a good fit for users running long NeMo-RL workloads on Brev who need reproducible records and controlled disk usage.
Optionally loads /home/ubuntu/RL/.env without exposing its secrets; checks capacity with df -h for /home/ubuntu/RL and /ephemeral; creates per-user, campaign, and experiment directories under /ephemeral; assigns Hugging Face, PyTorch, Triton, uv, pip, W&B, Ray, and temporary paths; directs logs, checkpoints, evaluation dumps, and other generated assets to the experiment directory; records absolute paths in the nemo-rl-auto-research TSV; rechecks disk space during long runs; and limits cleanup to clearly named directories belonging to the current campaign.
- A NeMo-RL researcher launches a nemo-rl-auto-research campaign on Brev and needs separate locations for code, experiments, logs, and caches.
- A training engineer runs a long job that may create checkpoints, Ray temporary files, profiler traces, or large logs and wants to avoid filling the checkout volume.
- A user downloads models or datasets from Hugging Face and wants a shared cache under `/ephemeral` across experiments.
- A user runs W&B or Hugging Face authenticated workflows and needs to load credentials from `.env` without exposing them.
What are this skill's strengths and limitations?
- Clearly separates the `/home/ubuntu/RL` source checkout from generated state on `/ephemeral`.
- Covers model, dataset, PyTorch, Triton, uv, pip, W&B, and Ray cache locations.
- Defines disk checks, secret-handling rules, logging practices, and cleanup boundaries.
- Records absolute artifact paths to support reproducibility.
- Assumes Brev-specific directory conventions and requires path changes elsewhere.
- Provides no test suite, automation scripts, or capacity thresholds for different instance sizes.
- Does not define NeMo-RL training recipes or concrete experiment commands.
How do you install this skill?
Install it through the default skills CLI: npx skills add nvidia/skills --skill nemo-rl-brev-etiquette --yes. The README does not document another single-skill installation method.
How do you use this skill?
On a Brev instance, trigger it before launching a NeMo-RL campaign, experiment, training job, or large download. Example: Launch a nemo-rl-auto-research campaign on Brev and configure storage, caches, logs, and authentication according to the NeMo-RL Brev runbook. The skill supplies shell patterns for checking disk space, loading .env, creating /ephemeral directories, exporting paths, and recording locations; it does not prescribe a specific training recipe or experiment command.