NeMo AutoModel Recipe Development
Build, modify, and validate NeMo AutoModel training and evaluation recipes.
The skill mainly provides configuration and development guidance, requests no credentials, network access, or elevated permissions, and explicitly warns against exposing secrets in prompts, logs, or output, so no red-line risk is evident. Points are deducted because user confirmation, rollback, data-flow disclosure, dependency security, and execution-side-effect controls are not specified; attribution is supported by NVIDIA metadata and repository governance material, but the license metadata is NOASSERTION.
The document provides reasonably clear routing boundaries, execution flow, configuration examples, and common pitfalls; BENCHMARK.md reports accuracy results for three tasks. Points are deducted because this review is static, key paths were not reproduced, no skill-specific test suite is supplied, and abnormal-input, version-compatibility, and failure-feedback behavior is underspecified; static calibration limits the score to 10 or less.
The audience, use cases, trigger conditions, and non-fit range are clearly described for recipe construction, training, evaluation, YAML, builders, and CLI routing. Points are deducted because input/output boundaries, negative-trigger examples, and environment prerequisites are not systematically defined, and Chinese-language support is not documented. Core functionality does not entirely depend on an overseas online service, so no additional mainland-China reachability deduction is applied.
The documentation includes front matter, layered sections, examples, routing boundaries, dependency guidance, known pitfalls, benchmark material, and a skill card; the skill declares Apache-2.0 and the repository supplies license text. Points are deducted because versioning, changelog, maintenance ownership, and update path are incomplete; the benchmark reports an author-format issue, and repository license metadata is NOASSERTION.
The skill gives actionable paths, configuration snippets, and commands for new recipes, _target_ usage, validation, and checkpointing; BENCHMARK.md reports 95% Codex correctness and 93% Codex effectiveness across three positive tasks. Points are deducted because coverage is only three positive tasks with no negative cases, and no independently verifiable representative outputs or real project results are included; static calibration limits the score to 7 or less.
Revision-pinned source material, evaluation tasks, expected behaviors, and a benchmark report provide some auditability. Points are deducted because evaluation coverage is narrow, no independently rerunnable test suite for this skill is included, and the benchmark results cannot be verified during this static review; static calibration limits the score to 5 or less.
- Before use, verify the current NeMo AutoModel version, actual module paths, and CLI behavior; examples may drift from upstream.
- Do not treat the guidance as a validated training configuration; confirm data, model, checkpoint paths, resource costs, and overwrite risks before execution.
- Add negative-trigger evaluations, rerunnable skill-specific tests, a compatibility matrix, named maintenance ownership, and a changelog.
What does this skill do, and when should you use it?
This skill supports development of NeMo AutoModel training, fine-tuning, and evaluation recipes. It explains YAML structure, `_target_` callables, component builders, CLI routing, and execution flow. Coverage includes concrete recipe guidance for LLMs, VLMs, diffusion, and retrieval. It also specifies key settings for validation, checkpointing, resume, and local testing.
Guides an agent to locate recipe files and YAML sections under nemo_automodel/recipes/, identify builders for the model, optimizer, dataloaders, loss, schedulers, and checkpointing, and explain how _target_ invokes a fully qualified Python callable with keyword arguments. It provides minimal YAML and CLI override examples, including automodel finetune llm -c <config.yaml>, and helps check validation cadence, checkpoint cadence, restore paths, batch-size arithmetic, CLI route registration, and dataset-collator/model-shape mismatches.
- A NeMo AutoModel developer is adding an LLM fine-tuning or pretraining recipe variant and needs the relevant file, builders, CLI route, example YAML, and test path.
- A training engineer is adding YAML fields for an optimizer, dataset, or scheduler and needs to understand `_target_` and nested CLI overrides.
- A model engineer is configuring validation data, validation intervals, checkpoint intervals, or resume-from-checkpoint behavior.
- A troubleshooter is investigating a first-step crash, forward shape mismatch, validation OOM, or checkpoint restore failure within a recipe.
- A test author needs a small CPU-compatible unit test for a new recipe.
What are this skill's strengths and limitations?
- Covers recipe YAML, builders, CLI routes, execution flow, training-loop details, validation, and checkpoint configuration.
- Includes concrete configuration structures, commands, execution ordering, and common failure modes.
- Defines a clear boundary around recipe construction and execution-flow questions.
- It is focused on recipe development and does not cover standalone cluster launchers or distributed-strategy selection.
- The provided material does not enumerate this skill's own test files, platform matrix, or version compatibility range.
- Examples depend on the corresponding NeMo AutoModel repository and `automodel` environment being available.
How do you install this skill?
Install the specific skill with the command documented in the repository README: npx skills add nvidia/skills --skill nemo-automodel-recipe-development --yes. Add --agent codex to target Codex. The provided source does not specify other installation methods or version requirements.
How do you use this skill?
After installation, ask the agent about a recipe task, for example: Create a new NeMo AutoModel LLM fine-tuning recipe variant and identify the YAML, builder functions, CLI route, and CPU validation command. A recipe can be launched with automodel finetune llm -c config.yaml, with nested overrides such as --optimizer.lr and --step_scheduler.max_steps.