Cosmos Reason Video QA Fine-Tuning
A Cosmos-RL playbook for FSDP-based SFT, LoRA, evaluation, and AutoML of Cosmos3-Nano video QA models.
The documentation discloses HF_TOKEN use, gated-model access, Docker environment passing, data paths, model provenance, and preflight requirements, with some user confirmation points. However, Bash is broadly allowed, there is no complete least-privilege boundary, universal launch confirmation, rollback procedure, or detailed token-lifecycle guidance, so points are deducted. No malware, covert exfiltration, credential theft, or destructive default was shown.
Training, evaluation, dataset validation, error patterns, and checkpoint handling are documented in substantial detail. Static calibration caps the score at 10, and the materials contain important inconsistencies, including model_max_length values of 40960 versus 81920, llava versus daft format claims, and references to schemas, skill_info, scripts, and platform skills not supplied here, so points are deducted.
Trigger phrases, target model, train/evaluate/inference/quantize scenarios, and some non-fit boundaries are reasonably clear, with hardware, Docker, and dataset requirements stated. Chinese-language interaction and localization are not documented, and the core workflow depends on Hugging Face, Docker, S3, and high-end NVIDIA GPUs without evidence of mainland-China reachability, so points are deducted.
The material uses progressive disclosure and covers credentials, data contracts, parameters, error patterns, AutoML, and checkpoint handling; it also states Apache-2.0 licensing, version 0.1.0, and NVIDIA authorship. However, standard Instructions and Examples sections are missing, the author format is incomplete, hierarchy and file naming are inconsistent, and no clear changelog, maintenance owner, or update path is provided, so points are deducted.
The skill covers Cosmos3-Nano video-QA SFT/LoRA, evaluation, AutoML, dataset preflight, and troubleshooting, so it could produce directly useful configuration and workflow guidance. Under static calibration the score cannot exceed 7; the benchmark contains only one positive task, reports weak Codex correctness, discoverability, and efficiency, and provides no representative verified training outputs, so points are deducted.
The revision-pinned repository context, one evaluation task, expected behavior, and NVSkills-Eval report provide limited audit evidence. Coverage is only one task with no negative-trigger cases, the report does not independently reproduce key training paths, and no committed test suite or CI coverage is supplied, so points are deducted.
- Different reference files conflict on model_max_length, dataset formats, and template behavior; unify the authoritative configuration and add consistency checks before publication.
- Training depends on gated Hugging Face access, HF_TOKEN, Docker/NVIDIA Container Toolkit, S3 or high-end GPUs; reachability and resource compatibility should be validated separately for mainland-China users and target platforms.
- The documentation recommends downloads and training but does not define a unified confirmation flow for irreversible actions, token least privilege, or rollback after failure.
- The benchmark covers only one positive task, with weak Codex correctness, discoverability, and efficiency results, so it is not comprehensive execution validation.
What does this skill do, and when should you use it?
This skill guides training and evaluation of Cosmos3-Nano and compatible Cosmos Reason video question-answering models with Cosmos-RL. It covers dataset preflight, nested spec construction, FSDP parallelism, SFT, LoRA evaluation, checkpoint conversion, and AutoML/HPO. The packaged default base model is nvidia/Cosmos3-Nano from Hugging Face, which may require HF_TOKEN access. Docker, nvidia-container-toolkit, and suitable NVIDIA GPUs are required.
Loads packaged action templates, schemas, and reference guidance; applies user overrides to nested YAML specifications; validates annotation JSON and visible media paths; runs Cosmos-RL training, evaluation, inference, or quantization; routes eligible training runs through AutoML; converts native Cosmos3 checkpoints to a Qwen3-VL Hugging Face safetensors directory with the packaged Python helper when required by the selected image; and supports checkpoint, result, and metric handling.
- A team fine-tuning Cosmos3-Nano for video question answering on one or more NVIDIA GPU nodes.
- A researcher applying SFT or LoRA to a Cosmos Reason video QA model and evaluating the result.
- An engineer launching training or evaluation against llava or daft VLM datasets stored locally, on S3, or on a mounted filesystem.
- A practitioner searching training hyperparameters with the skill's AutoML/HPO workflow.
- An operator whose Cosmos-RL image cannot load the native Cosmos3 checkpoint format and needs the upstream conversion path.
What are this skill's strengths and limitations?
- Covers launch intake, preflight, training, evaluation, inference, quantization, LoRA evaluation, AutoML, and checkpoint conversion.
- Documents the nonstandard Cosmos-RL FSDP parameters dp_shard_size and dp_replicate_size for GPU and node sizing.
- Includes concrete requirements for dataset formats, annotation validation, media visibility, model paths, frame handling, and common failure patterns.
- Can be installed as a single skill through the NVIDIA skills CLI.
- Requires Docker, nvidia-container-toolkit, and appropriate NVIDIA hardware; the guidance recommends 8×80GB A100 or H100 GPUs for the 8B-parameter models.
- Gated Hugging Face models require an accepted model agreement and a read-capable HF_TOKEN.
- Some images require a separate Cosmos3-to-Qwen3-VL checkpoint conversion before training or evaluation.
- The supplied material does not provide standalone benchmark results or independent platform-validation evidence for this skill.
How do you install this skill?
Install the individual skill with the repository's documented skills CLI flow:
npx skills add nvidia/skills --skill tao-finetune-cosmos-reason --yes
The supplied README does not document a separate manual installation procedure for this skill.
How do you use this skill?
After installation, prompt an agent with a supported request such as "fine-tune Cosmos3 Nano on my video QA dataset", "Run Cosmos-RL SFT for nvidia/Cosmos3-Nano", or "Evaluate my Cosmos3-Nano video QA checkpoint". Provide the Docker/GPU environment, dataset locations, and HF_TOKEN when gated Hugging Face access is needed. Training defaults to AutoML when the packaged train schema and template are available; explicitly requesting disabled AutoML or plain training applies an off override for that run.