TAO Video Action Recognition
Train, evaluate, export, and run inference for RGB, optical-flow, and multi-stream video action models with NVIDIA TAO.
The skill limits declared tools to Read and Bash and clearly names data directories, checkpoints, and output paths; no malware, credential theft, or covert exfiltration is evident. However, it requires Docker/GPU infrastructure, may download and extract datasets, defaults Weights & Biases to enabled, and lacks user-confirmation rules, network/telemetry disclosure, sensitive-video handling, permission isolation, and rollback guidance, so points are deducted.
The documentation covers train, evaluate, inference, export, AutoML, checkpoint handling, and several error patterns, with deterministic-training defaults. There are no executable tests or failure-output examples, AutoML depends on an external tao-run-automl skill, and “single-node oriented” conflicts with schemas that permit num_nodes greater than one. Edge-case and dependency-failure feedback is incomplete, so the score is conservatively limited by static calibration.
Trigger phrases, RGB/optical-flow/joint inputs, action boundaries, GPU requirements, and dataset expectations are reasonably clear. Non-fit cases, TAO/container versions, dataset-format boundaries, Chinese-language use, and mainland-China network reachability are not documented; Docker, NVIDIA Container Toolkit, external images, and possible downloads may be operational constraints, so points are deducted.
The skill includes structured schemas, templates, skill_info, an evaluation report, and a version field. It lacks dedicated Instructions, Examples, FAQs, changelog, and a clear maintenance/update path. The skill declares Apache-2.0 while the supplied metadata says NOASSERTION and repository documentation describes Apache/CC-BY dual licensing, leaving governance and licensing ambiguity, so points are deducted.
The skill provides concrete data paths, spec overrides, checkpoint selection rules, action mappings, and AutoML routing that could reduce configuration effort. Static evidence does not verify directly usable outputs; examples depend on fixed catch/smile data and an unspecified environment. The benchmark has one task, 0% discoverability, and 20% Codex correctness, so only limited effectiveness credit is justified.
A fixed revision, structured schemas, templates, skill_info, evals.json, and a benchmark report provide some auditability. There is no committed executable test suite, CI coverage, execution log, or independently reproducible artifact; the benchmark covers only one task and cannot be fully verified from static files, so the score remains low.
- wandb.enable defaults to true, so runs may send experiment metadata to an external service; confirm telemetry, network, and data-compliance requirements before processing sensitive video.
- Before execution, verify Docker, NVIDIA Container Toolkit, the TAO image, GPU memory, and dataset-download availability, especially on mainland-China networks.
- Checkpoint paths, label_map, sequence lengths, and directory paths must match training; no executable preflight validation or automatic rollback is documented.
- Clarify how Apache-2.0, CC-BY-4.0, and the NOASSERTION metadata relate to this skill's licensing.
What does this skill do, and when should you use it?
This skill guides NVIDIA TAO Toolkit workflows for temporal action classification from video clips. It supports RGB, optical-flow, and joint multi-stream inputs, covering training, evaluation, inference, and ONNX export. Training is routed through AutoML by default when the packaged schemas and templates are available, with a per-run option to disable it. It is suited to users who already have video data and a Docker-based TAO environment.
Reads the packaged TAO schemas, templates, and skill configuration to resolve training AutoML policy; constructs required training, validation, evaluation, and inference directory paths; runs TAO training, evaluation, inference, or export workflows; handles label maps, checkpoints, pretrained weights, resume training, and ONNX output paths; and explains common failures such as sequence-length mismatches and missing label maps.
- A computer-vision engineer wants to train temporal classifiers for actions such as catch and smile from RGB video.
- A model developer wants to use optical flow or joint RGB-plus-optical-flow inputs.
- An ML engineer wants AutoML to select training settings while minimizing val_loss.
- A deployment engineer needs to export a trained action-recognition checkpoint to ONNX.
- An evaluator needs to run TAO evaluation or inference against an existing action-recognition model.
What are this skill's strengths and limitations?
- Covers training, evaluation, inference, and ONNX export workflows.
- Supports RGB, optical-flow, and joint multi-stream inputs.
- Documents AutoML routing, dataset-directory requirements, checkpoint inference, and common failure modes.
- Supports Lightning-managed single-node multi-GPU training.
- Requires Docker, nvidia-container-toolkit, and an NVIDIA GPU; 16GB or more of VRAM per GPU is recommended.
- The scope is action recognition; it does not provide video collection, annotation, or archive-download tooling.
- Dataset archives must be extracted into directories rather than passed as .tar.gz paths to the action-recognition entry points.
- The provided SKILL.md does not include an independent test suite, performance benchmark, or cross-platform validation report.
How do you install this skill?
Install the specific skill with the skills CLI flow documented by the repository README:
npx skills add nvidia/skills --skill tao-train-action-recognition --yes
After installation, place the skill in the target agent's skill directory; the README does not specify a skill-specific manual-copy path. The runtime also requires Docker, nvidia-container-toolkit, and an NVIDIA GPU.
How do you use this skill?
After installation, prompt the agent with phrases such as “train action recognition” or “train an RGB + optical flow action model with TAO.” Provide extracted dataset directories and dataset.label_map for training, evaluation, and inference. AutoML is enabled by default for training when the packaged schema and template are available; request “disable AutoML” or set automl_policy: off for a direct run. If using TAO sample archives, download and extract train.tar.gz, test.tar.gz, or test/smile.tar.gz before launching the container. For export, provide a concrete training checkpoint and an ONNX output path.