TAO Depth Anything v2
Train, evaluate, export, and deploy monocular depth models from single RGB images with NVIDIA TAO.
The documentation discloses data mounts, read-only inputs, output paths, container boundaries, and some external download locations, with no evidence of malware, credential theft, or covert exfiltration. However, it grants Read/Bash without narrow command scoping, user confirmation, sensitive-data guidance, rollback procedures, or dependency-integrity checks, so points are deducted.
The workflow, parameter matrices, troubleshooting notes, and NaN checks are detailed enough to make common configuration failures diagnosable. Static evidence is limited because there is no independently reproducible test suite or CI coverage, availability of the TAO image, tao-run-automl, and external weights is not established, and the export documentation conflicts on opset 17 versus an example using opset 16; therefore the score remains below the static ceiling of 10.
The intended audience, train/evaluate/inference/export/quantize scenarios, data formats, and hardware requirements are fairly clear, including several non-fit cases. Triggering still relies mostly on phrases, there is no Chinese-language interaction guidance or mainland-China network plan, and core workflows depend on externally hosted NVIDIA/Facebook resources with uncertain reachability, so points are deducted.
The main file plus references provide progressive disclosure for parameters, recipes, deployment, and troubleshooting, and the skill declares Apache-2.0, version 0.1.0, and an author. It lacks standard Instructions and Examples sections, the author format is incomplete, changelog and explicit maintenance/update ownership are absent, and the benchmark reports schema issues; therefore maintainability is only partial.
The skill covers dataset preparation, model selection, training, evaluation, export, deployment, and artifact checks, with templates that could be directly useful. Static review cannot verify result correctness; quantization has a documented failure path, training can yield NaN, and deployment misconfiguration can produce non-physical KPIs, so the score stays below the static ceiling.
The files provide many source-oriented configuration references, recognizable failure symptoms, and observable output checks, while BENCHMARK.md records one evaluation report. They do not provide independently verifiable committed tests, CI coverage, or multi-source reproduction, and several claims rely on referenced external implementation details, so evidence coverage is limited.
- Before execution, confirm reachability of the TAO PyT/Deploy images, tao-run-automl, and external pretrained weights, and pin versions or checksums where possible.
- Do not treat Execution status: PASS as proof of successful training; inspect finite, decreasing train_loss and verify model type, input shape, and opset across export and deployment.
- Because Bash is broadly allowed, obtain explicit user confirmation for sensitive-data mounts, output locations, and external network access.
What does this skill do, and when should you use it?
This skill guides NVIDIA TAO Toolkit workflows for monocular depth estimation with Metric Depth Anything v2 and Relative Depth Anything. It uses the unified TAO depth_net CLI for training, evaluation, inference, export, quantization, and TensorRT deployment actions. The instructions cover annotation generation, dataset-class selection, AutoML policy, distributed training, checkpoint handling, and troubleshooting. Docker, NVIDIA Container Toolkit, and an NVIDIA GPU are required.
Reads model, training, dataset, export, and inference specifications; generates line-based image/depth annotations with depth_net convert; selects dataset classes for metric depth, relative depth, or raw NYU uint16 depth; runs train, evaluate, inference, export, quantize, and TAO Deploy TensorRT actions inside containers; produces checkpoints and evaluation or inference artifacts; and verifies results through exit codes, status.json, KPI data, and training-loss inspection.
- A computer-vision engineer needs metric depth in meters from individual RGB images and wants to fine-tune or run a MetricDepthAnything model.
- A researcher works with disparity-encoded data and needs relative monocular depth estimation using RelativeDepthAnything.
- A team uses NYU sync_depth PNG files and needs the dataset class that performs millimetre-to-metre conversion and Eigen cropping.
- A model engineer needs to export a trained monocular-depth model or generate a TensorRT engine using the TAO Deploy workflow.
- An infrastructure team needs to run TAO depth training on one GPU, multiple GPUs, or multiple nodes with Lightning-managed distribution.
What are this skill's strengths and limitations?
- Covers training, evaluation, inference, export, quantization, and TensorRT deployment workflows.
- Explicitly distinguishes metric depth, relative depth, and NYU raw-depth encodings to avoid dataset mismatches.
- Includes guidance for AutoML, multi-GPU and multi-node training, checkpoint loading, and known error patterns.
- Can be installed as an individual skill through the NVIDIA skills CLI.
- Requires Docker, NVIDIA Container Toolkit, and an NVIDIA GPU; the ViT-Large encoder is memory intensive.
- Dataset paths, annotation structure, substring matching, and file extensions must be configured precisely; mistakes can produce empty masks or NaN training loss.
- The entrypoint may report PASS even when train_loss is NaN, so results require additional validation.
- The profile applies only to this monocular-depth skill, not to sibling skills in the NVIDIA/skills monorepo.
How do you install this skill?
Install the specific skill with the NVIDIA skills CLI: npx skills add nvidia/skills --skill tao-train-depth-anything-v2 --yes. To target an agent, for example Codex, use: npx skills add nvidia/skills --skill tao-train-depth-anything-v2 --agent codex. The skill becomes available the next time the agent loads skills.
How do you use this skill?
Ask the agent to “train monocular depth,” “use DepthAnything v2,” or perform “monocular depth estimation.” Prepare RGB images, optional ground-truth depth files, and a line-based annotation file accessible inside the container. Select model_type and dataset_name according to the data encoding, write the action-specific YAML specification, and run depth_net in Docker. AutoML is enabled by default for training; new launch prompts should expose automl_policy: on or off. After execution, check the container exit code, status.json KPI block, and whether per-step train_loss is finite.
How does this skill compare with similar options?
The skill supports two principal model paths: Metric Depth Anything for metric depth in meters and Relative Depth Anything for relative or disparity-encoded depth. For raw NYU uint16 millimetre depth, it provides the specialized NYUDV2 and NYUDV2Relative dataset classes.