TAO VCN Classification Gap Analysis
Find the weakest VCN classification samples and turn them into augmentation targets.
The skill documents inputs, mount paths, container outputs, and several failure conditions, and declares Apache-2.0. However, it requires --gpus all, --ipc=host, broad workspace mounting, external image pulling, and a subsequent chown operation without an explicit user-confirmation step or sufficient disclosure of sensitive image/log data flow, image integrity verification, or reversible recovery; points are deducted accordingly.
SKILL.md specifies input columns, threshold logic, common failures, and an unreachable-KPI feedback path. However, it contains inconsistencies between results_dir and output_dir, kpi_gaps.parquet and gaps.parquet, and invocation forms; static review also cannot verify the container's key paths, so the score is capped at 6.
The audience, input directories, output artifacts, use cases, and non-fit ranges for more complex analysis are relatively clear. However, the skill requires a CUDA GPU, Docker, NVIDIA Container Toolkit, and a runtime image, does not document Chinese-language interaction support, and does not address reachability of overseas registries, so points are deducted.
Front matter provides name, description, license, author, version, and dependencies, while the body uses layered Inputs/Setup/Method/Outputs/Pitfalls sections. It lacks explicit maintenance ownership, update path, changelog, and standard FAQ/Examples sections; the benchmark also reports schema and author-format findings, and referenced files are absent from the supplied material.
The computational goal, threshold selection, per-label top-K ranking, and parquet output are clearly defined, and the benchmark reports correctness and effectiveness results for one evaluation task. However, only one task is covered, results require manual visual review, and documentation/hook filename mismatches may cause artifact checks to fail; static evidence supports only 6 points.
The material supplies a pinned revision, input columns, output schema, metric artifacts, and hook validation logic, with limited benchmark records. It lacks multi-task coverage or a complete independently reproducible CI/test suite, and the hooks conflict with the main documentation on filenames; therefore only 4 points are awarded.
- Execution grants the container GPU access, host IPC access, and broad workspace read/write mounts; require user confirmation and narrow the mounted paths.
- The main document specifies kpi_gaps.parquet while several hooks check gaps.parquet; results_dir and output_dir are also inconsistent and should be reconciled before publication.
- The skill depends on a runtime NVIDIA image, Docker, a GPU, and potentially network-restricted registries; no equivalent fallback is documented for GPU-less or image-unreachable environments.
- Add the referenced files, maintenance ownership, update path, changelog, Chinese usage guidance, and retention/redaction rules for sensitive images and session logs.
What does this skill do, and when should you use it?
This skill analyzes NVIDIA TAO VCN Classify experiment results around the PASS/NO_PASS decision boundary. It runs threshold selection and weakness ranking inside the TAO data-services container, then produces a per-label Parquet queue expanded by lighting condition. It also specifies a small visual review of representative weak samples to distinguish mislabeled data, edge cases, data-quality problems, and systematic issues. It is a good fit for teams that already have VCN inference outputs, training configuration, dataset files, and an NVIDIA GPU environment.
Reads inference/inference.csv from an experiment directory, the VCN training YAML, and the dataset image root; consumes input_path, object_name, label, and siamese_score columns; invokes the tao_toolkit.data_services image declared in versions.yaml through docker run with gap_analysis vcn_aoi; sweeps candidate thresholds and selects the best F1 threshold meeting min_recall; computes signed weakness for every row; keeps top_k_per_label weakest samples for each ground-truth label; expands each sample by the lighting map and image extension in the training YAML; writes kpi_gaps.parquet, threshold.txt, metrics.json, and weak_samples_breakdown.txt; writes unreachable_kpi.txt when the recall target cannot be reached; otherwise reviews five weakest PASS and five weakest NO_PASS samples and copies them into rca_images/.
- A VCN team has completed classification inference and needs a ranked queue for SDA augmentation.
- A quality engineer is auditing PASS/NO_PASS boundary cases and model misclassifications.
- A data team needs the weakest examples grouped by ground-truth label in Parquet format.
- A visual-inspection team wants a small, structured review to classify weak samples as mislabeled, edge case, data quality, or systematic.
What are this skill's strengths and limitations?
- Uses a compact workflow centered on one TAO data-services container invocation plus a small visual spot check.
- Ranks weak samples per ground-truth label and expands them by lighting condition for downstream augmentation.
- Produces threshold, confusion-matrix, metric, and per-label weakness artifacts.
- Documents important operational failure modes involving GPU checks, mounts, the -e specification, and top_k_per_label.
- Requires Docker, nvidia-container-toolkit, a CUDA GPU, and the specified TAO container image.
- Targets NVIDIA TAO VCN Classify binary inference results and does not cover broader model-audit dimensions.
- Requires local access to inference CSV data, training configuration, and image files; processing may stop at the container stage when the KPI is unreachable.
- The source provides no independent test results or platform-coverage evidence for this individual skill.
- Repository metadata is listed as NOASSERTION, while the README states Apache 2.0 plus CC BY 4.0 for the repository; the individual SKILL.md declares Apache-2.0.
How do you install this skill?
Install the skill through the CLI command documented by the repository README: npx skills add nvidia/skills --skill tao-analyze-gaps-visual-changenet --yes. The README does not document additional client-specific installation steps for this skill.
How do you use this skill?
After installation, ask an enabled agent to perform a request such as “Analyze the gaps in this NVIDIA TAO VCN Classify experiment and produce a per-label ranked augmentation queue.” Before execution, provide an experiment result directory, training YAML, and dataset directory, and ensure Docker, nvidia-container-toolkit, a CUDA GPU, and the TAO data-services image from versions.yaml are available. Always provide a positive integer for top_k_per_label. Results are written under a timestamped rca_results/ directory beneath the experiment directory. The exact container arguments, image-resolution command, and visual-review procedure are defined in SKILL.md.