TAO MAL Weakly Supervised Segmentation
Train segmentation models from point or box annotations and produce masks with minimal labeling.
The skill limits declared tools to Read/Bash and clearly identifies Docker, NVIDIA Container Toolkit, GPUs, data sources, and checkpoint mappings. However, Bash, containers, data upload, and default-enabled Weights & Biases can create external effects; user confirmation, sensitive-data handling, telemetry disclosure, and rollback are not specified. NVIDIA provenance supports attribution only, not safety quality, so points are deducted.
SKILL.md, skill_info.yaml, templates, and schemas consistently describe train/evaluate/inference flows, data paths, checkpoint handling, and OOM mitigation, with deterministic CUDA defaults documented. There are no execution logs, key-path test coverage, or detailed abnormal-input diagnostics; the static calibration therefore requires a conservative deduction.
The name, trigger phrases, workflows, COCO requirements, GPU/VRAM requirements, and CSV incompatibility boundary are reasonably clear. Chinese-language support, mainland-China reachability, and broader non-fit boundaries are not documented, while Docker, NVIDIA GPUs, and potentially W&B are environmental dependencies, so points are deducted.
The skill includes an Apache-2.0 declaration, version 0.1.0, author metadata, schemas, templates, and error patterns. It lacks dedicated Instructions, Examples, FAQs, changelog, and explicit maintenance/update ownership; the supplied benchmark also reports missing recommended sections and incomplete author formatting, so points are deducted.
The goal, input format, spec overrides, train/evaluate/inference outputs, and checkpoint-selection rules are sufficiently concrete to guide the core workflow. Directly usable representative outputs are not verified, success depends on compatible data, pretrained weights, and substantial GPU resources, and the benchmark contains only one task; static review therefore supports only a conservative score.
Schema metadata, a committed evaluation task, a benchmark report, and static-validation findings provide limited auditability. There is only one positive task, no negative coverage, and no committed test suite or CI covering key paths; the benchmark cannot be independently reproduced from this static review, so points are deducted.
- W&B is enabled by default, but the documentation does not specify what data is uploaded or how telemetry is disabled; confirm network and telemetry policy before processing sensitive data.
- COCO-style annotations.json, matching image file_name values, pretrained weights, and at least 24GB GPU memory are required; CSV or generic archive datasets are not directly compatible.
- AutoML routing depends on packaged schemas/templates and the external tao-skill-bank capability; missing artifacts trigger direct-training fallback, which was not executed or verified here.
- Failure diagnostics, examples, FAQs, changelog, and mainland-China network or Chinese-language support are not clearly documented.
What does this skill do, and when should you use it?
This skill supports NVIDIA TAO Toolkit’s MAL (Mask Auto-Label) model for weakly supervised segmentation from minimal point or box annotations. It uses a ViT-MAE backbone and covers training, evaluation, and inference, with mIoU as the monitoring metric. AutoML is enabled by default for training but can be disabled per run. Docker, NVIDIA Container Toolkit, and at least one GPU are required.
Reads the model’s packaged configuration, training schemas, and templates; builds required data-source overrides; and routes training, evaluation, or inference through the relevant TAO workflow. It requires image archives named images.tar.gz and COCO-style annotations.json files whose file_name entries match the prepared image paths. Training can use the TAO AutoML route or direct model training; evaluation and inference use a requested or resolved checkpoint and can produce the MAL inference JSON output.
- A computer-vision team training weakly supervised segmentation from point or box annotations.
- A TAO user fine-tuning a MAL model with a ViT-MAE backbone.
- An engineer training on COCO-style segmentation data while monitoring mIoU.
- A team evaluating or running inference from a selected training or AutoML checkpoint.
- A user operating MAL training across multiple GPUs or nodes with Lightning-managed distribution.
What are this skill's strengths and limitations?
- Covers MAL training, evaluation, and inference workflows.
- Defines concrete COCO annotation, data-path, and checkpoint-resolution requirements.
- Supports AutoML, single-GPU, multi-GPU, and multi-node training.
- Includes guidance for CUDA out-of-memory errors and crop_size misconfiguration.
- Requires Docker, NVIDIA Container Toolkit, and an NVIDIA GPU; 24GB or more of VRAM per GPU is recommended.
- The documented input format requires compressed image data plus COCO-style JSON; unconverted datasets cannot be used directly.
- The supplied sources provide no standalone test suite, sample dataset, or measured performance results.
- The repository metadata supplied here says NOASSERTION, while the skill frontmatter says Apache-2.0 and the README describes the repository as dual-licensed Apache 2.0 and CC BY 4.0.
How do you install this skill?
Install the skill from NVIDIA’s collection with the skills CLI: npx skills add nvidia/skills --skill tao-train-mask-auto-label --yes. The README says the CLI handles the installation destination; no more specific local path is documented.
How do you use this skill?
After the skill is loaded by an Agent, use a trigger such as “train MAL,” “Mask Auto-Label,” or “weakly-supervised segmentation.” Provide images.tar.gz and annotations.json for the requested train, evaluation, or inference data source, and include the corresponding paths in spec_overrides. Set train.pretrained_model_path for ViT-MAE weights. To disable AutoML for one run, explicitly request “disable AutoML” or “plain training.”