Mask Grounding DINO Training & Deployment
Train and deploy text-guided open-vocabulary instance segmentation models.
The skill declares only Read and Bash, documents Docker/NVIDIA Container Toolkit requirements, exposes data paths, S3 examples, checkpoint handling, and encryption_key mappings, and directs TensorRT work to a dedicated reference. Deductions apply because it does not specify user confirmation, tightly scoped Bash use, sensitive-data handling, external network/W&B data flows, rollback, or recovery; the templates also enable wandb by default. No malware, credential theft, or other red-line behavior is shown.
SKILL.md, skill_info.yaml, and the spec templates are broadly consistent across train, evaluate, inference, export, quantize, and deployment mappings, with guidance for CUDA OOM, deployment fields, and model-shape errors. Deductions apply because static files do not provide executable coverage of key paths, dependency/container availability is underspecified, and abnormal-input handling, recovery, and diagnosable failure feedback remain limited; the single benchmark task cannot justify exceeding the static ceiling of 10.
The name, description, trigger phrases, supported actions, data formats, hardware requirements, and AutoML boundaries define a plausible TAO Mask Grounding DINO audience. Deductions apply because non-fit scenarios, complete input/output contracts, and exclusion triggers are not well specified; Chinese-language interaction and mainland-China network reachability are undocumented, while Docker, model-weight, and possible external-service compatibility lack evidence.
The material includes metadata, per-action data requirements, parameters, templates, deployment references, error patterns, and a version number; repository files provide licensing, synchronization, and update context. Deductions apply for missing recommended Instructions/Examples sections, incomplete author formatting, governance ambiguity between the skill's Apache-2.0 declaration and the repository's CC-BY-4.0/Apache-2.0 documentation/source licensing, and no clear changelog, named maintainer, or skill-level update path. The benchmark also records folder-depth and extra-schema warnings.
The skill covers training, evaluation, inference, export, quantization, and TensorRT deployment, with directly reusable spec_overrides, data-source fields, and important parameters, so the core value proposition is clear. Deductions apply because no real model outputs, end-to-end results, or comparative benefit over manual/alternative workflows are verified in the static material; the benchmark has only one plan-oriented task and reports limited Codex correctness and effectiveness.
The pinned source set includes SKILL.md, generated-schema metadata, configuration templates, an evaluation task, and a benchmark report, giving key claims some auditability. Deductions apply because there is only one evaluation task, no committed test suite or real CI evidence covering the skill's key paths, and the benchmark lacks reproducible execution details; static evidence therefore supports only a modest score.
- The default templates enable Weights & Biases; confirm whether sending run metadata or code to an external service is permitted.
- Training and deployment incur GPU, container, model-weight, and possibly cloud-storage access costs; rollback, cleanup, and failure-recovery steps are not documented.
- Before use, verify the actual repository layout, presence and parseability of generated schemas/templates, and availability of Docker, NVIDIA Container Toolkit, the TAO container, and model weights in the target network environment.
- Clarify the relationship between the skill's Apache-2.0 metadata and the repository's statement that documentation is CC-BY-4.0 before publication or redistribution.
What does this skill do, and when should you use it?
This skill guides NVIDIA TAO Toolkit workflows for Mask Grounding DINO, a Grounding DINO extension with a mask-prediction head. It covers training, evaluation, inference, export, quantization, and TensorRT deployment. Training is AutoML-enabled by default when the packaged schemas and templates are available, while non-training actions remain in the model skill. Docker, nvidia-container-toolkit, and an NVIDIA GPU are required; 24GB or more of VRAM per GPU is recommended.
Reads model metadata, training policy, action schemas, specification templates, and TensorRT deployment references; constructs mandatory train, validation, evaluation, inference, and quantization data-source paths; runs or routes TAO Mask Grounding DINO train, evaluate, inference, export, and quantize actions; sends eligible training runs through tao-run-automl; and applies SDK parameter mappings for parent checkpoints, output directories, ONNX files, and TensorRT engine files.
- A computer-vision engineer training text-prompted instance segmentation on custom ODVG or COCO data.
- A model developer evaluating a trained Mask Grounding DINO checkpoint against COCO-format validation annotations.
- A researcher running open-vocabulary segmentation with prompts such as “person” or “bicycle.”
- An engineer exporting, quantizing, or generating a TensorRT engine for GPU deployment.
What are this skill's strengths and limitations?
- Covers the model workflow from training through export, quantization, and TensorRT deployment.
- Supports open-vocabulary, text-prompted instance segmentation.
- Includes action schemas, specification templates, parameter mappings, and error-pattern guidance.
- Supports AutoML with val_loss as the minimization metric.
- Requires Docker, nvidia-container-toolkit, and an NVIDIA GPU; 24GB or more of VRAM per GPU is recommended.
- The mask head adds memory overhead over Grounding DINO, so batch size may need to be reduced.
- Every action requires action-specific data-source configuration and prepared ODVG, COCO, or archive files.
- The supplied material does not document an independent test suite or concrete performance benchmarks for this skill.
How do you install this skill?
Use the README-supported skills CLI command: npx skills add nvidia/skills --skill tao-train-mask-grounding-dino --yes. The CLI handles the installation destination; cloning the repository or copying folders manually is not required. The runtime also requires Docker and nvidia-container-toolkit.
How do you use this skill?
After installation, ask a compatible Agent Skills client to “train Mask Grounding DINO” or to “run text-prompted instance segmentation inference.” Supply the required action-specific data sources. For training, use automl_policy: on or off to control AutoML for that run. For TensorRT engine generation, TensorRT evaluation, or TensorRT inference, follow references/tao-deploy-mask-grounding-dino.md.
How does this skill compare with similar options?
The skill explicitly extends Grounding DINO and reuses parts of its training setup. Mask Grounding DINO adds a mask-prediction head, making it suitable for instance segmentation rather than a detection-only Grounding DINO workflow.