TAO Deformable DETR Object Detection
Guides end-to-end training, evaluation, export, quantization, and deployment of lightweight Deformable DETR detectors in NVIDIA TAO.
The skill declares only Read and Bash and documents containers, data sources, checkpoint handling, and encryption-key mappings. However, it does not require user confirmation, adequately disclose data flows involving S3, WandB, Docker/containers, and model data, or provide rollback and sensitive-data handling rules. No red-line risk is evident, but permission isolation and external-effect controls are incomplete, so points are deducted.
The documentation covers actions, parameters, architecture consistency, deterministic training settings, and common errors. However, the supplied material does not include the referenced schemas contents, skill_info.yaml and the documentation disagree about export inputs, templates contain unresolved ??? values, and no skill-specific tests cover key paths. The static cap is 10, so the score is limited to 8.
The audience, trigger phrases, training/evaluation/export/quantization/inference scenarios, and TensorRT non-fit boundary are fairly clear, with GPU, Docker, and multi-node requirements stated. It does not define non-target scenarios, Chinese-language support, or mainland-China reachability, and S3, WandB, and container acquisition may constrain environments, so points are deducted.
Front matter provides the name, Apache-2.0 license, version, and author, while the README identifies NVIDIA provenance, daily synchronization, and an update path. However, dedicated Instructions, Examples, FAQs, changelog, and complete maintenance ownership are absent; the author format fails the reported schema check; and the repository's NOASSERTION license metadata leaves uncertainty despite the skill declaration. Points are deducted.
The skill gives directly useful guidance for dataset fields, checkpoint propagation, AutoML routing, export, and deployment, making the core task plausible. However, dependency and template completeness are not sufficiently demonstrated, and the benchmark has only one task with limited Codex correctness and effectiveness results. The static cap is 7, so the score is 5.
The skill_info.yaml, multiple configuration templates, evaluation task, and committed benchmark report provide limited traceability. However, there is only one positive task, no negative-trigger coverage, no dedicated test suite for this skill, and no corroborating independent reproduction evidence. The static cap is 5, so the score is 4.
- Before training, AutoML, export, or deployment, confirm Docker/NVIDIA container availability, GPUs, data paths, WandB/S3 access, and credential scope; avoid sending sensitive data to logs or external services.
- Verify that schemas/<action>.schema.json, templates, and skill_info.yaml are actually packaged and consistent; do not use templates containing unresolved ??? values.
- Preserve architecture-affecting parameters and exact checkpoint paths across train, evaluate, inference, quantize, export, and deployment to avoid shape mismatches or invalid evaluation.
- The benchmark covers only one positive planning task and does not establish real training or deployment success rates or mainland-China network usability.
What does this skill do, and when should you use it?
This skill supports NVIDIA TAO Toolkit workflows for 2D object detection with Deformable DETR and deformable attention for multi-scale features. It covers training, evaluation, inference, export, and quantization, including TensorRT engine generation through TAO Deploy. The instructions define dataset formats, mandatory data-source overrides, checkpoint handling, architecture-sensitive parameters, distributed training, and failure patterns. It is best suited to teams already operating Docker, nvidia-container-toolkit, and NVIDIA GPU infrastructure.
Reads the skill's TAO metadata, packaged action schemas, and deployment templates; guides the agent in constructing spec_overrides for COCO or COCO raw training, evaluation, export, inference, and quantization; routes eligible training runs through AutoML; resolves exact checkpoint artifacts; exports models to ONNX; generates FP32, FP16, or INT8 TensorRT engines through TAO Deploy; and applies metric-selection, multi-GPU, and troubleshooting guidance.
- A computer-vision engineer training Deformable DETR on a custom COCO dataset uses the skill to configure classes, data sources, checkpoints, and training parameters.
- A model developer evaluating a checkpoint uses the skill to select AP50 or COCO-style mAP metrics and provide the required evaluation annotations.
- A deployment engineer exporting a trained detector to ONNX and TensorRT uses the skill to select the TAO Deploy container and deployment templates.
- A researcher running hyperparameter optimization, or requesting plain training with AutoML disabled for one run, uses the skill to apply the correct per-run policy.
- A GPU platform engineer configuring Lightning-managed DDP or FSDP across one or more nodes uses the skill to align GPU settings and distributed environment variables.
What are this skill's strengths and limitations?
- Covers the workflow from Deformable DETR training through TensorRT deployment.
- Specifies action-specific dataset files, configuration keys, and mandatory data-source overrides.
- Includes guidance for AutoML routing, exact checkpoint selection, architecture propagation, distributed training, and common errors.
- The skill identifies version 0.1.0, NVIDIA Corporation as author, and Apache-2.0 as its skill license.
- Requires Docker, nvidia-container-toolkit, an NVIDIA GPU, and at least one GPU; four GPUs with 16GB or more VRAM each are recommended.
- Scope is limited to object_detection with coco and coco_raw formats; broader task support is not evidenced.
- The PyT model container cannot natively run gen_trt_engine; TensorRT generation requires the TAO Deploy container.
- The supplied material provides no standalone test suite, numeric performance benchmarks, or specific TAO/container version requirements.
How do you install this skill?
Install the individual skill with the command documented in the repository README:
npx skills add nvidia/skills --skill tao-train-deformable-detr --yes
The source does not document a separate source-build installation path. The runtime also requires Docker, nvidia-container-toolkit, and an NVIDIA GPU.
How do you use this skill?
After installation, ask the agent for a concrete workflow such as "train deformable-detr on my COCO dataset", "evaluate my TAO Deformable DETR checkpoint", or "export and quantize a Deformable DETR model". Supply the action-specific data-source paths required by the skill. Training defaults to AutoML, while a run can request AutoML off. For TensorRT generation, export the model first, then use the TAO Deploy container and deployment templates.
How does this skill compare with similar options?
The skill explicitly compares Deformable DETR with DINO: Deformable DETR uses a smaller feed-forward dimension and is described as lighter, but it supports a more limited backbone set. Its documented defaults include dropout_ratio 0.3 and dim_feedforward 1024, compared with the cited DINO values of 0.0 and 2048. The source does not provide a broader product comparison.