TAO BEVFusion Training
Train, evaluate, and run inference on BEV 3D detection models that fuse LiDAR and camera data with TAO.
The skill limits declared tools to Read and Bash and specifies a fixed TAO 5.5 container; no malware, credential theft, or covert exfiltration is evident. However, Docker, GPUs, datasets, checkpoints, AutoML, and possible network access are involved without explicit user confirmation, least-privilege mounts, sensitive-data guidance, rollback, or data-flow disclosure, so points are deducted.
The documentation covers container selection, conversion, distributed launch, checkpoint handoff, and SIGSEGV diagnosis, with useful failure guidance. However, there is no executable test suite or CI evidence, the template defaults to one GPU while the text states two GPUs minimum, and key runtime and AutoML prerequisites are not statically reproducible, so points are deducted.
The audience, trigger phrases, actions, and KITTI/BEVFusion use case are reasonably clear. Non-fit boundaries, input/output contracts, and negative trigger conditions are incomplete; Chinese-language support is absent, and mainland-China reachability of the nvcr.io container dependency is not addressed, so points are deducted.
The skill includes SKILL.md, references, templates, schemas, an evaluation report, and version metadata; the skill declares Apache-2.0 and NVIDIA ownership is traceable. However, recommended Instructions/Examples/FAQ sections are missing, the author format is nonstandard, no clear changelog or maintenance/update path is provided, and repository license metadata is NOASSERTION, so points are deducted.
Commands, data paths, parameters, templates, and common-error guidance could support conversion, training, evaluation, and inference. There is no verified representative output beyond static materials; the benchmark has one plan-only task, with 20% Codex correctness and 48% effectiveness, so substantial environment and dataset preparation remains and points are deducted.
The pinned revision, schemas, templates, skill_info, evaluation task, and benchmark report provide limited auditability. The benchmark has one task and one attempt, with no real CI or committed tests covering key paths; static review cannot establish that the commands, container, and templates run, so points are deducted.
- Before execution, confirm Docker, NVIDIA Container Toolkit, GPU count/VRAM, nvcr.io reachability, and image access permissions.
- Do not treat template defaults as validated configuration; specifically reconcile the one-GPU defaults with the stated two-GPU minimum and verify dataset paths, checkpoints, and AutoML schemas.
- Treat camera images, LiDAR, annotations, and checkpoints as potentially sensitive; confirm mounts, upload exclusions, and logs do not expose data.
- Benchmark evidence is very limited and reports weak Codex correctness and effectiveness; add real CI, key-action tests, and failure-path tests before broad deployment.
What does this skill do, and when should you use it?
This skill supports NVIDIA TAO Toolkit workflows for BEVFusion multi-sensor 3D object detection. It guides an agent through bird's-eye-view fusion of LiDAR point clouds and camera images, including dataset conversion, training, evaluation, and inference. It requires Docker, NVIDIA Container Toolkit, and the dedicated TAO 5.5.0 BEVFusion container. Training supports AutoML routing, multi-GPU, and multi-node configurations, but requires substantial GPU resources and a specific dataset layout.
Reads skill metadata, training schemas, and packaged specification templates, then constructs action-specific data paths and spec_overrides; runs bevfusion convert, training, evaluation, and inference in nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt; dataset conversion produces KITTI-style information pickle files and training/velodyne_reduced; training uses torchrun and supports checkpoint handling, AutoML, GPU, and multi-node settings.
- An autonomous-driving perception developer needs to train a BEVFusion 3D detector on KITTI-style LiDAR and camera data.
- A TAO user needs to convert a dataset before launching multi-GPU BEVFusion training.
- A model engineer needs to evaluate or run inference from an exact upstream training checkpoint.
- A platform engineer needs to run BEVFusion training across multiple Dockerized GPU nodes.
What are this skill's strengths and limitations?
- Covers the core BEVFusion workflow from dataset conversion through training, evaluation, and inference.
- Documents the required TAO 5.5 container, directory names, and checkpoint handoff behavior.
- Supports AutoML, multi-GPU, and multi-node training configurations.
- Includes troubleshooting guidance for SIGSEGV outcomes, missing modalities, and container dependency errors.
- Requires at least 2 GPUs; 4 GPUs and 24GB or more VRAM per GPU are recommended, with A100 strongly recommended.
- Depends on the dedicated TAO Toolkit 5.5.0 BEVFusion container and `mmdet3d`.
- Requires a specific KITTI-style directory structure and explicit data-source overrides for every action.
- The supplied source does not include concrete benchmark metrics or tested-platform coverage for this individual skill.
How do you install this skill?
Install the NVIDIA skills collection with npx skills add nvidia/skills. To install this skill by name, use npx skills add nvidia/skills --skill tao-train-bevfusion --yes. The README does not document a separate manual installation procedure for only this directory; the skill is loaded by the agent on a subsequent skills load.
How do you use this skill?
After installation, ask the agent something such as “train BEVFusion” or “run LiDAR + camera fusion 3D detection inference.” Provide a KITTI-style dataset root and run dataset_convert first, followed by training, evaluation, or inference as needed. AutoML is enabled by default for training when its packaged schema and template are runnable; specify automl_policy: off to use direct model training for that run. Use the BEVFusion 5.5 container rather than the shared TAO PyTorch 7.0 RC image, which lacks mmdet3d.
How does this skill compare with similar options?
Compared with the shared TAO PyTorch 7.0 RC image, this skill requires the dedicated TAO Toolkit 5.5.0 BEVFusion container because the shared image does not package mmdet3d. For training, it also defaults to the tao-run-automl route when the packaged training schema and template are available; direct model training is used when AutoML is disabled or unavailable.