VSS Multi-Camera 3D Detection and Tracking
Deploy multi-camera 3D perception with DeepStream and BEV Fusion.
The skill clearly separates MV3DT from the full warehouse blueprint, requires confirmation before host-permission changes, destructive recovery, or volume resets, and recommends UID-scoped ACLs instead of 777 or chown. It also documents credential and mount flows. Points are deducted because deployment, image downloads, .env writes, and credential reads have external or host-side effects without a complete pre-deployment confirmation, secret-minimization, and rollback protocol.
Routing, prerequisites, calibration chaining, camera-count synchronization, health checks, failure classification, and evidence capture before recovery are detailed, with intelligible diagnostic commands. Static review cannot establish actual execution, and the supplied files do not provide complete reproducible key-path execution evidence, so the score is capped at the static limit of 10.
The skill covers the sample dataset, local videos, and RTSP, explains minimal versus extended profiles, GPU stream caps, single-camera non-fit, and routing to related skills. Points are deducted because Chinese-language support is not addressed and reachability of NGC, HuggingFace, STUN, and WebRTC dependencies from mainland China is not adequately assessed; some core paths depend on these external services.
The information architecture is strong, with routing questions, progressive reference documents, prerequisites, examples, troubleshooting, teardown, version metadata, an Apache-2.0 license, and related-skill links. Points are deducted because the metadata lacks an author, explicitly reported by BENCHMARK, and there is no clear maintainer, changelog, or formal update path.
The goals, input modes, calibration destination, deployment order, and verification criteria are substantially specified, and the committed eval files and benchmark report provide task-level evaluation signals. Points are deducted because these are static repository artifacts, not independent proof that the current revision deploys successfully, and the full workflow still requires considerable environment preparation and manual handling of calibration anomalies.
The documentation supplies many auditable commands, paths, environment variables, API endpoints, and acceptance checks, while the eval files define concrete checks. Points are deducted because the supplied material contains no independently verifiable CI workflow or multi-source reproduction; the benchmark report cannot substitute for independent validation of this revision, so the score is capped at 5.
- Before execution, confirm authorization to modify .env files, host ACLs, Docker containers, and named volumes, and confirm acceptance of NGC/HuggingFace download, network, disk, and GPU costs.
- Never expose NGC_CLI_API_KEY, HF_TOKEN, or other credentials in logs or responses; the skill reads and writes credentials but does not define a complete redaction policy.
- Mainland-China networks may not reliably reach HuggingFace, NGC, Google STUN, or the random UDP required by WebRTC; validate image, model, signaling, and media reachability and alternatives first.
- Placeholder group, region, and place values can distort BEV maps and region analytics; a successful startup does not prove spatial-result correctness.
What does this skill do, and when should you use it?
This skill deploys and operates the RTVI-CV-3D microservice in MV3DT mode, combining per-camera DeepStream perception with BEV Fusion over calibrated cameras. It supports the bundled four-camera sample dataset, local video files, and RTSP streams, and can chain to the video-calibration skill when calibration is missing. Its scope is the MV3DT service and supporting infrastructure, not the full warehouse blueprint's agent, LLM, or VLM stack. It also covers camera configuration, verification, troubleshooting, and teardown.
It uses routing questions to select an extended or minimal profile, data source, calibration path, detector, and dataset slug. It checks the Video Search and Summarization repository, NGC API key, GPU hardware profile, Docker NVIDIA runtime, extracted application data, video files, and calibration files, then guides environment settings, camera-name normalization, NUM_STREAMS synchronization, VST sensor state, and scoped host ACLs. When calibration is missing, it chains to the AutoMagicCalib workflow; it then starts MV3DT, BEV Fusion, MQTT, broker, VST, and optional extended services with Docker Compose from the warehouse MV3DT application directory. After startup, it verifies perception FPS, fusion readiness, mdx-bev output, and the VST video wall, while mapping listed symptoms to likely causes.
- A warehouse-video engineer enabling multi-camera 3D tracking on the bundled four-camera sample.
- A deployment operator running MV3DT on local MP4 files named for their cameras.
- A systems integrator connecting multiple live RTSP cameras after calibration.
- A user who needs bounding-box overlays in the VST video wall and therefore selects the extended profile.
- An edge or IGX Thor operator who only needs Kafka/Redis metadata and prefers the smaller minimal profile.
What are this skill's strengths and limitations?
- Supports sample data, local videos, and RTSP streams.
- Includes a documented handoff to automatic calibration when calibration is absent.
- Clearly separates MV3DT deployment from the full warehouse blueprint.
- Provides concrete checks for GPU stream limits, camera naming, permissions, and common failures.
- Requires an NVIDIA GPU, NGC image access, the Docker NVIDIA runtime, and a separate application-data directory.
- Each GPU has a defined MV3DT stream cap; excess cameras are trimmed at deployment time.
- RTSP deployments require calibration and a camera_info.json Sensor Info File.
- The supplied material provides no standalone test suite, platform matrix, or complete reference-file contents.
- Recovery actions involving host ACLs, Docker volumes, or VST sensor state require careful confirmation.
How do you install this skill?
Install the individual skill with the NVIDIA skills CLI: npx skills add nvidia/skills --skill vss-deploy-detection-tracking-3d --yes. The README does not specify the exact installation directory for every client; the CLI prompts for the destination. Before use, the environment also needs the Video Search and Summarization repository, an NGC CLI/API key, a supported NVIDIA GPU, the Docker NVIDIA runtime, and a separately extracted vss-warehouse-app-data directory.
How do you use this skill?
Prompt an agent with a concrete request after the skill is loaded, for example: "Deploy RTVI-CV-3D on my videos here: <path/to/videos>." The agent then asks about profile size, data source, calibration coverage, and, when automatic calibration is required, the detector and dataset slug. The source references the detailed deployment commands but does not include their complete contents, so the full command sequence is not reproduced here. The extended profile is the default and supports video-wall overlays; the minimal profile reduces the container footprint but does not provide overlays.
How does this skill compare with similar options?
Compared with vss-deploy-detection-tracking-2d, this skill targets multi-camera 3D detection and tracking with BEV Fusion, while the 2D skill targets single-camera detection. Compared with vss-deploy-profile, it deploys MV3DT only and excludes the full warehouse blueprint's agent, LLM, and VLM stack. Compared with vss-generate-video-calibration, it consumes calibration output and completes MV3DT deployment; the calibration skill owns AMC deployment, capture, calibration APIs, and export.