What does this skill do, and when should you use it?
This skill helps users deploy inference endpoints from TAO model checkpoints. It resolves architecture-specific container images, builds job payloads and inner commands, and delegates execution to platform skills for local Docker, Brev, Slurm, or Kubernetes. It maintains a service registry, checks readiness, and routes requests to a specific running service. Model weights must come from the HuggingFace Hub or a local container path; cloud-storage URIs are unsupported.
Collects network_arch, model_path, platform, and num_gpus; reads architecture configuration, service.yaml, request.yaml, code-templates.yaml, and the selected platform skill; resolves mapped or environment-overridden container images; builds job JSON, environment variables, and architecture-specific startup commands; starts and stops containers or platform jobs; writes the /tmp/tao-inf-ms-state.json registry; allocates distinct ports for concurrent local Docker and Brev services; polls readiness; resolves a target endpoint; sends OpenAI-compatible POST requests to /v1/chat/completions; and handles 200, 202, 400, 500, and 503 responses.
- A TAO user wants to start an inference microservice from a HuggingFace Hub checkpoint.
- A developer needs to run a cosmos-rl or cosmos-predict2.5 endpoint on local Docker.
- A platform engineer wants to deploy a TAO inference container on Brev, Slurm, or Kubernetes.
- A user runs multiple inference services and needs request routing by job_id or network_arch.
- An operator needs to stop a selected TAO inference service and clean its registry entry.
How do you install this skill?
- Execution starts or stops containers and may invoke SSH, kubectl, port forwarding, and remote platform operations; obtain explicit confirmation of the target, platform, ports, namespace, and stop target before execution.
- skill_info.yaml requires cloud URIs, cloud credentials, and cloud outputs, while SKILL.md and service.yaml explicitly reject cloud storage; reconcile the documentation before use to avoid invalid commands.
- The cosmos-predict2.5 template uses an HF_TOKEN placeholder rather than an explicit os.environ-based secret read; fix secret injection and verify the image, CLI, and registered model key.
- This assessment is static only; containers, platform commands, requests, and stop flows were not executed. The BENCHMARK.md PASS is not independent runtime validation.
- Shell / CLI
- Network access
- Local filesystem
DockerPythonkubectlSSH
Install the named skill from NVIDIA/skills with the skills CLI: npx skills add nvidia/skills --skill tao-run-inference-service --yes. The README does not state whether this command automatically installs the platform skills required by this skill; confirm those separately from the repository documentation.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/tao-run-inference-service" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Once installed, send your agent any of these to trigger it:
- Start a TAO inference service on local-docker with network_arch=cosmos-rl, model_path=hf_model://org/model, and num_gpus=1.
Give the loaded agent a request such as: “Start a TAO inference service on local-docker with network_arch=cosmos-rl, model_path=hf_model://org/model, and num_gpus=1.” You must provide network_arch, model_path, and platform; num_gpus defaults to 1. model_path may be hf_model://<org>/<model> or a local container path, but not s3://, gs://, or az://. Provide a job_id when querying or stopping a service. Before each request, the agent must ask about max_tokens, top_p, temperature, and any applicable architecture-specific parameters.
What are this skill's strengths and limitations?
- Covers local Docker, Brev, Slurm, and Kubernetes execution paths.
- Includes service registration, readiness checks, multi-service routing, and concurrent port allocation.
- Defines a clear environment-variable policy for secrets such as HF_TOKEN.
- Separates the startup commands and request constraints for cosmos-rl and cosmos-predict2.5.
- Execution depends on platform skills whose full contents are not included in the supplied material.
- Supports HuggingFace Hub and local container paths only; cloud-storage URIs are unsupported.
- Sampling parameters must be addressed before every request, and multi-service requests cannot silently fall back to latest.
- The supplied material provides no test suite, performance figures, or evidence of actual platform validation.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| TAO Inference Microservice Deployment this page ✓ NVIDIA · Official | 45 · Not recommended | ★ 3.8k | today | Apache-2.0 |
| TAO NVIDIA GPU Host Setup ✓ NVIDIA · Official | 55 · Use with care | ★ 3.8k | today | Apache-2.0 |
| TAO Jobs on Kubernetes ✓ NVIDIA · Official | 53 · Use with care | ★ 3.8k | today | Apache-2.0 |
| TAO on Brev GPU Instances ✓ NVIDIA · Official | 51 · Use with care | ★ 3.8k | today | Apache-2.0 |
| TAO Local Docker Runner ✓ NVIDIA · Official | 48 · Use with care | ★ 3.8k | today | Apache-2.0 |
How did FollowSkills review this skill?
The documentation explicitly forbids requesting or hard-coding secrets and restricts model paths, cloud URIs, and some environment variables. However, it permits Bash and Write and directs Docker, SSH, kubectl, and service-stop side effects without explicit pre-start confirmation, clear privilege boundaries, or a complete rollback plan. skill_info.yaml also contradicts the main document on cloud storage and credentials, so points are deducted.
The main workflow, platform branches, registry handling, readiness polling, and several error types are documented in useful detail. Static files do not demonstrate that the key paths run, platform skills and dependency versions are outside the selected skill evidence, and templates contain placeholders and unverified environment handling. The score is therefore conservative and respects the static ceiling.
Use cases, inputs, platforms, architectures, routing rules, and several non-fit cases are fairly clear. Chinese-language support is not addressed, platform and network assumptions are substantial, mainland-China reachability is not discussed, and the cloud-storage contradiction can misdirect input collection and triggering.
The SKILL.md has a readable structure, references data files and platform skills, and includes some version and maintenance signals. However, the benchmark reports a missing recommended Examples section, nesting and author-format findings remain, changelog and maintenance/update ownership are incomplete, skill_info.yaml is stale and contradictory, and license metadata is NOASSERTION.
With the required environment and platform skills available, the workflow covers service startup, requests, routing, readiness checks, and stopping, giving it practical value. There is no statically verifiable real output, execution depends on external images, runtimes, platforms, and omitted configuration, and contradictory documentation can cause setup errors, so only limited core-task credit is warranted.
Versioned reference files, code templates, an evaluation report, and one positive evaluation task provide some auditability. There is no committed test suite or real CI evidence covering the key paths, and the benchmark is largely self-reported, so independent reproduction is thin and the static ceiling applies.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →