Nemotron Speech ASR Customization Orchestrator
Chooses and coordinates the lowest-cost path for adapting speech recognition to a domain or language.
The skill requires scoping first, baseline measurement, least-cost path selection, and user consultation before additional tuning cycles; it also distinguishes offline evaluation from served evaluation, reducing operational confusion. Points are deducted because handling of sensitive speech/customer data, least-privilege permissions, dependency security, rollback, and explicit pre-deployment confirmation are not specified.
The workflow, path ladder, sub-skill ownership, and placeholder fallback guidance are broadly self-consistent, and missing sub-skills are to be disclosed rather than silently assumed. Points are deducted because this review is static and does not reproduce key paths; detailed version compatibility, command failures, and abnormal-input diagnostics are largely delegated to external sub-skills.
Triggers, audience scenarios, input questions, path branches, and non-fit cases are relatively clear, including exclusions for pure deployment, OpenAI Whisper, and text-LLM fine-tuning. Points are deducted because Chinese-language support and mainland-China network reachability are not addressed, and the core workflow depends on multiple external NVIDIA/Riva/NeMo components.
The documentation is layered and includes workflow, path selection, sub-skill registry, planning answers, limitations, version 1.0.0, Apache-2.0 licensing, and repository update guidance. Points are deducted for missing recommended Instructions and Examples sections, nonconforming author metadata, and incomplete skill-level changelog, maintainer, and troubleshooting details.
The skill provides useful high-level ASR adaptation planning, cost-aware routing, delegation, and evaluation criteria; the committed benchmark report indicates positive evaluation results, though it is only source-file evidence here. Points are deducted because the skill does not itself execute training, evaluation, or deployment, final utility depends on other skills, and no independent direct-output verification was performed.
A revision-pinned source, cited primary documentation, evaluation guidance, evals.json, and an NVSkills-Eval report provide some auditability. Points are deducted because independently reproducible execution logs, raw evaluation artifacts, and corroboration from multiple evidence types are absent, so a higher static evidence score is not justified.
- Do not treat this orchestration skill as a training or deployment executor; confirm required sub-skills, versions, and external-service availability first.
- Before processing customer audio, transcripts, or synthetic data, establish privacy, access-control, retention, and data-transfer requirements.
- Before deployment or expensive training, obtain confirmation of budget, hardware, model versions, rollback, and the current Riva/NeMo support matrix.
- For mainland-China users, separately verify reachability of NVIDIA documentation, images, models, and service endpoints.
What does this skill do, and when should you use it?
This skill orchestrates domain and language adaptation for NVIDIA Nemotron Speech (Riva) / NeMo ASR. It scopes audio data, evaluation, target metrics, latency, and hardware before choosing among word boosting, n-gram language modeling, and fine-tuning. It owns planning, routing, and answers about cost, time, and data, while dedicated sub-skills handle training, evaluation, and deployment. It fits teams trying to reduce WER, correct jargon errors, handle noise, or add a language, but it is not a step-by-step training manual.
Collects the target domain or language, error patterns, evaluation metric, real-audio volume, deployment target, and latency or hardware budget; selects word boosting, custom vocabulary/pronunciation, n-gram LM, fine-tuning, or cross-language transfer; routes data generation, training, evaluation, and deployment to named sub-skills; plans blends of real and synthetic data; requires normalized domain-set WER plus a general-set forgetting check; loops for targeted improvements or proceeds to Riva/NIM deployment; and answers questions about data volume, GPUs, time, and cost.
- A call-center team sees industry jargon misrecognized in noisy calls and needs to decide whether boosting, a language model, or fine-tuning is appropriate.
- A speech product team wants to adapt Riva ASR to a new language and needs to assess checkpoints, data requirements, and cross-language transfer.
- An enterprise has domain text but little labeled audio and wants to pilot an n-gram LM for domain phrasing before investing in acoustic fine-tuning.
- An ASR engineering team has transcribed target-domain audio and needs a plan covering NeMo fine-tuning, WER evaluation, forgetting checks, and deployment.
What are this skill's strengths and limitations?
- Uses a cost-ordered ladder from word boosting and language modeling to fine-tuning.
- Covers data planning, training, evaluation, forgetting checks, and deployment routing.
- Requires domain WER and a general-set A/B check instead of relying only on training logs.
- Addresses scarce real data, synthetic data, noise, GPU choice, time, and cost questions.
- Orchestration only; it does not provide the complete training or deployment execution itself.
- Some required sub-skills may be placeholders and then provide only interim guidance.
- Training paths require a GPU, and NeMo/Riva commands, configurations, and model support can change by release.
- The supplied sources provide no skill-specific test results, concrete pricing, or guaranteed WER improvement.
How do you install this skill?
Install the individual skill from NVIDIA’s collection with the skills CLI:
npx skills add nvidia/skills --skill nemotron-asr-finetune --yes
The README does not document a client-specific destination; the CLI prompts for the installation target.
How do you use this skill?
After installation, give the agent a concrete ASR customization request, such as: "Fine-tune ASR for my noisy call-center domain and reduce WER." The agent should first ask about real-audio volume, the target evaluation set, latency/hardware budget, and deployment target, then select the cheapest sufficient path. Available sub-skills perform the detailed training, evaluation, and deployment work; missing sub-skills are identified as placeholders.
How does this skill compare with similar options?
The skill explicitly chooses among word boosting, custom vocabulary/pronunciation, n-gram LM, fine-tuning, and finally training from scratch or cross-language transfer. These are staged ASR customization paths rather than competing products.