Clinical ASR Fine-Tuning
Fine-tune Parakeet TDT v2 with NeMo for clinical vocabulary and close the loop with offline KER evaluation.
The skill defines Stage 4 gates and requires an explicit YES before creating a billed Brev instance, while disclosing compute, storage, stop/delete choices, and NVIDIA_API_KEY use. However, it lacks clinical-audio de-identification, least-privilege, secret-handling, retention, and complete dependency-integrity guidance; Docker and model supply-chain controls are also limited, so points are deducted.
Workflow gates, path rewriting, missing-audio checks, OOM handling, precision fallback, and overfitting guidance are detailed. However, there is no committed test suite or executable driver covering key paths, important command details depend on referenced material, and the loader may silently drop rows; static evidence therefore remains limited and below the 10-point ceiling.
Audience, trigger phrases, non-fit cases, KER/manifest thresholds, and routing are clearly stated. The default is English/en-US, with no Chinese clinical-speech adaptation guidance, and Brev, NVIDIA container, and API reachability from mainland China are not assessed, so points are deducted.
The documentation has useful layering, version 1.0.0, author, tags, license, artifacts, limitations, troubleshooting, and references. Maintenance ownership, changelog/update path, and dependency governance are incomplete, and the supplied license metadata is NOASSERTION, so full marks are not justified.
It specifies a coherent split, NeMo SFT, offline cycle N+1 evaluation, decision table, and expected artifacts; the benchmark file reports an internal passing evaluation. Static review cannot independently confirm the empirical result, and users must still build the driver, manage data, and review the model, so the score is conservatively limited.
The files provide reference-manifest KER values, per-category results, benchmark claims, and evaluation cases, giving some auditability. There is no independent external corroboration, committed key-path test suite, or reproducible execution record; the empirical claims remain primarily author-provided, so the score is low.
- Clinical audio may contain sensitive health information; confirm de-identification, access control, encryption, retention, and third-party cloud-processing compliance before use.
- Do not rely solely on the documented KER improvement; reproduce it on an independent, representative, leakage-free validation set and check WER/CER/SER plus category-level regressions.
- Brev instances, container images, model downloads, and NVIDIA API dependencies may create cost or reachability issues; validate mainland-China access in advance and ensure instances are stopped.
- The workflow uses Bash, Docker, GPU cloud resources, and model-file writes; review permissions, dependency versions, and recovery procedures before execution.
What does this skill do, and when should you use it?
This skill covers Stage 4 of NVIDIA’s Clinical ASR Flywheel for users who already have a baseline KER result and an audio manifest. It runs standard supervised fine-tuning in the NeMo container, with nvidia/parakeet-tdt-0.6b-v2 as the recommended base. The stage is gated on priority-category KER above 0.3, at least 100 manifest rows, and at least 5 rows per priority category. After training, it performs offline cycle N+1 evaluation and uses the KER change to decide whether to keep the model, expand the manifest, or stop fine-tuning.
Reads the cycle-N manifest and evaluation result, then creates row-disjoint train.jsonl and validation.jsonl files stratified by entity_category; invokes /opt/NeMo/examples/asr/speech_to_text_finetune.py inside nvcr.io/nvidia/nemo:25.11.01 with Parakeet TDT v2 and the prescribed hyperparameters; produces adapted_model.nemo and training_run_info.json; uses NeMo offline transcribe() to re-transcribe audio, scores WER, CER, KER, and SER, and writes leaderboard_cycle<N+1>.md; optionally hands the .nemo model to riva-asr-custom for deployment.
- A clinical speech team has baseline KER results for medication, condition, or procedure categories and wants targeted SFT with at least 100 manifest rows.
- An ASR engineer wants to improve Parakeet TDT v2 recognition of clinical drug names.
- A researcher needs to compare two evaluation cycles and determine whether fine-tuning materially improved KER.
- A deployment team has a fine-tuned .nemo model and needs to pass its TDT architecture explicitly to Riva NIM tooling.
- A user is considering SFT for Nemotron Speech Streaming and needs to understand its validation-time UNK collapse failure.
What are this skill's strengths and limitations?
- Defines concrete KER gates, dataset-size gates, model selection guidance, and outcome decisions.
- Reports a reference result for Parakeet TDT v2: KER decreased from 0.513 to 0.128, a 75% relative reduction.
- Uses standard NeMo SFT to avoid the known adapter NaN failure on TDT/RNNT decoders.
- Connects fine-tuning, offline re-evaluation, and optional Riva deployment.
- Requires a CUDA host, Docker, the NeMo container, and substantial GPU memory; 24 GB VRAM is comfortable, while 16 GB requires smaller batches.
- Brev introduces per-second compute charges and idle disk costs that the user must manage.
- It is not intended for generic word boosting, users without baseline evaluation, or users without a manifest.
- The default recipe is en-US-specific; other locales require a different base model and revalidated recipe.
- There is no turnkey driver; users organize output paths, run names, and leaderboard rendering themselves.
How do you install this skill?
Install the skill with the skills CLI: npx skills add nvidia/skills --skill digital-health-clinical-asr-finetune --yes. Alternatively, run npx skills add nvidia/skills for interactive selection of the skill and installation destination. The skill also requires finetune-asr and riva-asr-custom to be installed alongside it; the source does not provide separate installation commands for those skills.
How do you use this skill?
Prepare a manifest, baseline KER result from digital-health-clinical-asr-eval, a stratified train/validation split, and a CUDA host. Example prompts include “Drug KER 0.42, 130 rows. SFT?” and “Fine-tune ASR on my clinical vocabulary”. The workflow should use stock NeMo SFT, Parakeet TDT v2, and offline cycle N+1 evaluation. Do not use the broken TDT/RNNT adapter-mixin path or fine-tune nvidia/nemotron-speech-streaming-en-0.6b. Brev is an option when no local GPU is available, but its cost must be acknowledged before provisioning; the source does not provide a turnkey training driver.
How does this skill compare with similar options?
Unlike /finetune-asr, this skill performs clinical ASR supervised fine-tuning rather than generic word boosting or language-model fusion. Unlike /riva-asr-custom, it prepares and evaluates the .nemo model offline, while production serving is delegated to that skill. Users without a baseline evaluation or manifest should use /digital-health-clinical-asr-eval or /digital-health-clinical-asr-build respectively.