What does this skill do, and when should you use it?
This skill is for developers training language models with NeMo MBridge, Megatron-LM, and Megatron Bridge. It covers mock- or real-data training, MLM-versus-Bridge loss-correlation tests, common recipes, and multi-GPU examples. It also documents CLI-to-configuration translation, Megatron-Core submodule switching, and practical troubleshooting. It is a good fit for reproducing experiments, comparing implementations, or investigating loss-curve divergence after a code change.
It guides users through running MLM with pretrain_gpt.py and Bridge with scripts/training/run_recipe.py, using vanilla_gpt_pretrain_config for matched correlation tests. It provides executable one-GPU and two-GPU TP=2 examples, explains how to compare lm loss values, documents Megatron-Core version switching and submodule resynchronization, and flags stale checkpoints, PYTHONPATH, scheduler overrides, and MoE memory pitfalls.
- A training engineer validating that Megatron-LM and Megatron Bridge produce matching BF16 losses under equivalent settings.
- A researcher running a quick GPT training correlation test with mock data.
- A NeMo MBridge user translating Megatron-LM CLI arguments into Bridge configuration overrides.
- A developer diagnosing loss curves that diverged after a repository commit.
- An engineering team testing a two-GPU, tensor-parallel-size-2 training setup.
How do you install this skill?
- Before execution, confirm that nemo_experiments contains only disposable prior experiments and use backup or a recoverable cleanup approach.
- CUDA, PyTorch, Megatron-Core, Megatron-LM, and Bridge compatibility versions are not pinned; verify the local environment before real training.
- Do not use sensitive real data in mock or correlation examples, and do not expose credentials through logs, configurations, or repository output.
- The evaluation evidence covers only one positive smoke task and does not establish usability across multi-GPU, real-data, translation-script, or failure paths.
- Shell / CLI
- Local filesystem
uvPythonPyTorchMegatron-LMMegatron BridgeGit
Install the skill from NVIDIA’s catalog with: npx skills add nvidia/skills --skill nemo-mbridge-mlm-bridge-training --agent codex --yes. The README documents no separate installation procedure for this skill; after installation, it becomes available when the agent next loads skills and encounters a relevant task.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/nemo-mbridge-mlm-bridge-training" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Ask the agent questions such as How do I run an MLM vs Bridge correlation test?, How do I compare MLM and Bridge loss curves?, or Why did the loss curves diverge after a commit? For execution, use uv run python -m torch.distributed.run; MLM commands require PYTHONPATH=3rdparty/Megatron-LM:$PYTHONPATH, and nemo_experiments should be removed before a fresh Bridge correlation run.
What are this skill's strengths and limitations?
- Includes copyable one-GPU and multi-GPU training commands.
- Names the Bridge recipe, entry points, launcher, and fresh-run cleanup explicitly.
- Covers correlation testing, common recipes, Megatron-Core switching, and common failure modes.
- Defines a concrete BF16-rounding criterion for comparing losses.
- Focused on Megatron-LM and Megatron Bridge rather than general distributed-training guidance.
- Examples emphasize small configurations and mock data; coverage of real datasets, specific hardware, and multi-node deployment is limited.
- The SKILL.md provides no standalone test suite or platform compatibility matrix.
- The repository metadata is `NOASSERTION`, while the skill declares Apache-2.0; licensing should be checked for the intended use case.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| NeMo MBridge Training Guide this page ✓ NVIDIA · Official | 44 · Not recommended | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge GPU Memory Tuning ✓ NVIDIA · Official | 61 · Recommended | ★ 3.8k | today | Apache-2.0 |
| Megatron Bridge Activation Recompute ✓ NVIDIA · Official | 59 · Recommended | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge CPU Offload ✓ NVIDIA · Official | 55 · Use with care | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge CUDA Graphs ✓ NVIDIA · Official | 55 · Use with care | ★ 3.8k | today | Apache-2.0 |
The skill explicitly compares Megatron-LM (MLM) with Megatron Bridge through their training entry points, configuration mapping, and loss curves, making it useful for checking behavioral agreement under matched parameters.
How did FollowSkills review this skill?
The skill constrains entry points, environment setup, and modification of the third-party directory, and states that no external credentials are identified. However, it includes an unconfirmed rm -rf nemo_experiments command and lacks backup, rollback, and data-flow guidance; boundaries around real data and training artifacts are incomplete, so points were deducted.
SKILL.md gives concrete single-GPU, multi-GPU, scheduler, PYTHONPATH, OOM, and submodule-switching paths with several operational pitfalls. This is only a static review, however: the skill has no dedicated test suite or CI covering its key training paths, abnormal-input handling and failure diagnostics are limited, and the static ceiling is 10, so the score remains moderate.
The audience, trigger phrases, MLM-versus-Bridge comparison, correlation testing, and multi-GPU scenarios are reasonably clear, with some parameter-mapping and MoE limitations disclosed. Non-fit boundaries, hardware and version prerequisites, Chinese-language support, and mainland-China network fit are not addressed, and trigger precision is supported by only one positive task, so points were deducted.
The skill includes front matter, an Apache-2.0 declaration, command examples, recipes, pitfalls, limitations, and a skill card. It lacks the recommended Instructions and Examples sections, an author field, in-file version/change history, and a sufficiently explicit maintenance owner or update path; the benchmark also records schema omissions, so points were deducted.
The commands and parameters are concrete enough to support correlation smoke tests, common recipes, and multi-GPU examples. The benchmark reports a passing single positive task with strong results, but coverage is very narrow and the execution cannot be independently confirmed from this static review; users still need to adapt to local dependencies and hardware, so the score is below the static maximum.
The files identify evidence files, validation statuses, an evaluation task, and reported results, and disclose that CI correlation testing remains to be added. There is no skill-specific committed test suite or reproducible CI chain, while the report covers only one task and one attempt, leaving limited independent corroboration; therefore only limited points were awarded.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →