What does this skill do, and when should you use it?
This skill is for engineers extending context parallelism in Megatron-Bridge. It explains nested context-parallel process groups configured with cp_comm_type="a2a+p2p" and hierarchical_context_parallel_sizes. It covers configuration constraints, Megatron-Core and Bridge code anchors, Transformer Engine integration, failure modes, and log-based verification. Bridge currently supports hierarchical context parallelism only through MPU, and no dedicated end-to-end test is provided.
Provides a minimal Bridge override for context_parallel_size, cp_comm_type, hierarchical_context_parallel_sizes, and use_decentralized_pg; identifies validation, process-group initialization, and Transformer Engine attention integration points; documents product and sequence-length constraints, silent failure risks on older stacks, and log markers for confirming HCP activation; supplies a decentralized-PG unit-test command and a 4-GPU manual smoke-test command.
- A Megatron-Bridge engineer needs to scale context parallelism beyond KV heads using multi-level process groups.
- A distributed-training engineer is investigating OOM or a performance regression after changing CP configuration.
- A maintainer needs to validate that hierarchical_context_parallel_sizes matches context_parallel_size when using a2a+p2p.
- A test engineer needs to verify HCP activation through an existing unit test, a 4-GPU run, and initialization logs.
How do you install this skill?
- Do not use the decentralized-PG path when HCP is expected; the supplied evidence says it creates only flat CP groups.
- Before execution, confirm the Transformer Engine version, the product of hierarchical sizes, sequence-length divisibility, and logs containing HIERARCHICAL_CONTEXT_PARALLEL_GROUPS.
- No checked-in Bridge HCP end-to-end training test or formal recipe is provided; add validation and a rollback plan before production use.
- The benchmark report and skill card provide publication evidence but do not replace independent reproduction; license metadata remains NOASSERTION.
- Shell / CLI
- Local filesystem
Megatron-BridgeMegatron-CoreTransformer Engine >= 1.12.0uvPyTorch distributed4 GPUs for the manual smoke check
Install the specific skill with the skills CLI command documented in the repository README:
npx skills add nvidia/skills --skill nemo-mbridge-perf-hierarchical-context-parallel --yes
The README does not document a separate manual-copy procedure or other skill-specific installation path.
tmp="$(mktemp -d)"
git clone --depth 1 https://github.com/NVIDIA/skills.git "$tmp"
mkdir -p ~/.claude/skills
cp -R "$tmp/skills/nemo-mbridge-perf-hierarchical-context-parallel" ~/.claude/skills/
rm -rf "$tmp"Generated from the source repository and skill path; it copies only this skill's folder. If the author's install steps above differ, follow those first. To scope it to one project, replace ~/.claude/skills with that project's .claude/skills.
How do you use this skill?
Ask a compatible Agent Skills client to configure, troubleshoot, or verify Megatron-Bridge hierarchical context parallelism, including OOM or regression investigations. For a manual check, run:
CUDA_VISIBLE_DEVICES=0,1,2,3 uv run python -m torch.distributed.run --nproc_per_node=4 \
scripts/training/run_recipe.py \
--recipe llama32_1b_pretrain_config \
model.context_parallel_size=4 \
model.cp_comm_type=a2a+p2p \
"model.hierarchical_context_parallel_sizes=[2,2]" \
train.train_iters=2
Confirm that logs create HIERARCHICAL_CONTEXT_PARALLEL_GROUPS and that training completes at least one step. If only CONTEXT_PARALLEL_GROUP appears, HCP is inactive.
What are this skill's strengths and limitations?
- Includes a directly usable Bridge configuration and verification commands.
- States the product, sequence-length, and Transformer Engine version constraints explicitly.
- Maps configuration and initialization code while documenting silent failure risks and log-based diagnosis.
- Bridge currently supports HCP only through MPU; decentralized PG leaves HCP unset.
- There is no dedicated Bridge HCP end-to-end test or checked-in Bridge recipe exercising HCP directly.
- Single-GPU load helpers clear hierarchical_context_parallel_sizes.
- The manual smoke check requires 4 GPUs, and the skill provides no evidence of broader platform coverage.
How does this skill compare with similar options?
Side by side with related skills; every score comes from the same FSRS standard.
| Skill | FS score | Stars | Last updated | License |
|---|---|---|---|---|
| NeMo MBridge Hierarchical Context Parallelism this page ✓ NVIDIA · Official | 51 · Use with care | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge CUDA Graphs ✓ NVIDIA · Official | 55 · Use with care | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge GPU Memory Tuning ✓ NVIDIA · Official | 61 · Recommended | ★ 3.8k | today | Apache-2.0 |
| NeMo MBridge Parallelism Strategy Guide ✓ NVIDIA · Official | 58 · Recommended | ★ 3.8k | today | Apache-2.0 |
| Megatron-Bridge Communication Overlap Guide ✓ NVIDIA · Official | 54 · Use with care | ★ 3.8k | today | Apache-2.0 |
The skill explicitly references a2a+p2p, pure a2a, and p2p as context-parallel communication options and points to a decision tree, but it does not provide complete performance comparison data.
How did FollowSkills review this skill?
The skill only guides configuration and log inspection; no malware, credential handling, or covert exfiltration is evident. It discloses the MPU-path, TE-version, and process-group constraints. Points are deducted because execution confirmation, rollback, permission boundaries, and data-flow disclosure are incomplete, and the supplied license metadata is NOASSERTION.
Configuration constraints, pitfalls, and failure-log signals are broadly consistent, with diagnosable assertions and log criteria. Static review cannot establish that the commands run successfully; there is no dedicated Bridge end-to-end test, and dependency or abnormal-environment feedback is thin, so the score remains below the static ceiling.
The audience, trigger terms, CP=4 example, and MPU-only boundary are clear, including the non-fit decentralized-PG path. Broader input boundaries, model-family coverage, and Chinese-language guidance are absent. Mainland-China reachability is not specifically assessed, but the core function does not depend on an overseas online service, so no major additional deduction is applied.
The skill is readable and layered around enablement, code anchors, implementation mapping, pitfalls, and verification; card.yaml adds limitations, evidence, and follow-up work. It lacks the recommended Instructions and Examples sections, author/tags, explicit maintenance ownership, changelog, and a clear stable-version policy; the evaluation files record these omissions.
It directly supports the core task of supplying configuration, constraints, the TE requirement, and log proof, and the supplied evaluation report shows limited correctness and effectiveness results. However, there is only one positive task and no Bridge HCP end-to-end training validation, so practical correctness and marginal benefit over manual work still require review; the static score is therefore conservative.
Megatron-Core and Bridge code paths, assertions, log criteria, validation statuses, and an evaluation task provide some traceability. The evidence is mainly repository-local, without independent reproduction, tests covering the key HCP path, or multi-source corroboration, so the score stays below the static ceiling.
Open a dimension to read why it scored that way
Evidence confidence：Low — Mostly static review, author material or a limited demo; useful for discovery, not high-risk decisions.
See the full review method →