MoE Dispatcher Selection Guide
Choose alltoall, DeepEP, or HybridEP from the hardware, expert-parallel degree, and optimization stage.
The material mainly provides configuration-selection and performance-triage guidance, with no credential access, exfiltration, or destructive defaults, and it discloses backend import and topology-mismatch risks. It does not define user confirmation, dependency security checks, rollback, or complete data-flow boundaries; the supplied license metadata is also NOASSERTION, so points are deducted.
The SKILL.md is internally consistent and covers dispatcher conditions, tuning parameters, and import-failure feedback. However, the skill directory provides no executable tests or reproduced key paths, and several performance claims lack diagnostic validation procedures. Static review caps this dimension at 10, so 8 is awarded.
Audience, hardware scenarios, EP scale, and trigger phrases are reasonably clear, and the NVL72 limitation and backend-availability gate are stated. Non-fit ranges, formal input/output expectations, and Chinese-language support are not systematically defined, so adaptability evidence is incomplete and points are deducted.
The guide uses useful progressive sections for hardware, models, parameters, interactions, and pitfalls; license, owner, validation date, and version appear in companion metadata. Recommended Instructions, Examples, author, and tags are missing, while changelog, maintenance ownership, update path, and version dates are incomplete or inconsistent, so points are deducted.
The skill can directly produce hardware/EP dispatcher recommendations, flex backend names, SM starting values, and the correct interpretation of import failures. Its core value is clear, but it lacks a complete runnable configuration example and independent verification, leaving users to remeasure several performance claims. Static calibration therefore limits this to 6.
Committed evaluation material, one positive smoke task, and summarized measured results provide limited auditability. Coverage is thin, there are no negative-trigger cases, and the benchmark sources are not supplied as independently reproducible primary artifacts. Static calibration limits this dimension to 3.
- Do not treat DeepEP or HybridEP import failures as performance data; verify the runtime package first, then compare under the same container, routing mode, PP layout, and CUDA Graph scopes.
- HybridEP is topology-sensitive; a mismatched NUM_OF_HYBRID_EP_RANKS_PER_NVLINK_DOMAIN setting may reduce performance and can affect correctness.
- Treat the reported performance ratios and recommendations as empirical summaries; publication would benefit from primary benchmark artifacts, a version matrix, negative-trigger tests, and complete runnable examples.
What does this skill do, and when should you use it?
This skill guides NeMo MBridge users through MoE token-dispatcher selection for performance tuning. It compares alltoall, DeepEP, and HybridEP across H100, B200, and GB200/GB300 NVL72 systems, EP sizes, and model families. It also covers backend availability checks, communication SM tuning, routing mode, CUDA Graph scopes, CPU affinity, and pipeline layout. It is useful for bring-up and regression diagnosis, but it is not an independent benchmarking tool.
Provides decision tables by hardware, EP degree, and model family; documents the flex-backend settings and communication SM controls for DeepEP and HybridEP; requires checking runtime-package availability before interpreting timings; summarizes the supplied evidence for DSV3, Qwen3, Qwen3-Next, and MoE VLM workloads; and identifies interactions and failure causes involving CUDA Graphs, EP overlap, FP8, CPU affinity, and pipeline layout.
- A NeMo MBridge user bringing up cross-node EP on H100 or B200 and choosing between alltoall and DeepEP.
- An engineer running a large MoE model on GB200 or GB300 NVL72 who needs both throughput and memory headroom.
- A performance engineer tracing an MoE regression or crash after a dispatcher configuration change.
- A team comparing BF16 or FP8 behavior for DSV3, Qwen3, Qwen3-Next, or MoE VLM workloads.
What are this skill's strengths and limitations?
- Gives concrete alltoall, DeepEP, and HybridEP guidance by hardware, EP scale, and model family.
- Clearly separates missing-backend environment failures from performance evidence.
- Covers practical tuning variables including communication SM counts, NVLink-domain size, routing mode, and CUDA Graph scope.
- Considers throughput, memory headroom, correctness bring-up, and regression diagnosis together.
- DeepEP and HybridEP results depend on hardware, topology, routing mode, and the surrounding software stack; they are not universal conclusions.
- In the supplied H100 Qwen3 30B run, DeepEP and HybridEP failed before the first iteration because their runtime packages were absent, so it does not rank H100 flex-dispatcher throughput.
- The supplied materials do not include this skill's own test suite, standalone reproduction scripts, or complete benchmark tables.
- The skill provides decision and configuration guidance; it does not automatically run training, benchmark jobs, or repair the environment.
How do you install this skill?
Install the specific skill with the skills CLI command documented in the repository README:
npx skills add nvidia/skills --skill nemo-mbridge-perf-moe-dispatcher-selection --yes
The README says the CLI installs the skill to a selected destination; cloning the repository or copying folders manually is not required.
How do you use this skill?
After installation, ask the agent a dispatcher-selection or regression-triage question, for example: "Which dispatcher should I use for Qwen3 MoE on a GB200 NVL72 system?" or "Trace this MoE regression after changing the dispatcher configuration." The skill uses hardware, EP degree, optimization stage, and runtime-package availability to frame its recommendation. The supplied source does not document a standalone launcher or execution command.
How does this skill compare with similar options?
The skill compares three named MoE token dispatchers. alltoall is positioned for initial correctness bring-up, small EP configurations, and debugging; DeepEP for Hopper, B200, and visible cross-node EP communication; and HybridEP for GB200/GB300 NVL72, large EP degrees, and cases where memory headroom matters.