Mixture of experts

Grouped expert routing

Experts are organized into groups; the router first picks groups, then experts inside them. Friendlier to multi-GPU serving.

First page of DeepSeek-V3 Technical ReportIntroduced inDec 2024DeepSeek-V3 Technical ReportDeepSeek-AI · arXiv 2412.19437 ↗
Grouped routing: pick a group, then expertsgroups map to devices · each token’s experts stay on a couple of GPUstokenGPU 0GPU 11. pick the group   2. pick experts inside it

With hundreds of experts spread across devices, letting a token pick any k experts scatters traffic everywhere. Grouped routing first selects the best expert groups (which map to devices), then the best experts within those groups. DeepSeek introduced this ‘node-limited’ routing to cap cross-device communication; you will see it in graphs as a two-stage top-k.

Adoption over time

Share of new models that have included grouped expert routing over time.

3%202220232024202520263%20222023202420252026

See it in real models

Open any of these on hfviewer to find this block in the interactive architecture graph.

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4text-generation · ↓ 1.5M · ♡ 429Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16text-generation · ↓ 965k · ♡ 813Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16text-generation · ↓ 963k · ♡ 423Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8text-generation · ↓ 649k · ♡ 357Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4text-generation · ↓ 533k · ♡ 174Open in visualizer nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4text-generation · ↓ 442k · ♡ 128Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16text-generation · ↓ 426k · ♡ 329Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4text-generation · ↓ 330k · ♡ 311Open in visualizer internlm/Intern-S1-Pro architecture graphinternlm/Intern-S1-Proimage-text-to-text · ↓ 186k · ♡ 278Open in visualizer inclusionAI/LLaDA2.0-mini architecture graphinclusionAI/LLaDA2.0-minitext-generation · ↓ 177k · ♡ 70Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8text-generation · ↓ 151k · ♡ 275Open in visualizer nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16text-generation · ↓ 136k · ♡ 184Open in visualizer

Browse all 57 models with this in the catalog →

Related concepts

MoE router Shared expert MoE experts

hfviewer renders the full architecture of 4,000+ Hugging Face models as interactive graphs. Hover any block to see what it does, with this model’s real numbers.

Browse all model graphs →