Mixture of experts

Grouped expert routing

Experts are organized into groups; the router first picks groups, then experts inside them. Friendlier to multi-GPU serving.

First page of DeepSeek-V3 Technical ReportIntroduced inDec 2024DeepSeek-V3 Technical ReportDeepSeek-AI · arXiv 2412.19437 ↗
Grouped routing: pick a group, then expertsgroups map to devices · each token’s experts stay on a couple of GPUstokenGPU 0GPU 11. pick the group   2. pick experts inside it

With hundreds of experts spread across devices, letting a token pick any k experts scatters traffic everywhere. Grouped routing first selects the best expert groups (which map to devices), then the best experts within those groups. DeepSeek introduced this ‘node-limited’ routing to cap cross-device communication; you will see it in graphs as a two-stage top-k.

Adoption over time

Share of new models that have included grouped expert routing over time.

1%202220232024202520261%20222023202420252026

See it in real models

Open any of these on hfviewer to find this block in the interactive architecture graph.

nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4text-generation · ↓ 2.8M · ♡ 415Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16text-generation · ↓ 931k · ♡ 805Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16text-generation · ↓ 899k · ♡ 414Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4text-generation · ↓ 765k · ♡ 174Open in visualizer nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8text-generation · ↓ 613k · ♡ 355Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16text-generation · ↓ 469k · ♡ 303Open in visualizer internlm/Intern-S1-Pro architecture graphinternlm/Intern-S1-Proimage-text-to-text · ↓ 386k · ♡ 279Open in visualizer nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4text-generation · ↓ 233k · ♡ 279Open in visualizer nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 architecture graphnvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8text-generation · ↓ 208k · ♡ 271Open in visualizer inclusionAI/LLaDA2.0-mini architecture graphinclusionAI/LLaDA2.0-minitext-generation · ↓ 151k · ♡ 70Open in visualizer nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4 architecture graphnvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-NVFP4text-generation · ↓ 129k · ♡ 128Open in visualizer stelterlab/NVIDIA-Nemotron-3-Nano-30B-A3B-AWQ architecture graphstelterlab/NVIDIA-Nemotron-3-Nano-30B-A3B-AWQtext-generation · ↓ 74k · ♡ 11Open in visualizer

Browse all 35 models with this in the catalog →

Related concepts

MoE router Shared expert MoE experts

hfviewer renders the full architecture of 3,100+ Hugging Face models as interactive graphs. Hover any block to see what it does, with this model’s real numbers.

Browse all model graphs →