An always-on expert that every token passes through, alongside whatever the router picks.
DeepSeek-style MoE adds one or more shared experts that process every token unconditionally, in parallel with the routed top-k experts. The shared expert captures common knowledge every token needs, freeing the routed experts to specialize harder - which measurably improves quality at the same compute. Most 2025-generation MoE models (DeepSeek, GLM, Qwen-MoE, Nemotron) include one.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →