A recurrent state-space scan replaces attention: constant memory per token, linear time in sequence length.
State-space blocks (Mamba-2's SSD scan here) carry information through a fixed-size hidden state that is updated token by token, instead of comparing every token against every other one. That makes both compute and memory linear in sequence length. Hybrid models such as Nemotron-H use mostly Mamba blocks with a few attention layers mixed in for tasks that need exact token recall.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →