Several query heads share one key/value head, shrinking the KV cache several-fold with little quality loss.
In classic multi-head attention every head has its own keys and values, and all of them must be cached for every generated token. GQA keeps many query heads but lets groups of them share a single key/value head - with one shared KV head this becomes multi-query attention. Since KV-cache size is what limits batch size and context length at inference time, nearly every recent LLM uses it.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →