Merges neighboring visual tokens into fewer, wider ones so the language model reads a shorter sequence.
A full-resolution vision encoder emits far more tokens than a language model wants to attend over. Patch mergers (or pixel-shuffle/unshuffle stages) fuse each small neighborhood of visual tokens into one wider token - commonly a 4× reduction - before handing them to the LLM. It is a large inference-cost saving with modest information loss.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →