A small MLP that translates vision-encoder features into the language model's embedding space.
The projector (often just one or two linear layers) maps visual features into vectors that look like text-token embeddings to the LLM, which then attends over them like any other tokens. It is tiny compared to the towers it connects but is where 'alignment' between modalities is learned - many multimodal training recipes tune the projector first, alone.
Open any of these on hfviewer to find this block in the interactive architecture graph.
hfviewer renders the full architecture of 2,300+ Hugging Face models as interactive graphs - hover any block to see what it does, with this model's real numbers.
Browse all model graphs →