Architecture timeline · vision-language

How we got to Qwen3‑VL.

Eleven years from attention over pixels to a natively multimodal frontier model. In vision-language the paper and the weights almost always land together. Every stop links to a plain-English explanation or a live interactive graph.

8 stops 6 papers 7 model graphs 2014 to 2025
Concept Model Paper and model in one release
Seeing becomes sequencing2014 to 2020

Attention learns to look across modalities, and images become token sequences.

ConceptSep 2014Cross-attentionQueries from one sequence, keys and values from another. The first wiring that let language models look at something other than text.other sequence Neural Machine Translation by Jointly Learning to Align and TranslatearXiv 1409.0473 · Sep 2014Read the glossary entry →
Paper + modelOct 2020google/vit-base-patch16-224An image becomes a sequence of patch tokens and the transformer takes over vision. The checkpoints ship with the paper.google/vit-base-patch16-224 architecture graphimage-classification · ↓ 5.5M · ♡ 981An Image is Worth 16x16 Words: Transformers for Image Recognition at ScalearXiv 2010.11929 · Oct 2020Patch embeddingOpen in visualizer →
Pixels meet words2021 to 2023

Contrastive encoders align images with text, then a small projector glues them to a language model.

Paper + modelJan 2021openai/clip-vit-base-patch32One embedding space for images and text. Its vision tower becomes the eyes of a whole generation of VLMs, and it is still the most downloaded vision model on the Hub.openai/clip-vit-base-patch32 architecture graphzero-shot-image-classification · ↓ 23.9M · ♡ 975Learning Transferable Visual Models From Natural Language SupervisionarXiv 2103.00020 · Mar 2021Open in visualizer →
Paper + modelMar 2023google/siglip-so400m-patch14-384Swap CLIP’s softmax for a sigmoid loss and the contrastive encoder trains better at scale. The so400m tower becomes the default eyes of open VLMs.google/siglip-so400m-patch14-384 architecture graphzero-shot-image-classification · ↓ 1.8M · ♡ 681Sigmoid Loss for Language Image Pre-TrainingarXiv 2303.15343 · Mar 2023Open in visualizer →
Paper + modelApr 2023llava-hf/llava-1.5-7b-hfGlue a CLIP tower to an open LLM with a small projector and instruction-tune on images. The 1.5 checkpoint is the one everyone runs.llava-hf/llava-1.5-7b-hf architecture graphimage-text-to-text · ↓ 3.3M · ♡ 367Visual Instruction TuningarXiv 2304.08485 · Apr 2023Multimodal projectorOpen in visualizer →
Native multimodality2024 to 2025

Vision stops being a bolt-on: resolution goes dynamic and the tower trains with the language model.

Paper + modelSep 2024Qwen/Qwen2-VL-7B-InstructNative dynamic resolution and multimodal RoPE, with a patch merger shortening what the language model reads. Vision stops being a bolt-on.Qwen/Qwen2-VL-7B-Instruct architecture graphimage-text-to-text · ↓ 1.8M · ♡ 1kQwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any ResolutionarXiv 2409.19606 · Sep 2024Patch mergerOpen in visualizer →
ModelJan 2025Qwen/Qwen2.5-VL-7B-InstructThe open VLM workhorse: ten million downloads a month for document parsing, grounding and video understanding.Qwen/Qwen2.5-VL-7B-Instruct architecture graphimage-text-to-text · ↓ 10.3M · ♡ 2kOpen in visualizer →
Oct 2025 · where it all lands

Qwen/Qwen3-VL-8B-Instruct

Every idea above, alive in one graph: a ViT tower reading patches, a merger shortening them, and a language model treating vision as just more tokens. Open it and follow the pixels.

Qwen/Qwen3-VL-8B-Instruct architecture graph
Vision encoderPatch embeddingPatch mergerGQARoPEQK-Norm
Open Qwen3-VL in the visualizer →

Want the definitions behind each stop? The glossary explains all 39 concepts, and the catalog has interactive graphs for 2,300+ models.

AlsoHow we got to DeepSeek-V4-Pro · the LLM timeline →AlsoHow we got to FLUX.2 · the image generation timeline →